TLDR:

pip install --target path/to/folder package-to-install

(and run this from an AWS Lambda base runtime image)

Why do you need to do this?

AWS Lambda requires you to create a zipped folder that includes:

• A file to run your code (the “entry point”)
• All libraries needed to execute the code

The entry point is just a file with code that will accept the parameters Lambda gives it and run your code. For example, your entry point file could be called main.py and might look like this:

def hello_world(event, context):
    print("Hello world!")

The actual names of the entry point file and function does not matter. They can both be configured in Lambda. In this example, you would configure Lambda to run main.hello_world

What happens if I don’t?

Here’s what happens if you used a third party package like requests but didn’t include the library in the zip file for Lambda:

import requests

def hello_world(event, context):
    resp = requests.get("example.com")
    print(resp)

Runtime.ImportModuleError: Unable to import module 'main': No module named 'requests'

Install Example

To fix this, you would need to install the requests pacakge to the directory that contains main.py - your entry point file. For example:

pip install --target ./lambda_example requests

Now, if you list the lambda_example directory you’ll see the requests folder/module and it’s dependencies:

ls -a ./lambda_example:

bin                                     charset_normalizer-3.3.2.dist-info      requests
certifi                                 idna                                    requests-2.31.0.dist-info
certifi-2023.11.17.dist-info            idna-3.6.dist-info                      urllib3
charset_normalizer                      main.py                                 urllib3-2.1.0.dist-info

Plot-twist: Lambda uses a specific base image for a runtime

TLDR: Install the needed libraries from a Lambda runtime base image from here

Third party libraries installed on your mac-os won’t always work on lambda’s runtime. AWS Lambdas use specific runtimes based on a specific architecture.

To fix this, you’ll need to:

• pull and build the runtime specific docker image for your Lambda. You can find these here (choose runtime from branch name): github.com/aws/aws-lambda-base-images

• Run the above pip commands from that docker container:

  docker run -v /local_directory:/working -it --rm lambda3.9:local

The -v parameter sets up a volume to the path you choose locally. Make this where you want to install the libs to.

Basically, you need to run a container that has the same runtime AWS Lambda will have and install 3rd party libraries there. Then you can upload those with your Lambda zip file. Now, you won’t have compatibility issues.