Create an AWS Lambda using Python with dependencies

Create an AWS Lambda using Python with dependencies

Whenever you have a Python code/script which uses modules, that are not included by default in python and you want to use it as an AWS Lambda function, you are left with two choices

Create a lambda layer, which will contain all the required modules, and connect this layer to your lambda function.
Create a .zip file by zipping the python script and the modules together, upload this .zip file to the AWS lambda function & run.

Before we explore these options, let’s complete the prerequisite first

Prerequisite

Before you are able to run & test the lambda, we are going to create a small script, which will use python’s request module and hit an open-source API which will return my Public IP Address.

Let’s begin with creating a new directory on the terminal, create a new virtual environment(using virtualenvwrapper) and install the requests module using pip

mkdir my_ip_address
cd my_ip_address
mkvirtualenv my_ip --python=python3
workon my_ip
pip3 install requests

Let’s create the python script

touch my_ip_lambda.py
vim my_ip_lambda.py

And the script code looks like this

You can create the lambda function by logging into the AWS console & Selecting the Lambda service. After that, you need to create a lambda function (author from scratch) & select Python 3.9 as the runtime environment.

After creating this lambda if you try to paste your script’s code in the lambda code editor and will try to run it, it will raise an error because it can not find the required requests module, and that’s the problem we are trying to address over here.

Once you are done with the prerequisite for the example, let’s visit both approaches one by one.

Approach 1): Create a layer & connect it with your AWS Lambda function

Install the library locally

In your terminal, create a directory named python

mkdir python
cd python

After this step, install your required library locally in the current directory

pip3 install requests -t .

This structure is required because AWS Lambda layers need to follow a specific structure for each runtime. You can check this for more details.

Creating a .zip file

Let’s zip this directory by using this code on the parent directory

zip -r <name_for_your_layer>.zip .

NOTE: Your .zip file must have all the content inside either in directory python or python/lib/python<your lambda runtime python version>/site-packages

Create a lambda layer on the AWS console

Once all the given steps are completed, you need to create an AWS Lambda layer. For this navigate to Lambda > Layers and then click on Create Layer button

After this upload your <name_for_your_layer>.zip file & create a new layer. Once the layer is created, you need to connect this layer to your lambda function and you are good to go.

Approach 2): Combine Python code & Dependencies in a single .zip file

Bundling the lambda code with modules

Back in the terminal, inside our project directory, we will create a .zip file with all the dependencies/modules and then will add the lambda code to the same .zip file.

cd ~/.virtualenvs/my_ip/lib/python3.9/site-packages
zip -r9 ${OLDPWD}/function.zip .
cd $OLDPWD
zip -g function.zip my_ip_lambda.py

Uploading this bundle on AWS Lambda

Back in the AWS console, in the lambda function’s code section after clicking on the “Upload From” dropdown select the .zip file. Upload your .zip file & save it.