Pandas in AWS lambda gives numpy error is a Python error which occurs when you try to use pandas and numpy in a AWS Lambda and you do it the wrong way.
In this article I am going to solve the error while I try explaining why the error is popping up in the first place, I will also introduce some solutions which worked for other developers and we will see if those solutions can solve the error in your unique situation.
Explaining the Error : Pandas in AWS lambda gives numpy error
Pandas in AWS lambda gives numpy error is a Python error which occurs when you try to use pandas and numpy in a AWS Lambda and you do it the wrong way.
The error message should look like the error in the example bellow, make sure you have the same error message in order to avoid confusion.
#
Unable to import module 'lambda_function': Missing required dependencies ['numpy']
#
Bellow I make my best attempt at solving the error and present multiple possible options to get rid of it.
Solution : work with Lambda layers in order to use pandas and numpy in a AWS Lambda
First you need to have Docker ready to go, start by creating a fresh directory.
Create a requirements text file named requirements.txt.
Add the following lines to it.
#
pandas==<the last version> # replace <the last version> with newest version you can find
pytz==<the last version> # replace <the last version> with newest version you can find
#
Create a .sh file named get_layer_packages.sh
Add the following code to it.
#
#!/bin/bash
export PKG_DIR="python"
rm -rf ${PKG_DIR} && mkdir -p ${PKG_DIR}
docker run --rm -v $(pwd):/foo -w /foo lambci/lambda:build-python3.6 \
pip install -r requirements.txt --no-deps -t ${PKG_DIR}
#
Stay in the directory, run the command bellow
#
chmod +x get_layer_packages.sh
#
Followed by this command
#
./get_layer_packages.sh
#
And finally, this command
#
zip -r pandas.zip .
#
Create an S3 bucket on amazon aws and upload the layer by running the command below.
#
aws lambda publish-layer-version --layer-name pandas-layer --description "Description of your layer"
--content S3Bucket=<bucket name>,S3Key=<layer-name>.zip
--compatible-runtimes python3.6 python3.7
#
Use the inline editor in Lambda console and add your code there
Select the version of pandas-layer by doing Layers> Add a layer> Compatible layers and add the AWSLambda-Python36-SciPy1x layer.
Now, the error should be gone forever. I hope this has solved your problem or at least guided you in a good direction.
The solutions above should be enough to solve the problem, if you like our effort make sure to consider donating to our Kofi account, there is a red button that you can use if you are feeling generous.
Summing-up
This is the end of our article, I hope the solutions I presented worked for you, Learning Python is a fun journey, do not let the errors discourage you.
Keep coding and cheers. If you want to learn more about Python, please check out the Python Documentation : https://docs.python.org/3/