{"id":25,"date":"2023-01-24T18:48:00","date_gmt":"2023-01-24T18:48:00","guid":{"rendered":"http:\/\/localhost:8080\/?p=25"},"modified":"2026-08-22T18:55:02","modified_gmt":"2026-08-22T18:55:02","slug":"lambda-with-oracle-layers","status":"publish","type":"post","link":"https:\/\/jamesmonek.com\/?p=25","title":{"rendered":"Lambda with Oracle Layers"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">Overview<a href=\"https:\/\/jamesmonek.com\/posts\/lambda-with-oracle-layers\/#overview\"><\/a><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Recently, I came across a need to finally use Lambda in AWS. I was looking to query an Oracle database and export the Oracle table to a CSV file. While using Python I realized the Oracle client is not available in Lambda so I needed a way to add libraries and dependancies in order to execute my code. Lambda layers allows you to do this. You can add several layers to your Lambda function. This article documents how to create the Oracle Layer for Python in Lambda with the required libraries.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The main limitation that I discovered using Lambda layers is the size limit of 250 megs for all your code and libraries. Fortunately, I was just under this limitation but there are several ways to get around this if you run into this problem. One is to break your code into various micro services. The other is to to use EFS and place all of your libraries in the EFS volume and mount the volume on your Lambda function.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Building the Oracle Layer<a href=\"https:\/\/jamesmonek.com\/posts\/lambda-with-oracle-layers\/#building-the-oracle-layer\"><\/a><\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Optional, but preferred, commission an EC2 instance so you can start from a clean environment.<\/li>\n\n\n\n<li>Verify the Python version<\/li>\n<\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code>python3 --version<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Create your project directory<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mkdir project<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Create a directory called python<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mkdir python<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Make library directory<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>mkdir lib<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Install cx_Oracle Python library<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>pip3 install cx_Oracle -t python\/<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Download the latest Oracle Client from\u00a0<a href=\"https:\/\/www.oracle.com\/database\/technologies\/instant-client\/linux-x86-64-downloads.html\">https:\/\/www.oracle.com\/database\/technologies\/instant-client\/linux-x86-64-downloads.html<\/a><\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>wget https:\/\/download.oracle.com\/otn_software\/linux\/instantclient\/218000\/instantclient-basic-linux.x64-21.8.0.0.0dbru.zip -O oracli.zip<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Unzip the Oracle Client into the library folder<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>unzip -j oracli.zip -d lib\/<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Ensure Libaio is insalled on the EC2 instance (it should be)<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>sudo yum install libaio -y<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Copy Libaio to library<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>cp \/lib64\/libaio.so.1 lib\/libaio.so.1<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Zip up all your libraries to be used for the Lambda Layer<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>zip -y -r oracletable.zip python\/ lib\/<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Copy your zipped library file to a S3 bucket so that the Lambda function can access it.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Adding Layer into Lambda<a href=\"https:\/\/jamesmonek.com\/posts\/lambda-with-oracle-layers\/#adding-layer-into-lambda\"><\/a><\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Copy the url to be use when creating the layer<\/li>\n\n\n\n<li>Go to lambda and create a layer using\u00a0<a href=\"https:\/\/docs.aws.amazon.com\/lambda\/latest\/dg\/chapter-layers.html\">AWS Lambda Layer documentation<\/a>.<\/li>\n\n\n\n<li>Create the Lambda function using\u00a0<a href=\"https:\/\/docs.aws.amazon.com\/lambda\/latest\/dg\/getting-started.html\">AWS Lambda Documentation<\/a>.<\/li>\n\n\n\n<li>Add your code<\/li>\n\n\n\n<li>Add the layer your created<\/li>\n\n\n\n<li>Run the Lambda function<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Using CloudFormations<a href=\"https:\/\/jamesmonek.com\/posts\/lambda-with-oracle-layers\/#using-cloudformations\"><\/a><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Below is an example of the CloudFormation code to create a Lambda Layer and Lambda function.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Note: Amazon Python Power Tools Layer documentation is at https:\/\/awslabs.github.io\/aws-lambda-powertools-python\/2.6.0\/<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Role:\n  Type: 'AWS::IAM::Role'\n  Properties:\n    RoleName: lambda-role-name\n    AssumeRolePolicyDocument:\n      Version: \"2012-10-17\"\n      Statement:\n        - Effect: Allow\n          Principal:\n            Service:\n              - lambda.amazonaws.com\n              - s3.amazonaws.com\n          Action:\n            - 'sts:AssumeRole'\n    Path: \/\n    Policies:\n      - PolicyName: lambda-policy\n        PolicyDocument:\n          Version: \"2012-10-17\"\n          Statement:\n# Add access to secrets manager to give access to Lambda function to use\n            - Effect: Allow\n              Action: \n                - secretsmanager:GetResourcePolicy\n                - secretsmanager:GetSecretValue\n              Resource: 'arn:aws:secretsmanager:&lt;region>:&lt;account>:secret:&lt;secretname>'\n            - Effect: \"Allow\"\n              Action:\n                - s3:ListBucket\n              Resource:\n                - !Sub 'arn:aws:s3:::&lt;S3 Bucket>'\n            - Effect: \"Allow\"\n              Action:\n                - s3:GetObject\n                - s3:PutObject\n                - s3:PutObjectAcl\n              Resource:\n                - !Sub 'arn:aws:s3:::&lt;S3 Bucket>\/*'\n                - !Sub 'arn:aws:s3:::&lt;S3 Bucket>\/*'\n    ManagedPolicyArns:\n      - 'arn:aws:iam::aws:policy\/service-role\/AWSLambdaBasicExecutionRole'\n      - 'arn:aws:iam::aws:policy\/service-role\/AWSLambdaVPCAccessExecutionRole'\n\n# Create the Oracle Layer for Python\nOrcaleLayer:\n  Type: AWS::Lambda::LayerVersion\n  Properties:\n    CompatibleRuntimes:\n      - python3.7\n    Content:\n      S3Bucket: &lt;S3 Lambda Bucket with code>\n      S3Key: &lt;layer ip file>\n    Description: \"Oracle Layer\"\n    LayerName: oracle-layer\n\n# Create Lambda Function\nOracleFunction:\n  Type: AWS::Lambda::Function\n  Properties:\n    FunctionName: oracle-lambda\n    Handler: oraclelambda.lambda_handler\n    Layers:\n# Oracle Client Library Layer\n      - !Ref OracleLayer\n# Add Python Power Tools\n      - 'arn:aws:lambda:us-east-1:017000801446:layer:AWSLambdaPowertoolsPythonV2:13'\n    Role:\n      Fn::GetAtt:\n        - Role\n        - Arn\n    Environment:\n#Pass Secret Name, Target Bucket, and S3 Prefix to the Python code\n      Variables:\n        SecretName: &lt;secret name>\n        TargetBucket: &lt;target bucket>\n        S3Prefix: &lt;target prefix>\n# Location of the Python Code\n    Code:\n      S3Bucket: &lt;S3 Lambda Bucket with code>\n      S3Key: &lt;pthon code zip file>\n    Runtime: python3.7\n    Timeout: 90\n    TracingConfig:\n      Mode: Active\n    VpcConfig:\n      SecurityGroupIds: &lt;security group id>\n        - !Ref SecurityGroupId\n      SubnetIds:\n        - &lt;subnet 1>\n        - &lt;subnet 2><\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">What\u2019s next?<a href=\"https:\/\/jamesmonek.com\/posts\/lambda-with-oracle-layers\/#whats-next\"><\/a><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Currently, the process for updating the layers and code is very manual. I\u2019m looking to find ways to automate getting the CloudFormation templates and Lambda to get updated when new changes are made<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Overview Recently, I came across a need to finally use Lambda in AWS. I was looking to query an Oracle database and export the Oracle table to a CSV file. While using Python I realized the Oracle client is not available in Lambda so I needed a way to add libraries and dependancies in order [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":26,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-25","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/jamesmonek.com\/index.php?rest_route=\/wp\/v2\/posts\/25","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jamesmonek.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jamesmonek.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jamesmonek.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jamesmonek.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=25"}],"version-history":[{"count":1,"href":"https:\/\/jamesmonek.com\/index.php?rest_route=\/wp\/v2\/posts\/25\/revisions"}],"predecessor-version":[{"id":27,"href":"https:\/\/jamesmonek.com\/index.php?rest_route=\/wp\/v2\/posts\/25\/revisions\/27"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/jamesmonek.com\/index.php?rest_route=\/wp\/v2\/media\/26"}],"wp:attachment":[{"href":"https:\/\/jamesmonek.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=25"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jamesmonek.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=25"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jamesmonek.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=25"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}