AWS Lambda with Serverless

If you are a dev interested in how to implement a project using AWS Lambda, this article could be interesting for you. Let me show you the basics of how to create, build and deploy a project using AWS Lambda with Serverless.

Serverless is a framework that allow us to deploy our project to AWS platform with a simple configuration in a YML file. This is infrastructure-as-a-code, but what does it mean? It means that we’ll define our infrastructure in a YML file and we’ll keep it in the repository with the rest of the code, we don’t need to make changes in AWS console at all. Serverless is in charge of create all the infrastructure in AWS for us.

Let’s get start with an application that receives a HTTP request and trigger an asynchronous process like send an email or perform a validation.
We can implement this app using a Lambda, a SQS and Api Gateway:

Architecture

Requirements:
- Python 3.9 or latest
- Pip3
- Python virtual environment
- NodeJS 16 or latest
- AWS CLI installed and configured with an AWS account

Install Serverless and start project from scratch

Create package.json file, add Serverless dependency, then run
$ npm install.

Running $ serveless, the process will ask about runtime language and project type, select “Python and HTTP API” and set a name for the project.
Finally select “No” to register and deploy questions.

This will create the startup of the project, a directory with the project name with 3 files inside: serverless.yml, handler.py, README.md.

Let’s focus on the serverless.yml:

service: aws-python-http-api-project
frameworkVersion: '3'

provider:
  name: aws
  runtime: python3.9

functions:
  hello:
    handler: handler.hello
    events:
      - httpApi:
          path: /
          method: get

This is the main file of the project, here we define the infrastructure for the application, runtime language, resources, and permissions.

service is the project name.

frameworkVersion is the version of Serverless.

provider grouped config for the underline platform:
- name, AWS platform
- runtime, lambda’s runtime language

functions has the declaration of the lambdas. It contains name, handler and invocation events. The template add a lambda called “hello”, with a handler method hello() in a file handler.py, invoked by a GET request to / path. We’ll modify it soon.

handler.py contains the python code that will run when the lambda is invoked:

import json


def hello(event, context):
    body = {
        "message": "Go Serverless v3.0! Your function executed successfully!",
        "input": event,
    }

    response = {
        "statusCode": 200, 
        "body": json.dumps(body)
    }

    return response

On a GET request to /, the lambda responds with a 200 code with a JSON body that contains “message” and “input” fields.

Architecture implementation

Now we want the lambda to receive an entity id from a HTTP request and push it to a SQS. Then a consumer could read from the SQS and process the message (not covered in this article).

serverless.yml file was modified to include the SQS, change lambda’s name and handler method:

service: aws-python-http-api-project
frameworkVersion: '3'

provider:
  name: aws
  runtime: python3.9
  stage: ${opt:stage, 'dev'}

functions:
  publisher:
    handler: handler.process
    events:
      - httpApi:
          path: /
          method: get
          
custom:
  mySqsName: ${self:provider.stage}-my-sqs
          
resources:
  Resources:
    MySQS:
      Type: AWS::SQS::Queue
      Properties:
        QueueName: ${self:custom.mySqsName}
        VisibilityTimeout: 30

resources section contains the declaration of the resources for the project such as SQS, Dynamo tables, SNS, etc.
I added a SQS resource for our application, setting the resource name as “MySQS” and resource Type as “AWS::SQS::Queue”.
Also I defined 2 properties, QueueName and VisivilityTimeout.

I want to leverage QueueName to explain you how reference other properties from the same file.
QueueName is the name will the SQS have in AWS platform, but what if we want to set the queue name depends on the environment? like, dev-my-sqs and prod-my-sqs.
The value of QueueName refers to custom.mySqsName property in the same file, then custom.mySqsName is building the name appending provider.stage + “-my-sqs” getting dev-my-sqs.

Adding logic to the handler method to process the request and interact with the SQS:

import json
import boto3

# client for sqs
sqs_client = boto3.client('sqs', region_name='us-east-1')

def process(event, context):
    # Read param from request
    entity_id = event.get('queryStringParameters', {}).get('entityId')
    
    # Get queue info
    queue_url_info = sqs_client.get_queue_url(QueueName='dev-my-sqs')
    
    # Send message to sqs    
    sqs_client.send_message(QueueUrl=queue_url_info.get('QueueUrl'), 
                      MessageBody=entity_id)

    body = {
        "message": "Message sent to SQS",
        "input": entity_id,
    }

    response = {
        "statusCode": 200, 
        "body": json.dumps(body)
    }

    return response

At this point we have our Lambda code ready to be deployed in AWS.

To do that we have to create a package with our code, python dependencies and AWS configuration files, upload to S3 and finally deploy in AWS.
Fortunately Serverless will do it for us.

For this we have to install “serverless-python-requirements” plugin, adding it to package.json file and configuring it in serverless.yml file:

...

plugins:
    - serverless-python-requirements

package:
  excludeDevDependencies: true
  individually: true
  patterns:
    - '!node_modules/**'
    - '!.idea/**'
    - '!.mypy_cache/**'
    - '!venv/**'
    - '!__pycache__/**'
    - '!.vscode/**'

# For not linux OS
# it will package python dependencies in a linux docker container
# docker installation required
custom:
 ...

  pythonRequirements:
    dockerizePip: non-linux

Include Python dependencies in requirements.txt:

boto3~=1.17.6

Then run
$ pip3 install -r requirements.txt

Finally we can deploy our project to AWS:
$ serverless deploy --stage dev

Serveless framework will create a package, upload to AWS and deploy the project infrastructure for us.

Congratulations 🎉 we have our project deployed in AWS platform.

This means now we can invoke our code doing
GET <baseURL>/?entityId=12345.

Note: baseURL is given when deploy command ends.

If we want to bring our project to a production environment, we can just change the stage to “prod”:
$ serverless deploy --stage prod

I hope this article have been useful for you to understand how Serverless can help you to structure your AWS Lambda project, making it easy to package, deploy and provisioning. Beside that, infrastructure as a code is helpful to define and track changes in the server infrastructure.

You can see complete project at Github.

Please left a comment if you have any question.

Thanks