x

[eBook Presented by AWS & Thundra] Mastering Observability on the Cloud 📖 Download:

Deploying Serverless Applications with the AWS CDK

May 14, 2020

 

Deploying Serverless Applications with the AWS CDK

In 2018 AWS released the Cloud Development Kit (CDK), an “infrastructure as code” (IaC) framework that simplifies work with CloudFormation (CFN).

CFN uses a declarative programming model based on JSON or YAML files to define resources, like virtual machines and databases, in the cloud. Many developers aren’t used to declarative programming, so the idea behind the CDK was to allow developers to define their cloud resources with an imperative programming language they already use to build other software, like JavaScript or Python. 

In this article you will learn the basics of using the CDK by implementing and deploying a sample serverless application.

What is the CDK?

The CDK is a command line application that takes IaC in an imperative programming language and creates CFN templates from it. Currently supported languages are JavaScript, TypeScript, Python, Java, and C#.

The CDK is a client-side abstraction that CFN doesn’t know or see in any way. The CDK generates CFN templates, and CFN will see only the finished product and work on the generated template.

While allowing developers to use a language other than YAML or JSON to create their IaC, it also comes with an abstraction called “construct.” What CFN calls resources the CDK calls constructs.

The main differences between CDK constructs and CFN resources are:

  • CDK constructs use pre-configured CFN resources to enable developers to use AWS best practices. This allows even beginners to adhere to best practices while building their infrastructure and frees them from the need to think about those practices.

  • CDK constructs can wrap one or more CFN resources, allowing developers to create higher-level services from different AWS resources with less code.

Creating a File Processor with S3 and Lambda

To illustrate CDK features, you will build a small sample serverless application that uses two S3 buckets and a Lambda function. The Lambda function will trigger when a file is uploaded to the first bucket, then process it in some way, and then upload it to the second bucket.

CDK Installation

The CDK is an NPM package, so to install it you need to install NPM, which comes with Node.js.

Once you’ve set up NPM, you can install the CDK with the following command:

$ npm install -g aws-cdk

Initializing a CDK Project

To initialize a CDK project, you’ll need to create a new directory and run the CDK with the init command. In this example, you’ll create a CDK project that uses JavaScript.

$ mkdir file-processor
$ cd file-processor
$ cdk init --language javascript

Installing Supporting NPM Packages

After the CDK has initialized the project, you’ll have to install two constructs. As mentioned before, constructs wrap CFN resources. For JavaScript they are implemented as NPM packages, so if you want to create S3 buckets and Lambda functions, you’ll have to install three packages with the following command:

$ npm install \
  @aws-cdk/aws-s3 \
  @aws-cdk/aws-lambda \
  @aws-cdk/aws-lambda-event-sources \
  @thundra/core

The first two are the construct packages for S3 and Lambda, and the third one helps you define event sources to trigger your Lambda function.

With the @thundra/core package, you can instrument your Lambda function and get insights about it.

Implementing the CDK Stack and the Lambda Function

As with CFN, you have to implement a “stack” as the root of your IaC application. For this, you’ll need to replace the content of your lib/file-processor-stack.js file with the following code:

const path = require("path");
const cdk = require("@aws-cdk/core");
const s3 = require("@aws-cdk/aws-s3");
const lambda = require("@aws-cdk/aws-lambda");
const lambdaEventSources = require("@aws-cdk/aws-lambda-event-sources");

const THUNDRA_API_KEY = "<YOUR_THUNDRA_API_KEY>";
const THUNDRA_AWS_ACCOUNT_NO = 269863060030;
const THUNDRA_LAYER_VERSION = 48;
const THUNDRA_LAYER_ARN =
  "arn:aws:lambda:" +
  process.env.CDK_DEFAULT_REGION +
  ":" +
  THUNDRA_AWS_ACCOUNT_NO +
  ":layer:thundra-lambda-node-layer:" +
  THUNDRA_LAYER_VERSION;

class FileProcessorStack extends cdk.Stack {
  constructor(scope, id, props) {
    super(scope, id, props);
    const sourceBucket = new s3.Bucket(
      this,
      "SourceBucket"
    );
    const targetBucket = new s3.Bucket(
      this,
      "TargetBucket"
    );
    const thundraLayer = lambda.LayerVersion.fromLayerVersionArn(
      this,
      "ThundraLayer",
      THUNDRA_LAYER_ARN
    );
    const processorFunction = new lambda.Function(
      this,
      "Processor",
      {
        runtime: lambda.Runtime.NODEJS_12_X,
        handler: "index.handler",
        environment: {
          TARGET_BUCKET_NAME: targetBucket.bucketName,
          thundra_apiKey: THUNDRA_API_KEY,
        },
        layers: [thundraLayer],
    	   code: lambda.Code.fromAsset(
          path.join(__dirname, "processor-function")
        ),
      }
    );
    sourceBucket.grantRead(processorFunction);
    targetBucket.grantPut(processorFunction);
    const uploadEvent = new lambdaEventSources.S3EventSource(
      sourceBucket,
      { events: [s3.EventType.OBJECT_CREATED] }
    );
    processorFunction.addEventSource(uploadEvent);
  }
}
module.exports = { FileProcessorStack };

Let’s go through this code step by step.

First, import all the required packages we installed before.

Then, set the Thundra API key, so it can later be passed as an environment variable to the Lambda function. The key can be found in the Thundra web console.

Next, create the stack that will hold all our constructs: Create the two S3 buckets, then create the Lambda function, then grant required S3 permissions to the Lambda function, and last, create an upload event for the source bucket and link it to the Lambda function.

The Lambda function will get the source bucket name in the object creation event, but not the target bucket name. This is why you need to set the target bucket name as an environment variable for the Lambda function.

The next step is to implement the Lambda function. For this, you have to create a new file, lib/processor-function/index.js, and insert the following code:


const AWS = require("aws-sdk");
const thundra = require("@thundra/core")();
const s3 = new AWS.S3();
exports.handler = thundra(async ({ Records }) => {
 for (let record of Records) {
   const { bucket, object } = record.s3;
   const { Body } = await s3
    .getObject({
    Bucket: bucket.name,
    Key: object.key
  })
    .promise();
   const processedData = Body.toString("utf-8").toUpperCase();
   await s3
    .putObject({
     Bucket: process.env.TARGET_BUCKET_NAME,
     Key: object.key,
     Body: processedData
   })
   .promise();
}
});

The code uses the AWS SDK for JavaScript, which is pre-installed in the Node.js Lambda runtime environment.

Next you define an asynchronous function, which the Lambda service calls when an event occurs. The function is also wrapped by the Thundra library, which allows you to debug it via the Thundra serverless monitoring service if anything goes wrong. It uses an API key that was passed to the Lambda function as an environment variable by the CDK.

The only event you’ve defined is the S3 object creation event, so you can be sure your event will come with a Records array you can loop over to get every file that was created in that event.

Next you do three steps. First, use the event data to get your file content from your S3 source bucket. Then, convert the data to upper case. This is your processing. And finally, save the processed data back to the target bucket.

In the CDK stack, you saved the name for the target bucket in an environment variable for the Lambda function, so you can now read it from there.

Setting AWS Credentials

First, set your AWS account credentials via environment variables.

Next, find the access key ID and the secret access key in your AWS IAM console.

Here are the commands for Windows:

$ set AWS_ACCESS_KEY_ID=<YOUR_KEY>
$ set AWS_SECRET_ACCESS_KEY =<YOUR_SECRET>

These are the commands for Linux and macOS:

$ export AWS_ACCESS_KEY_ID=<YOUR_KEY>
$ export AWS_SECRET_ACCESS_KEY =<YOUR_SECRET>

Deploying the CDK Application

If you’d like to look at the CFN template that CDK will generate, use the following command:

$ cdk synth

The actual CDK deployment command will do this implicitly before calling the CFN service. To deploy, use the following command:

  $ cdk bootstrap
  $ cdk deploy

The bootstrap command is needed to set up an S3 bucket that will hold your Lambda function’s source code.

Processing a File

Now that everything is implemented and deployed to AWS, we can finally upload a file to our S3 source bucket!

You can upload text files with the AWS Console to your source bucket. The names of your buckets should follow this pattern:

fileprocessorstack-sourcebucket<RANDOM_STRING>
fileprocessorstack-targetbucket<RANDOM_STRING>

After you’ve uploaded a file into the source bucket, your Lambda function should trigger automatically. Process it and save the processed content into the target bucket.

Why You Should Use the CDK

The CDK lets you write infrastructure as code in an imperative way, which is easier for some developers. Unlike CFN, the CDK allows you to use many different languages to define a CFN stack, such as JavaScript, TypeScript, Python, Java, and C#.

The CDK comes with constructs that wrap one or more CFN resources, enabling developers to use AWS recommended resource configuration best practices right from the start. Since one CDK construct can wrap multiple CFN resources, you can abstract a complex service with just one CDK construct.