7 minutes read

POSTED Sep, 2020 dot IN Serverless

How to build a serverless API from scratch

Serkan Özal

Written by Serkan Özal


Founder and CTO of Thundra

 X


API is the most important part of any customer facing web application since it’s directly tied up to the satisfaction of customers and the health of the business. Among numerous alternative ways of creating an API, serverless is one approach gaining popularity during the last few years because of its cost efficiency, scalability, and relative simplicity. AWS, as a leading serverless provider, has made a huge contribution to the world of serverless development, and in this article we’re aiming to understand general API implementation concepts using AWS Lambda and other AWS services. In a long-running video series “Happy Little APIs”, Serverless Developer Advocate Eric Johnson goes over the different ways of building an API with Lambda and API-Gateway. In this blog post, we’ll go over the real-life example and provide a guide to create a serverless API.

Why AWS Lambda?

AWS Lambda is our favorite AWS service here at Thundra that gets executed when triggered with various events in  the application. Trigger resources for AWS Lambda functions could be HTTP calls through API-Gateway; events from other AWS services like S3, Kinesis, or SNS; or just recurrent scheduled events. Application development teams should not be worried if there will be one or one million triggering events at the same time as AWS Lambda scales automatically to infinity (almost). Thanks to the pay-as-you-go payment model, you are charged only for the total execution time of your functions and do not pay for idle time. Of course, like any other service, Lambda has limits and is sometimes not suitable for certain tasks, such as very long-running jobs, heavy computing jobs, or processes that require control over the execution environment. However, AWS Lambda usually works perfectly for implementing APIs as we’ll cover now.

The Role of API Gateway

AWS API Gateway is a very easy-to-use and straightforward service that lets developers create and manage HTTP endpoints, map them to particular AWS resources. Besides, it has advanced capabilities that allows application teams to configure custom domains, authorizing mechanisms, caching, and other features for the APIs. API Gateway is the fundamental part of serverless API because it is responsible for the connection between a defined API and the function handling requests to that API.

HTTP APIs

API-Gateway is designed to respond to the needs of a wide range of applications that use different compute sources as their backend services. However, it also creates some complexity for serverless developers because it goes far from being simple. Probably for this reason, AWS announced the new HTTP APIs, a lite version of API Gateway, which dramatically simplifies the developer experience and provides better performance and lower costs for serverless APIs in late 2019. Although it is simple, HTTP APIs still contain important features like configuring CORS for all endpoints, JWT integration, custom domains, and VPC connections.

Understanding Serverless API Concepts

To demonstrate the main concepts of a serverless API, we’ll build a simplistic virtual whiteboard application. There will only be two basic endpoints POST for writing messages on a whiteboard, and GET for fetching the three most-recent messages. We’ll use Amazon DynamoDB as the serverless database component and we’ll build our infrastructure using AWS Serverless Application Model.

Amazon DynamoDB

We will make our project completely serverless by using Amazon DynamoDB for storing messages, as this database corresponds to serverless principles, is easy to use, and offers a pay-per-request model, which is really cost effective. DynamoDB is a NoSQL key-value database offered by AWS, where your data is stored across AWS servers and fully managed by Amazon.

AWS Serverless Application Model as IaC (Infrastructure as Code) solution

In order to continue further implementation, you’ll need an AWS account and AWS Serverless Application Model (SAM) installed and configured. SAM is a tool for creating, updating, and managing serverless applications and all the resources needed for the application to operate. AWS SAM lets you automate the process of creating required services for your applications instead of creating them one by one on AWS console. All you need to do is to describe all the things needed in the special template file.

After you’ve installed the CLI, navigate to the directory you are going to work in and run this command:

$ sam init -r nodejs12.x -n whiteboard

Figure 1: Initializing new project

Select the first option, then select “Quick Start from Scratch.” This will create a “whiteboard” directory with a minimum of setup files inside.

Define the Required Resources Needed

First, open the template.yml file and remove everything below the “Resources” section. Before moving to the API itself, let’s create secondary resources first. Define a DynamoDB table where messages will be stored:

Resources:

 BoardMessagesTable:
     Type: AWS::DynamoDB::Table
     Properties:

        TableName: board-messages-table
       AttributeDefinitions:
         - AttributeName:
partKey
           AttributeType:
S
         - AttributeName:
createdAt
           AttributeType:
N
       KeySchema:
         - AttributeName:
partKey
           KeyType:
HASH
         - AttributeName:
createdAt
           KeyType:
RANGE
       ProvisionedThroughput:
         ReadCapacityUnits: 5
         WriteCapacityUnits: 5

Figure 2: Declaring DynamoDB table

The code above will tell AWS to create a DynamoDB table, where attribute “partKey” will be a partition key that is the same for all records and “createdAt” will be a range key, allowing further sorting by timestamp. We may also add other keys and values into the records, but you are not required to define those.

Now, in the same file, just below the previous definition, declare the HTTP API to which all future endpoints and functions will be related.

  BoardHttpApi:
   Type: AWS::Serverless::HttpApi
   Properties:
     
StageName: Test
     CorsConfiguration:
True

Figure 3: Declaring HTTP API

The definition is very small and simple, since we just included the stage name and CORS configuration, which are not actually required either. This illustrates how simple and clean API creation can be. However, there are many possible properties to add, such as a reference to authorization function, definition of the domain to use, logging settings, and others.

Define API Handlers Functions

Finally, when we have the API defined, let’s also declare two functions connected to its particular endpoints.

  PostMessageFunction:
     Type: AWS::Serverless::Function
     Properties:
       Handler:
src/handlers/postMessage.handler
       Runtime:
nodejs12.x
       MemorySize: 128
       Timeout: 5
       Events:
         PostMessage:
           Type:
HttpApi
           Properties:
             ApiId:
!Ref BoardHttpApi
             Method:
POST
             Path:
/messages

        Policies:

          - AmazonDynamoDBFullAccess

 GetMessagesFunction:
     Type: AWS::Serverless::Function
     Properties:
       Handler:
src/handlers/getMessages.handler
       Runtime:
nodejs12.x
       MemorySize: 128
       Timeout: 5
       Events:
         GetMessages:
           Type:
HttpApi
           Properties:
             ApiId:
!Ref BoardHttpApi
             Method:
GET
             Path:
/messages

        Policies:

          - AmazonDynamoDBFullAccess

Figure 4: Declaring handlers for POST and GET requests

The above code is quite self-descriptive: two functions, one of which will be invoked upon a POST request to the /messages path, and the other of which will be invoked upon a GET request to the same path. Both functions have a capacity of 128 MB RAM and a five-second timeout. The functions’ code is found in the postMessage.js and getMessage.js files under the /src/handlers/ directory. We are going to create those right now. (Note that we’ve provided full access to the DynamoDB in the “Policies” section of each function, just to make things easier.) In a real project you should consider providing more granular access.

Coding the Functions

Navigate to the /src/handlers/ directory and create files there with the following content:

postMessage.js

const AWS = require('aws-sdk');
const dynamodb = new AWS.DynamoDB();

exports.handler =
async (event) => {
 
const { body } = event;
 
try {
   
const { author, text } = JSON.parse(body);

    if (!author || !text) {

      return {

        statusCode: 403,

        body: 'author and text are required!'

      }

    }

   
await dynamodb.putItem({
     TableName:
'board-messages-table',
     Item: {
       msgId: { S:
'board' },
       author: { S: author },
       text: { S: text },
       createdAt: { N: String(Date.now()) }
// still expects string!
     }
   }).promise();
   
return {
      statusCode: 200,
      body:
'Message posted on board!',
   }
 }
catch (err) {
   
return {
      statusCode: 500,
      body:
'Something went wrong :(',
   }
 }
};

Figure 5: POST request handler’s code

This function will run in response to POST requests and will parse the author and text of the message from the request body and save that data into the database. It also fills the “partKey” attribute with the same value for all records. Although usually this is not a good practice, it is completely fine for this example, as it allows you to sort by range key among all items with the same partition key. Note that DynamoDB always expects string data to be saved, even if the type of attribute is number, but in the end, under the hood it will consider it properly.

getMessages.js

const AWS = require('aws-sdk');
const dynamodb = new AWS.DynamoDB();

exports.handler =
async () => {
 
try {
   
const result = await dynamodb.query({
     TableName:
'board-messages-table',
     KeyConditionExpression:
'partKey = :partKey',
     ScanIndexForward:
false,
     Limit: 3,
     ExpressionAttributeValues: {
':partKey': { S: 'board'}}
   }).promise();

   
return {
     statusCode: 200,
     headers: {
       
'Content-Type': 'application/json',
     },
     body: JSON.stringify(result.Items),
   }
 }
catch (err) {
   console.log(err);
   
return {
     statusCode: 500,
     body:
'Something went wrong :(',
   }
 }
};

Figure 6: GET request handler’s code

In this function we first get records with “partKey” equal to “board,” then use “ScanIndexForward” set to “false” to sort messages so that the most recent is first, and finally we use the “Limit” property to limit results to three messages.

Deployment

Deployment with AWS SAM is easy and can be done with a single command and a few inputs. Navigate to the root directory of the project and run the following command:

$ sam deploy --guided

Figure 7: Deployment command

You will then be asked to enter the name of your app and the AWS region to use. You’ll also need to confirm some actions:

Figure 8: Fill in and accept settings

After you’ve completed all the confirmations, deployment will start, and you’ll see all the resources being created. This takes about a minute or less.

 

Figure 9: List of resources to be created and their statuses

When the process is finished, open the AWS web console in your browser, navigate to API Gateway service, find the newly created API, and copy the URL to the root endpoint of your API.

Figure 10: URL to API root endpoint

Testing the API

Let’s create a few messages on the board using the default “curl” tool. Use the following command, but replace placeholders with your own data.

curl -d '{"author":"name", "text":"Message text"}' -H "Content-Type: application/json" -X POST https://your-api-id.execute-api.your-region.amazonaws.com/test/messages

Figure 11: Performing POST request with curl

Send a few different requests with different messages. If everything is OK, you’ll see “Message posted on board!” in the console without any errors.

In order to fetch the last messages, run an even shorter command:

curl https://your-api-id.execute-api.your-region.amazonaws.com/test/messages

Figure 12: Performing GET request with curl

Voila! You’ve just built a simple HTTP API with AWS Lambda and AWS SAM. Of course, in a real project you would use additional features and configurations, but the principles remain the same: define resources, define configurations, write the code, and run deploy.

Thundra Monitoring for tracking the issues

Right after you build your serverless API, the first thing you should do is to enable the necessary monitoring to understand the lifecycle of an API request. You can plug Thundra into your AWS account as described in our quick start guide. Once you’ve connected Thundra, you’ll need to instrument the “postMessage” and “getMessages” Lambda functions in order to see detailed information about every single invocation and have a global picture of your application.

Select functions in the list and click the “Instrument” button, then confirm instrumenting by clicking “OK.”

Figure 13: Confirm Lambda function instrumenting

After you direct some traffic on your API, Thundra will show you aggregated analysis on average duration, exact timings on the resource usage, start and end time and the requests and responses you interchange with your customers.

Figure 14: Details about a single invocation

Wrapping up

We frankly believe that serverless is the fastest and easiest way to build performant and cost-effective APIs from scratch. With the new addition of HTTP APIs, it’s now much easier to build an API with API-Gateway, Lambda and DynamoDB.

We, as Thundra, proudly recommend using HTTP APIs for application teams and use the detailed monitoring provided by Thundra for end-to-end understanding of distributed transactions.

Thundra is free up to 250K requests per month which can be quite useful for small projects or startups. If you’d like to gain the full observability through serverless APIs, here is your place to start.