An automated delivery pipeline is crucial for microservice, distributed, and serverless applications. With serverless applications, components (e.g., a function) can change independently of others. A well-defined automated pipeline for deploying serverless applications is thus important for maintaining these applications in the long run.
With so many components in serverless applications—and each with different life cycles—integration issues between the different elements frequently occur. For example, a particular set of functions might be managed by one team with another set being controlled by another team in a different location. Yet both groups of functions must be able to work together. This requires a reliable automated delivery pipeline featuring all the necessary automated integration tests.
For those using AWS services, AWS offers a complete set of tools for automating the deployment and development of serverless applications. These tools cover everything from source control to complex CI/CD pipeline creation.
This blog post will explore the most popular tools for building automated tests and deployment pipelines.
AWS CodeCommit is a fully managed source control service for hosting Git-based repositories, similar to GitHub or Bitbucket. AWS CodeCommit eliminates concerns about managing the source control system and ensuring infrastructure can handle company growth.
AWS CodeDeploy is a managed deployment service for automating deployments to different compute services from AWS (e.g., EC2, Fargate, and Lambda). CodeDeploy facilitates the complex process of updating applications. One of the most beneficial features for Lambda is the possibility of Blue/Green or rolling deployments. This allows for the initial deployment of new function versions to only a small set of customers, with it later being deployed to everybody. Integration tests can thus be performed on new functions without impacting customers if issues arise.
AWS CodePipeline is a managed continuous delivery service to assist with automation of deployment pipelines for applications as well as infrastructure updates. The service, helps to define the workflow, beginning by defining where the code originates. One can choose from various remote source repositories, such as Github, Bitbucket, or CodeCommit. The next step is setting up a test for the applications followed by the deployments. Integration and/or acceptance tests can also be added to the pipeline.
AWS CodeBuild is a fully managed continuous integration service for building serverless applications, running tests, and packaging applications for later deployment. CodeBuild can be integrated with multiple well-known services—like Jenkins for continuous integration, BlazeMeter for for continuous testing, or Ghost Inspector for browser testing.
AWS CodeStar is a unified interface to develop, build, and deploy applications on AWS. It integrates with other AWS tools and displays the results in a single dashboard. It thus eliminates the need for many different tools. CodeStar also offers support for AWS Lambda.
To begin a CodeStar project, one of the existing project templates is selected. Next, the project team is created with the access management feature, allowing one to define user roles (e.g., owner, contributor, or viewer).
AWS CodeStar also integrates with other AWS tools, such as CodeCommit. The project code can thus be stored there remotely. It also integrates with CodePipeline for CI/CD pipeline creation. Each project features a default pipeline to test and deploy the project. CodeDeploy and CloudFormation are used for project deployment.
Projects in CodeStar can be modified using traditional editors, like Atom or Visual Studio Code, or with a collaborative editor such as Cloud9.
The project dashboard is one of CodeStar’s best features. The single management console offers a full view of tools and tool integration, allowing all commits, tests, and deployments to be managed from one place. In addition, it provides a Wiki for the project and integrates with CloudWatch so that the project can be monitored from the dashboard.
Tying It All Together
Using all of the tools previously discussed, AWS CodeStar performs a number of different actions. AWS CodePipeline also creates pipelines using AWS CodeCommit, AWS CodeBuild, and AWS CodeDeploy.
Figure 1: Deployment and development automation of serverless applications using AWS CodeStar and AWS CodePipeline
Using the project templates is an easy way to start a new project. Following is a step-by-step breakdown of the process:
- Create an AWS CodeCommit repository.
- Add the code.
- CodeStar creates a dashboard for the project and a basic pipeline for building, testing, and deploying the project using AWS CodePipeline and a Wiki page.
Figure 2: Screenshot from AWS Console
- Click on the left navigation bar “Pipeline.” AWS CodePipeline will open, and the automated pipeline can be modified there. From here, the pipeline can be viewed and customized. Complex actions can also be added to the pipeline. For example, if multiple stages are being used, this can be defined, allowing for building, testing, and deployment to R&D. If everything goes well, the same can then be done for production.
Different parts of the pipeline interact with specific services. For example, the source uses CodeCommit, build relies on CodeBuild, and CodeDeploy is used for deployment. It can be configured to use these or other services (e.g., GitHub for the source or Jenkins for building) as needed.
Additional steps can also be incorporated into the pipeline. For example, one might wish to add another step for testing. In this case, the desired service to be used for testing can also be defined.
Figure 3: Screenshot from AWS Console
Improving the CI/CD Pipeline
When it comes to maintaining a serverless production-ready application, these tools alone are insufficient. Automated testing and deployment are of course very important, but automated monitoring is also crucial.
Failure in a distributed system is inevitable. This is something that cannot be prevented. However, one can design a system that is resilient to failure.
Monitoring is of course an important component of building such a system. Monitoring tools help to decipher how a system is behaving under different loads. Monitoring systems also show system failures. These systems are essentially a window into system health.
A reliable monitoring tool eliminates the need to search the console for errors by identifying any anomalies. If something is not working as expected, the user is then instantly notified so that issue can be addressed.
Ensuring App Health in the Ever-Evolving Serverless World
Serverless applications are constantly evolving. With so many moving pieces and so many different tools involved, maintaining application health in production can be challenging. Development is only part of the picture, as without monitoring, the development environment will fail. Automated build, testing, and deployment tools combined with application monitoring tools are key for achieving application health.
A reliable monitoring system is critical for maintaining a serverless application. Thundra is a monitoring tool offering full observability for serverless applications. It allows for analysis of what is going on with an application at the request level.