What AWS CloudFormation Registry and CLI means for DevOps in the Cloud

Dec 17, 2019

 

aws-cloudformation-registry-and-cli

On the 18th of November, AWS announced the release of its AWS CloudFormation Registry and CLI which is meant to be an extension of AWS CloudFormation now supporting the creation of third party resources via the AWS CloudFormation console. This is also the first time the AWS service has been paired with launch partners such as Spotinst and Fortinet. Now with the support of such third party resources, AWS has enhanced the entire practice of building in the cloud as provisioning infrastructure now extends beyond simply AWS resources to also include series SaaS tools, empowering the way we build in the cloud.

Therefore, the purpose of this blog is to assert the importance of IaC in the domain of cloud computing and to highlight the importance of AWS’s developments to its AWS CloudFormation service. 

The Principles of IaC

IaC, short for Infrastructure as Code, is the practice whereby resources are provisioned via programmable defined scripts as compared to using management consoles to manually create environments of resources. Therefore, in the case of AWS CloudFormation, you do not need to use the AWS console or programmatic SDKs to create AWS resources. Moreover, with the AWS CloudFormation Registry you also do not need to use the console of third party tools which have their resources within the Registry. 

These scripts are machine-readable and allow for automatic deployment of the resources along with the required connections and interaction configurations between the resources. This is because IaC tools are expected to handle networks, virtual machines, load balancers, and connection topology. Moreover, every time an IaC script is applied it will always result in the same environment as described in the script.

Hence the benefits here become apparent. IaC is a common practice in DevOps as the goal of DevOps is to achieve automation of the software production process. This is because, with IaC, we are able to automate the building of our infrastructure, which is even more crucial in cloud environments. Even though cloud environments abstract a lot of the underlying architecture from developers, they require tedious configurations of individual cloud resources under the constraints of the resource vendor. Therefore IaC services such as AWS Cloudformation have always provided some form of respite from the need for repetitive configurations. 

Further benefits include state idempotency and templating. Since IaC allows us to model our infrastructure in a script based format, we can define the desired state of our cloud infrastructure. Therefore, if our infrastructure deviates from the desired state too much, we can automate its recovery using the IaC template.

1

Similarly, we can use the same template to replicate the desired state in multiple environments. This is extremely advantageous for testing purposes as we would like to mimic real-life scenarios. Hence instead of having to arduously configure each component to mirror the infrastructure to be tested, we can simply automate the provisioning of an identical infrastructure, followed by automated testing facilitated by CI/CD. Tools.

Therefore IaC tools can be seen as recipe journals for our cloud infrastructure. In fact, using this recipe book analogy for IaC services is so common that AWS’ jeff Barr cleverly titled his introductory blog of AWS CloudFormation ‘AWS CloudFormation – Create Your AWS Stack From a Recipe, back in 2011. However, people fail to realize that IaC tools are not like your ordinary recipe books, but rather entire automated kitchens that analyze these so-called recipes and cook entire cloud infrastructures for you. 

As a result, we can concur that IaC, in general, is imperative for a full DevOps experience. The question now is, what services does the largest cloud vendor out there, AWS, provide in terms of IaC? The answer before was CloudFormation.

CloudFormation and its Benefits with CloudFormation Registry

AWS CloudFormation was AWS' answer to IaC tooling.

AWS CloudFormation provides a common language for you to describe and provision all the infrastructure resources in your cloud environment. ~ AWS CloudFormation Page

CloudFormation allows you to define the desired AWS resources required along with their configurations and connections in blueprint documents called CloudFormation templates. These templates are then run within the AWS CloudFormation console to provision the defined infrastructure. In doing so, the service ensures that the cloud infrastructure components are deployed in the right manner according to the dependencies outlined in the CloudFormation template.

2

For example, if you would like to have an EC2 instance running within a VPC, then CloudFormation ensures that first the VPC is provisioned and then the EC2 instance. This also means that we no longer have to use the AWS management to configure and manually add the EC2 instance within the VPC.

However, one area where AWS CloudFormation lacked was the provisioning of third-party resources. Consider building in the cloud. Yes, AWS resources make-up the core of the application’s infrastructure, but these components would most likely communicate with third-party SaaS tools somewhere in the workflow. Going back to the example of an EC2 instance within a VPC, we may need this EC2 instance to then interact with the Stripe API.

Hence, even though we managed to bring DevOps style automation to the AWS side, we still lacked these privileges when it came to connecting our third-party tools to the core AWS infrastructure. This would often set us back to square one as the benefits of IaC discussed above were limited to only AWS resources.

This is where the AWS Cloud Registry comes in. Now with the new release, the problem of achieving IaC capabilities on external resources is solved. The AWS CloudFormation Registry allows the provisioning of these external third-party tools along with the AWS resources, with the release of this novel service, there are a total of seven SaaS tools offering their resources on the Registry. For example with Atlassian Opsgenie’s support for the AWS CloudFormation Registry, you can now provision Opsgenie resources such as users, teams and integrations along with your AWS resources. Hence you can automate configuring Opsgenie incident management services into your AWS infrastructure.

This means that we can now further benefit from DevOps practices as AWS has extended its IaC services over external technology stacks and is not only limited to AWS. Moreover, AWS CloudFormation Registry is open source so the community can constantly build more custom resources that can be provisioned automatically through the AWS CloudFormation CLI. This enhances development in the cloud, especially using an AWS stack.

A Step Further with AWS CloudFormation CLI

As discussed the AWS CloudFormation Registry provides third party resources for you to include in the recipe books of your desired cloud infrastructures. If we are to follow suit with this recipe analogy, the CloudFormation Registry can be considered as your pantry of resources, and at the moment your pantry is stocked with the resources provided by the third party SaaS partners of the AWS CloudFormation Registry launch. The question then is, what if you would like to expand this pantry?

3

This is where the CLI component comes in of the new AWS CloudFormation release. The AWS CloudFormation CLI arms us with the set of utensils that allows us to build our own custom resources that we can later include in AWS CloudFormation templates, giving us the freedom to expand our pantries. AWS provides the cfn (CloudFormation  Command Line Interface) that allows us to initialize our custom resource projects, auto-generates the basic code structure for us, and allows us to test our built resources along with registering it to our personal AWS CloudFormation Registries based on our account and regions.

AWS provides a comprehensive set of resources to get started in building these custom resources. Moreover, the push towards open-sourcing AWS CloudFormation resources means that we can soon expect a compendium of resources to be included in our AWS CloudFormation templates. Thus improving the entire experience of using the IaC service, increasing the velocity with which we build in the cloud. 

A Wrap up of what AWS CloudFormation means for DevOps

With AWS CloudFormation registry and CLI we see the benefits of having third-party, non-AWS, resources within our cloud infrastructure processes. We can be assured that using AWS CloudFormation for applications such as correcting cloud infrastructures for drift and automated CI/CD now covers your entire infrastructure and not only specific AWS resources. The flexibility of the AWS CloudFormation CLI and the reliability of expansion of the AWS CloudFormation Registry can only mean that over time, as more and more resources will be out there, we can expect development in the cloud to become much easier. No longer must we reinvent the wheel, we now just need to worry about the destination.