x

Meet with us at AWS re:Invent and learn a better way to troubleshoot and secure your applications. Schedule Now!

Provisioned Concurrency: The Last Cold Start Bender!

Dec 3, 2019

 

2-provisioned-concurrency

Today, AWS Lambda has announced “Provisioned Concurrency”. I think it is one of the biggest ones since the launch of AWS Lambda in 2014. As the cold start is one of the biggest pain points of AWS Lambda, people have been already asking such kind of feature. We frankly believe that Provisioned Concurrency will enable companies to migrate their existing applications to serverless a lot more easier than ever. Using that, companies will be able to make sure that they won’t experience a deadly cold starts with their existing applications. We, as an AWS Advanced Tier Partner and DevOps competency holder, are proud to help the serverless community by providing support for provisioned concurrency metrics, and mixing it with Thundra Custom Runtime for Java and decreasing the cold start delay by 98%. 

How will Provisioned Concurrency useful?

With Provisioned Concurrency, customers will be able to control the performance of their applications by keeping their AWS Lambda functions initialized and ready to answer the upcoming request. In this way, all the previous headaches with cold start are just gone! Provisioned Concurrency will be the best fit for the customers who want to control the initialization time. Software teams can easily configure the sufficient amount of concurrency according to the need of their app to make sure that cold starts won’t be a problem for them anymore. Note that Provisioned Concurrency adds a new pricing dimension for Lambda functions. You will pay for the amount of concurrency you configured and for the time that you configure. However it is still less expensive than EC2 as when the container is idle, it is in frozen state and it doesn’t use any CPU resource.

Provisioned concurrency is perfectly useful for building interactive applications which are quite time-sensitive We frankly believe that this will unlock the possibility of migrating existing applications to serverless especially written in Java. In this way, serverless adoption will increase a lot faster than it was until today. In order to prove the positive effects of this new update, we’ve run some benchmarks with a Java application. 

Benchmark

Here are some behaviors of Provisioned Concurrency according to our experiments: (Note that some of these might be internal details and can be changed over time by AWS)

  • Provisioned containers are still in the frozen state when there is no request is being handled. So no background task can run when the container is idle like before.
  • Timeout for initialization is 130 seconds or the configured function timeout whichever is higher. 
  • If initialization fails (by exception or by timeout), it will be tried 3 times. It might depend on the amount of provisioned concurrency configured and the number of failing initializations.
  • There can be more containers than the configured provisioned concurrency count at any time. Interestingly,  provisioned containers can be destroyed but before destroying them, new ones are created and initialized. Lambda will ensure that new containers are enabled with Provisioned Concurrency before taking down old containers.
  • If the function has an abnormal termination (e.g. due to time out or crash), the service restarts the container. During this time, the available concurrency is temporarily lowered.

Results

We made some benchmarks to see how Provisioned Concurrency can reduce cold-start delays. In our experiments, 

  • We have a Lambda function which basically reads an item from AWS DynamoDB table in the current region.
  • We used Java runtime as it is the most problematic runtime suffers from cold start delays mostly. 
  • We compared pure Java 8 runtime, Thundra Custom Runtime for Java functions. As you may know, Thundra Custom Runtime for Java has two different modes: fast start enabled or disabled (which is basically the native Java runtime for AWS Lambda). We tested both approaches during our benchmarks. 
  • We run the tests with different memory sizes. 
  • We used init+request duration reported by AWS CloudWatch as the cold start delay as init duration is only reported at cold start.  

imageLikeEmbed

As seen from the results, Thundra Java custom runtime with fast start mode enabled already gives 40%-50% startup performance boost on cold start. 

You can find more details about our Java custom runtime from here

With Thundra Java custom runtime, in addition to “Fast Start” (FS) mode, when we enabled “Provisioned Concurrency” (PC), the results are astonishing!  Cold start delay decreases by 98% since most of the initialization is done on initialization before the cold start request with enabled Provisioned concurrency.

How to enable Provisioned Concurrency

Provisioned Concurrency doesn’t require any change to existing code. It seamlessly works with all existing functions and runtimes. For this reason, you don’t need to do anything else to configure it from your app. You just need to configure Provisioned Concurrency by using the AWS Management Console, the Lambda API, the AWS CLI, AWS CloudFormation, or through Application Auto Scaling. 

How to take advantage of Provisioned Concurrency and Thundra Custom Runtime for Java applications

Thundra’s Custom Runtime for Java applications makes some initializations while initializing the container out of the box. But, in order to let you create your own initialization logic, we also provide a custom initializer support. 

Out of the Box Initializers

Thundra Java custom runtime provides some out-of-the-box initializers so it can apply some predefined initialization automatically.

  • Initialize SSL components: According to our experiments, initialization of SSL engine, context and ciphers are time-consuming operations. So initializatiın of SSL components can reduce your cold start overhead up to 1 second. 
  • Initialize AWS SDK: Loading AWS SDK classes and initializing with the taken credentials can also help you to reduce cold start delay hundreds of milliseconds. 

Custom Initializer Support

Thundra Custom Runtime for Java provides registering your own initializers to reduce the time spent during cold start delay. These initializers are called on container initialization. Inside your own initializers, you can:

  • Read configurations from remote such as S3 or KMS
  • Fetch initial data
  • Setup an initial connection to remote service. But you should be aware that, during the time between the container initialization and request is being handled, the connection might be terminated from remote due to some reasons such as keep-alive timeout.
  • Setup Spring context (specific to Java runtime). We are seeing that many legacy applications written in Java use Spring and when they just lift-and-shift into AWS Lambda. Due to Spring context initialization, cold start takes 10s of seconds (some cases it is even more than 30 seconds which is the timeout limit when you put your Lambda function behind API Gateway).  So with our custom initializer support, you can set up your Spring context during initialization in advance of the request.

Summing up

We are very happy to become the launch partner for provisioned concurrency and very excited about the promising potential of it. With this update, serverless will be useful for almost every compute operation because the cold starts will no longer be an issue. We are particularly excited about the potential of Java applications. Combined with our Java Custom Runtime, Provisioned Concurrency just destroys the cold start time. If you have an existing application, written in Java, you can now migrate it to serverless and get the unmatched level observability insights and secure them using Thundra. 

We are at re:Invent in booth #627. Swing by our booth to chat about the potential of this update and let us show how Thundra can vanish the tooling concerns while adopting serverless.  If you want to see a dedicated demo at AWS re:Invent, please shoot a meeting request from here. You can sign up to Thundra or see live demo to see Thundra in action at any time.