Learning the many ways your lambdas can be invoked
While AWS Lambda enables massive scale in computing for serverless applications, one important topic to understand well is how or who triggered your lambda.
At the moment of this writing, there are 3 invocation models:
- Synchronous Invocation method with
- Asynchronous Invocation method with
- Event Source Mapping method with
It is important to know who invoked your lambda, because the scaling behavior and error type/handler will depend on the caller.
An up-to-date list of services that can trigger lambda synchronous from AWS:
- Amazon API Gateway
- Elastic Load Balancing (Application Load Balancer)
- AWS Step Functions
- Amazon CloudFront (Lambda@Edge)
- Amazon Kinesis Data Firehose
- Amazon Cognito
- Amazon Alexa
- Amazon Lex
A common scenario for synchronous invocation is the API Gateway + Lambda integration:
In the diagram above, both clients need to wait for the whole roundtrip of requests and computing (red arrows), to react based on the response.
This invocation type is called
RequestResponse and is the default mode of any lambda. It keeps the connection open until the function returns a response or times out. The details about the function response, including errors, are part of the response body and headers.
You can perform
RequestRespons invocation using the CLI, SDKs and many other options. When using the synchronous style, you are responsible for inspecting the response and check if there are any errors and if you should retry or not.
An up-to-date list of services that can trigger lambda asynchronous from AWS:
- Amazon Simple Storage Service
- Amazon Simple Notification Service
- Amazon Simple Email Service
- AWS CloudFormation
- Amazon CloudWatch Logs
- Amazon CloudWatch Events
- AWS CodeCommit
- AWS Config
- AWS IoT
- AWS IoT Events
- AWS CodePipeline
A common scenario for asynchronous invocation is to configure services like Amazon S3 to invoke your function when an object is created:
In the diagram above, Amazon S3 will call the lambda asynchronous as soon as a new object is written to the bucket.
With asynchronous invocation, Lambda queues the event within its internal infrastructure before passing it to your function. The caller will receive a response as soon as the event is queued.
Your application or clients are unaware of errors. By default AWS Lambda sets a retry policy for asynchronous invocation, your function will be called twice before discarding the event. To avoid losses while using asynchronous lambda, always attach a dead-letter queue on the function to capture events that weren't successfully processed.
Another use case we can apply this technique (and the service is not in the list) is into building asynchronous endpoints with API Gateway. Because AWS Lambda will reply as soon as the event is queued, we can return a response to our clients, and let the long process run in the background without blocking our consumers:
As we can see in the diagram above, API Gateway will receive a
200 OK as soon as AWS Lambda receives the request. It is up to your front-end to handle the temporary/pending status of the process since the response doesn't contain any state.
You can use this behavior to perform batch operations, long workflows (e.g. AWS Step Functions) or to simply detach the synchronous effect from your front-end.
Event Source Mapping
An up-to-date list of services that can trigger lambda as event source mapping from AWS:
- Amazon Kinesis
- Amazon DynamoDB
- Amazon Simple Queue Service
A common scenario for event source mapping is the integration with DynamoDB Streams and AWS Lambda:
Services that generate a data stream or a queue can be used as an event source and be mapped to a lambda function. These services don't invoke Lambda function directly.
Data stream or queue are read in batches, the function receives multiple items in its execution. Batch sizes vary per service and you can configure the size that the event source mapping will send.
If the batch size is too large to send in one event, it has to be split up, resulting in events with smaller numbers of items than the batch size.
Handling errors with streams are based on data expiration from the event source. Using Kinesis Data Stream as example, records can be stored for 24 hours by default, or up to 168 hours. Each integration has its own details.
Handling errors from a queue, you can set the length of time between retries and destination for failed events, redrive policy and visibility timeout can be used in the source queue.
From managed services and servers to integrations without code, AWS Lambda is the perfect fit for many tasks. Understanding its behavior and how to use the correct invocation style is crucial for your system availability.
With that, we can rest our minds for a bit, there are many use cases for synchronous, asynchronous and event source mapping that we can explore in the future.