Whatever monitoring tool you use for AWS Lambda, privacy of the monitoring data is always a headache. It is very normal and common that monitoring data can include sensitive data or clues about sensitive data. To solve this, it is better to keep the monitoring data as a secret at your own instance(s). But, how will I visualize and extract insights from the data? Will I allocate time to query this data yourself? The queries will take too much, which fields will I index?
All those questions show that keeping your monitoring data private and having successful insights about data is the challenge for companies of every scale. Splunk comes at this point as a very powerful data to turn any kind of data into actionable intelligence. Using Splunk, you can forward your data from different resources to one place where data is indexed and store automatically. In this way, it is very easy to query and extract the intelligence from data. We have developed “Thundra Serverless Observability for Splunk” in order to help AWS Lambda developers to see their monitoring data powered by Thundra at their own Splunk instances. In this way, they will be able to enjoy the power of Splunk for querying and visualizing data while keeping it private. Until now and by default, Thundra agents send the data to Thundra Web Console. In order to send it to your Splunk instances, there is a checklist to complete.
Get a License Key from us
Your license key will allow your Thundra agents to redirect data to your Splunk instance. You will sign up the Thundra, will create the license key from Thundra Web Console. You can close the browser window and never turn back to Web Console provided that you have copied the license key to clipboard.
Install the Splunk AMI to your AWS account
Thundra agents will not send data to Thundra Web Console. It is ok. But.. Where they will send now? They will send it to a specific instance called Thundra-Splunk Integrator in which there is Thundra Receiver and Splunk Universal Forwarder. Receiver will make the data transformation, and forwarder will send it directly to your Splunk instance. To achieve that, you need to create an EC2 instance at your AWS account. Note that this instance doesn’t have to be at the same account with your Lambdas but should be public so that your Lambdas can send data to them.
To install the Thundra-Splunk Integrator, we have provided a CloudFormation template here. It includes everything necessary to send your data to your Splunk instance. Find below the architectural diagram of it.
To complete the installation of stack, you need to follow those steps. You will be able to input the address of your private Splunk instance during installation. Keep in mind that you will need to create an API key prefix or API keys during the installation which you will later use to configure your Thundra agents. You can change API key prefix and add new API keys whenever you want using the REST api of Thundra-Splunk integrator.
(Re)configure your Lambda
You have created your license key. You have installed the EC2 instance. Now it is time to send Thundra data to that EC2 instance from which it will be forwarded to your Splunk instance. In order to do that, you need to change/set the environment variable for `thundra_lambda_publish_rest_baseUrl` of your AWS Lambda function to the URL of your EC2 instance. You can find the URL of the EC2 instance for your Thundra-Splunk Integrator in the `Output` tab of your CloudFormation stack. You need to also add the API key prefix that you have set in previous step to `thundra_api_key` variable. See the detailed instructions here.
Now, you have finished to configuration of the Thundra to send its data to your Splunk instance. You are now able to see Thundra data with our application named “Thundra Serverless Observability for Splunk” with all of its helpful futures. I should say that you are gifted with the dashboards to help you identify any issues. For example; you can see AWS Lambda functions making requests to specific URL and see which URL is specifically slower than others. There might be many more cases but we will talk about those in our later posts.
You will also be able to make your own dashboards for your use cases by querying the Thundra data. Thundra Serverless Observability for Splunk has default alerts which are fully customizable according to your need.
We are giving great importance to customer needs about integrations. We will continue to come up with more integrations. So, stay tuned for upcoming “Thundra + something” news. Visit our website to learn more about Splunk integration!