Getting Started with Foresight
To get started, we’ll log in to Foresight and create a new project.
Step 1: Create an Account
Your first step will be to sign up for a Thundra Foresight account.
Figure 1: Sign up for a Thundra account
From the sign-up page, you can create an account with an existing GitHub or Google account, or with your email address.
Step 2: Create a New Project in Foresight
Once you sign up with Foresight and sign in to the platform, you’ll see three product options to choose from APM, Foresight, and Sidekick. Choose the Foresight product.
Figure 2: Choose Foresight from among Thundra’s products
Next, click on “connect repositories” to start integrating Foresight with your repository.
Figure 3: Connect a repository to get started with Foresight
After you’ve connected a repository, create a new project in Foresight for monitoring the application. Name your project and hit “Create.”
Figure 4: Create new project with Foresight
Figure 5: There are multiple platforms available with Foresight
Now that you’ve created a project, you have multiple options for getting started. In this article, we’re focusing on how Thundra Foresight can be integrated, so we’ll select the “Test Monitoring” option.
Figure 6: Test monitoring platforms supported
Since we’re integrating Foresight with Azure, we’ll choose the Azure DevOps option.
Step 3: Collect Project and API Keys
When you select Azure DevOps, Thundra automatically provides the project key ID and the API key. You’ll need this information for a future step, so make sure you record it.
Figure 7: Auto-generated configuration for getting started with Azure DevOps
Integrating the Azure DevOps CI Platform with Thundra Foresight
In order to integrate the Azure DevOps CI with Foresight, we’ll need to install the Visual Studio Extension.
Figure 8: Thundra Foresight Initializer installation
From the Thundra Foresight Initializer page, click the “Get it free” button, then select the organization you wish to install Foresight into. The Azure DevOps CI platform has functionalities that support users creating many organizations, and each organization can have its own project. So, when you integrate Thundra Foresight with Azure DevOps CI, it’s important to specify the organization you want to integrate Foresight with.
Once that’s complete, we’ll begin configuring for the pipeline in the Azure DevOps CI. You can choose between "Use the classic editor" for simplicity or creating a YAML config file. For our purposes, we’ll choose the YAML file option because it provides more flexibility.
We’ll now use the keys Thundra auto-generated for us earlier and copy them into the Azure DevOps YAML file.
In our sample end-to-end test job, we’ll assume we’re working with Java and the Maven build tool. The YAML file will look like this:
# Maven # Build your Java project and run tests with Apache Maven. # Add steps that analyze code, save build artifacts, deploy, and more: # https://docs.microsoft.com/azure/devops/pipelines/languages/java trigger: - main pool: vmImage: ubuntu-latest steps: - task: ThundraForesightInitializer@0 inputs: api_key: $(THUNDRA_APIKEY) project_id: $(THUNDRA_AGENT_TEST_PROJECT_ID) agent_version: '2.7.56' instrumenter_version: '0.0.6' build_run_type: 'Maven' - task: Maven@3 inputs: mavenPomFile: 'pom.xml' mavenOptions: '-Xmx3072m' javaHomeOption: 'JDKVersion' jdkVersionOption: '1.8' jdkArchitectureOption: 'x64' publishJUnitResults: true testResultsFiles: '**/surefire-reports/TEST-*.xml' goals: 'test'
For security reasons, we don’t usually expose the credentials to the pipeline. To get around this, we’ll also need to add environment variables into the pipeline in Azure DevOps.
Figure 9: Pipeline with environment variables setup for Thundra Foresight
Now we’re ready to trigger the pipeline.
Note that the configuration pipeline for Gradle and other build tools is similar to Maven. Foresight provides the template for each build tool, so you won’t have to figure out how to write them yourself. Instead, just copy the auto-generated template and paste it into the config YAML file for Azure DevOps.
Running the CI Pipeline from Azure’s DevOps CI Platform
With Foresight, we’re now able to monitor multiple aspects of our application, including:
- An overview of all tests run
- Slowest tests
- Useful logs and metrics
Figure 10: Overview of a test run in Thundra Foresight
The test run overview is important because we often have hundreds of tests. Going through all of them for details each time just isn’t feasible. This summary board illustrates which tests might have a problem, making them easy to target.
If we go into each test’s details, we’ll also see traces for the requests that have been made in our test. The below image is an example integration test among different services in the AWS platform. We can see that the request to the API gateway/products was successful and that its call to the Lambda function for creating a new product was also successful. However, we cannot create a product successfully because we lost the connection to the Amazon DynamoDB table.
Figure 11: Distributed traces collected from the test
Foresight also offers Time Travel Debugging, which gives us a log for each request that has been made in the test. Sometimes a test fails only once, and we can’t reproduce it. The Time Travel Debugging feature allows us to look back in time at test details so we can easily debug it. This saves testers the time they’d otherwise spend setting up an environment for reproducing the bug, and trying different ways to reproduce it.
Figure 12: Time Travel Debugging
We already have the capability to monitor the application and debug the integration tests. How about the end-to-end test that failed?
So often in software testing, end-to-end tests take the most effort to debug. That’s because in end-to-end tests, there are a lot of dependencies, or third-party apps, involved. Luckily, with Foresight, we can save screenshots when we run our end-to-end tests and easily access them from the dashboard. These screenshots provide a lot of information we don’t get through logs.
Figure 13: Screenshot for the end-to-end testing
Foresight offers a complete picture of what your application is doing, what the hidden issues are, why your integration tests are failing, and which of your end-to-end tests are flaky. Open-source software offers some of these capabilities, but it requires you to integrate a lot of tools and dependencies. You’d have to create a new service to handle the logs or to save screenshots for the end-to-end test.
These steps pull your entire software team away from building and improving your application. By integrating Foresight with Azure’s DevOps CI platform, you have all of it instantly.
Building a running CI pipeline might not be a difficult task, but maintaining it over time and getting fast feedback from your tests certainly can be. If your tests are flaky enough, your team may want to remove automated tests altogether in order to gain speed. But this is dangerous, as it can impact quality and lead customers to look elsewhere for better products.
With the help of Thundra Foresight, your team can always backtrack to easily find the root cause of issues. You can finally focus on developing new features and building applications that make your customers happy, rather than struggling to debug failed tests.
Take the headache out of debugging tests and start your Thundra Foresight journey today.