4 minutes read

POSTED May, 2019 dot IN Product

Announcing Manuel Instrumentation for .NET Agent

Hasan Fehmi Danaci

Written by Hasan Fehmi Danaci

Ex-Software Engineer @Thundra

manuel-istrumentation-dotnet X

When Microsoft announced the .NET in 2002, who would have thought that .NET would one day be open source and work on Linux? Hard to stand against the strong wind. Microsoft didn't ignore the open source software stream and what it brought, so started to open the source code and patents of .NET to everyone, progressively. It is undeniable that the pace of development and community growth has increased after this decision. Today the .NET platform supports many languages and platforms, thus reaching a community of considerable size. Opportunities should have amazed Amazon's appetite; In 2016, while Azure Functions is not yet in the market, Amazon has started supporting .NET core 1.0 runtime in Lambda. Nowadays Lambda supports the .NET core 2.1 version. Announcement of next LTS versions by Microsoft is expected for more current .NET runtime versions to be run by Lambda. 

While the performance of the .NET core 1.0 runtime is disappointing for some, we see that the performance improvements made in the .NET core 2.0+ work. As Thundra, we strongly recommend that users who are still using the .NET core 1.0 should migrate to .NET core 2.1. The performance of .NET core 2.1 runtime on Lambda is so amazing that Yun Zhi Lin's benchmark shows us that the best result in the average time belongs to .NET 2.1 runtime.

For some time, you have been able to follow the invocation and metric of the functions written in the .NET core through Thundra. We are glad to announce that you'll now be able to do the manual instrumentation with the OpenTracing compatible new API. Thanks to this new ability, you will be able to measure how long a part of your serverless application takes. Plus, you’ll be able to pass the values of local variables or objects to Thundra spans as tags. This is the first step to enable you to debug and troubleshoot your .NET applications of AWS Lambda. 

In monolithic services, it is easy to troubleshoot because you can count the number of the usual suspects with the fingers of a hand. On the contrary, trying to solve a problem in the Microservices architecture is often like looking for a needle in a haystack. The user called a service, it called another ten services, got a result from the database, updated cache, got NullPointerException, returned with response HTTP 500... At this point, you want to monitor the services as connected, until the response to the request is returned. Here at this point, distributed tracing comes into play. Although the term "distributed tracing" is not a new concept, the article titled "Dapper, a Large-Scale Distributed Systems Tracing Infrastructure" published by Google employees has become a milestone. Dapper created a mental map for distributed tracing, and it led open source monitoring applications to go to the right way. Dapper inspired Zipkin and Jaeger. The team that developed Jaeger saw the need for standardization for different monitoring tools and instrumentation libraries to work interoperably. Here at this point, OpenTracing comes into the picture. OpenTracing is a framework that aims to create a standard for distributed tracing with an abstract API, data model, conventions and mental map. Thundra's agents are fully OpenTracing compliant, achieving monitoring with industry standard. Thundra supports usage of other OpenTracing compatible instrumentation libraries with your application.

(Sarjeel Yusuf previously showed us how to integrate Thundra with the .NET runtime. If you have not done so please follow the article.)

Understanding .NET Serverless Apps Better

Now let's see how we can use the new Thundra agent that is fully compatible with the OpenTracing API. Let's think about a scenario that can be seen frequently in the life of a developer. We've written a Lambda that updates the database and cache. Like every piece of code that makes a database update, our Lambda first validates the request. If the update request is valid, the request terminates by updating the database and cache. Well, let's say one day our Lambda slowed down, what could be the root cause? Did the database become unresponsive to requests? Alternatively, do you have a cache problem? Since the entities have started to grow too large recently, have the update times increased (both database and cache)? Did the new developer get your complex validation code broken? These questions we ask to find the root cause of the problem also reveal what we should follow; 

(1) Request Time

(2) Validation Time

(3) Entity Update Time (DB Update + Cache Update)

(4) Database Update Time

(5) Cache Update Time

These steps map spans in OpenTracing world. If you are not familiar with OpenTracing terminology, you could check this documentation

The Thundra agent is already creating the root span automatically, where you can track the request time. It is evident that there must be separated spans for the validation time and the entity update time. Also, we should able to monitor database and cache updates separately, but by joining them together under a root span, we can make entity-related problems more visible (for example, entity size is too big). Let's dig into the code.

using System;
using System.Linq; using System.Threading; using Amazon.Lambda.Core; using Microsoft.Extensions.Logging; using Thundra.Agent.Lambda.Core; using Thundra.Agent.Log.AspNetCore; using OpenTracing.Util; using OpenTracing;   [assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.Json.JsonSerializer))] namespace thundra_dotnet_opentracing_dummy_example {   public class Function : LambdaRequestHandler<string, string>   {       public override string DoHandleRequest(string request, ILambdaContext context)       {           var loggerFactory = new LoggerFactory().AddThundraProvider();           var logger = loggerFactory.CreateLogger<Function>();             Console.WriteLine("Hello OpenTracing.");             ValidateEntity();           using (IScope updateEntityScope = GlobalTracer.Instance.BuildSpan("UpdateEntity").StartActive(finishSpanOnDispose: true))           {               updateEntityScope.Span.SetTag("updatedEntityID", 1);               updateEntityScope.Span.SetTag("entitySize","1KB");               using (IScope updateDatabaseScope = GlobalTracer.Instance.BuildSpan("UpdateDatabase").StartActive(finishSpanOnDispose: true))               {                   UpdateDatabase();               }               using (IScope updateCacheScope = GlobalTracer.Instance.BuildSpan("UpdateCache").StartActive(finishSpanOnDispose: true))               {                   UpdateCache();               }           }             return request?.ToUpper();       }         private void ValidateEntity() {           IScope validationScope = GlobalTracer.Instance.BuildSpan("Validation").StartActive(finishSpanOnDispose: true);           //some validations...           Thread.Sleep(50);           validationScope.Dispose();         private void UpdateDatabase()       {           //some database calls...           Thread.Sleep(1000);       }         private void UpdateCache()       {           //some cache calls...           Thread.Sleep(500);       }   } }

We did not take the controls into account to keep the example simple and to remain focused on tracing. Thread.Sleep() used for simulating the processes that will take time. ID and size of the entity are added to the UpdateEntity scope as tags. Now, we can monitor our lambda on Thundra.


Now you know what to blame when things go wrong. Of course, manual instrumentation is not practical. At Thundra, we have already begun to develop our .NET agent with the automatic instrumentation capability. So you won't have to change any code for monitoring.

Now easier: Plugging Thundra to your Serverless Apps With AWS SAM & Serverless Framework

For users who deploy serverless applications using SAM and Serverless framework, changing each lambda function one by one is very troublesome. For this reason, we have created a proxy that you can use Thundra by only changing the config file without changing your lambda functions. Examples are available in our documentation:

We’ll be happy to listen to your ideas and needs and come up with the best possible .NET support. Please submit your requests to support@thundra.io or come to our Slack and let’s chat. You can sign up to Thundra or see the live demo to see Thundra in action.