Structured Logging with Serilog and App Insights

Monitoring and Logging are critical pieces to the overall health and success of an application.

Logging, which we’ll be specifically focusing on in this article, gives you insight into the various code paths of your application.

A logging tool commonly used in .NET applications is Serilog. Serilog gives you the ability to log to files and the console as well as many other locations. One of these other locations is Azure Application Insights, and we’ll use it in this article. The other locations/sinks can be found here.

It is also a great idea to have your logs published to a centralized location accessible by the entire development team. This way you can set up custom alerts if a log message is or isn’t firing, if you haven’t seen enough of a specific log message, what errors may be getting thrown, track an identifiers’ journey through a service or multiple services, etc.

One problem I see initially in many applications that are using Serilog, is that the log statement format from the application is not ideal.

Its very common to see a log statement like the following that leverages string interpolation:

_logger.LogInformation($"Retrieving projectId: {projectId}");

Assume projectId holds the value of 1.

In App Insights the log message would appear normal like this:

The message appears to be correct, but the MessageTemplate is where the first issue comes in. Since we are using string interpolation, App Insights sees the entire message with the value already replaced. This would make it impossible for us to be able to create any useful queries based on this MessageTemplate.

Structured Logging

We can improve the output of the above by using the structured logging format. Let’s change the log statement to the following:

_logger.LogInformation("Retrieving projectId: {projectId}", projectId);

Let’s take a look at how this is reported up to App Insights.

As you can see, the message is still the exact same as before but notice the MessageTemplate and now a new attribute named projectId under the message’s CustomDimensions.

This enables you to write some better, more efficient and more meaningful queries in App Insights.

For example, we can now write a query like this:

traces
| project message, messageTemplate = customDimensions.MessageTemplate, projectId = customDimensions.projectId
| where messageTemplate == "Retrieving projectId: {projectId}"

The above query would output the following:

Notice in this query we can now search by a specific MessageTemplate, which gives us the ability to group all of the log messages output with this template with a simple query.

We can then extend the query to add on to the where clause to perform some filter based on the projectId.

Summary

Logging is an extremely important component in your applications. Serilog is a great tool that provides an easy way to get logging enabled in your applications and provides many different sinks to push those logs to a centralized location.

Using Serilog, while you can write log statements using string interpolation you won’t get the benefits of writing them using the structured log format. Changing your log statement format away from string interpolation is a relatively easy change that can provide a great amount of value for your monitoring, alerting, and log queries.