Using Azure Application Insights for centralized logging

A typical enterprise system usually runs on multiple servers behind a load balancer. The simplest logging option could be logging locally to a location on the server where the app runs. However, this setup makes triaging an issue using the logs difficult. The problem is more prominent if the system consists of multiple applications running on different servers, and the sessions are not sticky. When an error occurs, the developer has to go to the different servers and applications to gather the logs. Even when the developer has gathered all the logs, it may still be challenging to piece together the log events that belong to a same session in a chronological order to troubleshoot the issue.

In the past, I faced this problem with the applications I worked on. When troubleshooting an issue, I had to download the logs from multiple servers. Then, I used grep to search for keywords relating to the error. The process is time consuming, and I often got confused when I could not tell whether a log statement related to the error because of asynchronous events.

Things got much better after I integrated the apps with application insights which has powerful features for monitoring and logging.

In application insight, I can search for telemetry events including logs by entering keyword in the search bar. I can also filter by event types. For example, below screenshot shows all the exceptions that occurred in the last 24 hours.

I can then drill into a specific exception by clicking on the result which opens the End-to-end transaction details page, as shown in the below screenshot.

In the far right, I can see details of the exception, including the stack trace. In the middle, I see the different events leading up to the exception and also events following the exception. One interesting thing is app insights is able to detect the different types of the components. In the screenshot, the icon of the server with the cloud depicts my web application that runs on an azure virtual machine, The globe icon represents the web API running on premise with the globe icon. The word “Web Service” represents the third-party payment provider API, which is a SOAP based web service.

On the bottom right, I see the different options for viewing related items. The option I find the most useful is the “All available telemetry for this user session”. Clicking on this option, I can see all the trace logs which the applications sent for this session. This option is super helpful because I don’t have to search through the logs on the different servers, sort and filter out the ones that do not belong to the same session.

The above screenshot shows the log statements I setup in the app. For example, from the log, I can see that the error occurred because the user entered an invalid card number.

By integrating with application insights, I now have a centralized place to store the logs and a powerful framework to utilize the logs. In the next post, I’ll share how I setup the web APIs and angular applications to send telemetry to app insights. Until then, I encourage you to check out the links in the resource to learn more about application insights.

Resources

Application Insights overview

Application Insights transaction diagnostics – Azure Monitor | Microsoft Learn