Observability is Key
One of the great things about Node.js is how well it performs in a container. Its fast start up time, and relatively small size make it a favorite for microservice applications on OpenShift. But with this shift to containerized deployments comes some complexity. As a result, monitoring Node.js applications can be difficult. At times it seems as though the performance and behavior of our applications become opaque to us. So what can we do to find and address issues in our services before they become a problem? We need to enhance observability by monitoring the state of our services.
Instrumentation of our applications is one way to increase observability. Therefore, in this article, I will demonstrate the instrumentation of a Node.js application using Prometheus.
Continue reading “Monitoring Node.js Applications on OpenShift with Prometheus”
A number of the SystemTap script examples in the newly released SystemTap 4.0 available in Fedora 28 and 29 have reduced the amount of time required to convert the scripts into running instrumentation by using the
This article discusses the particular changes made in the scripts and how you might also use this new tapset to make the instrumentation that monitors system calls smaller and more efficient. (This article is a follow-on to my previous article: Analyzing and reducing SystemTap’s startup cost for scripts.)
The key observation that triggered the creation of the
syscall_any tapset was a number of scripts that did not use the
syscall arguments. The scripts often used
syscall.*.return, but they were only concerned with the particular
syscall name and the return value. This type of information for all the system calls is available from the
sys_exit kernel tracepoints. Thus, rather than creating hundreds of kprobes for each of the individual functions implementing the various system calls, there are just a couple of tracepoints being used in their place.
Continue reading “Reducing the startup overhead of SystemTap monitoring scripts with syscall_any tapset”
Production being affected by software issues is always an unwanted scenario. Diagnosing production issues, however, should never be an unplanned activity. Structured testing and QA efforts would ideally prevent any software bugs from entering production. So the dilemma is how to prepare for something unexpected in production that was not considered during the earlier testing and QA phases.
This article discusses Byteman, a tool that leverages the Java Instrumentation API to inject Java code into methods without the need to recompile, repackage, or even redeploy the application.
Continue reading “Diagnosing Java applications on the fly with Byteman”
SystemTap is a powerful tool for investigating system issues, but for some SystemTap instrumentation scripts, the startup times are too long. This article is Part 1 of a series and describes how to analyze and reduce SystemTap’s startup costs for scripts.
We can use SystemTap to investigate this problem and provide some hard data on the time required for each of the passes that SystemTap uses to convert a SystemTap script into instrumentation. SystemTap has a set of probe points marking the start and end of passes from 0 to 5:
- pass0: Parsing command-line arguments
- pass1: Parsing scripts
- pass2: Elaboration
- pass3: Translation to C
- pass4: Compilation of C code into kernel module
- pass5: Running the instrumentation
Articles in this series:
Continue reading “Analyzing and reducing SystemTap’s startup cost for scripts”
As every developer knows, debugging an application can be difficult and often enough you spend as much or more time debugging an application as originally writing it. Every programmer develops their collection of tools and techniques. Traditionally these have included full-fledged debuggers, instrumentation of the code, and tracing and logging. Each of these has their particular strengths and weaknesses.
Continue reading “Programmatic Debugging: Part 1 the challenge”