What are BPF Maps and how are they used in stapbpf

Compared to SystemTap’s default backend, one of stapbpf’s most distinguishing features is the absence of a kernel module runtime. The BPF machinery inside the kernel instead mostly handles its runtime. Therefore it would be very helpful if BPF provided us with a way for states to be maintained across multiple invocations of BPF programs and for userspace programs to be able to communicate with BPF programs. This is accomplished by BPF maps. In this blog post, I will introduce BPF maps and explain their role in stapbpf’s implementation.

Continue reading What are BPF Maps and how are they used in stapbpf


Introducing stapbpf – SystemTap’s new BPF backend

SystemTap 3.2 includes an early prototype of SystemTap’s new BPF backend (stapbpf). It represents a first step towards leveraging powerful new tracing and performance analysis capabilities recently added to the Linux kernel. In this post, I will compare the translation process of stapbpf with the default backend (stap) and compare some differences in functionality between these two backends.

Continue reading “Introducing stapbpf – SystemTap’s new BPF backend”


Find what capabilities an application requires to successful run in a container

Many developers would like to run their existing applications in a container with restricted capabilities to improve security. However, it may not be clear which capabilities the application uses because the code uses libraries or other code developed elsewhere. The developer could run the application in an unrestricted container that allows all syscalls and capabilities to be used to avoid possible hard to diagnose failures caused by the application’s use of forbidden capabilities or syscalls. Of course, this eliminates the enhanced security of restricted containers. At Red Hat, we have developed a SystemTap script (container_check.stp) to provide information about the capabilities that an application uses. Read the SystemTap Beginners Guide for information on how to setup SystemTap.

Continue reading “Find what capabilities an application requires to successful run in a container”


Red Hat Developer Toolset 3.1 now generally available

gnu logoToday, Red Hat has announced the general availability of Red Hat Developer Toolset 3.1. Available through the Red Hat Enterprise Linux Developer Program and related subscriptions, Red Hat Developer Toolset 3.1 streamlines application development on the world’s leading enterprise Linux platform, enabling developers to compile applications once and deploy across multiple versions of Red Hat Enterprise Linux.

Continue reading Red Hat Developer Toolset 3.1 now generally available


Using System Tap to test the GNU C Library

gnu logoWhite box testing?

Traditional white box testing verifies the internal implementation details of the software under test. As of today the GNU C Library (glibc) has very little if any white box testing. The general policy has been that we implement standards conforming interfaces and that as such we need to test those interfaces. This is a good start, but we need to test more if we are going to cover all cases and configurations, and this includes more detailed failure path testing.

Continue reading “Using System Tap to test the GNU C Library”


Malloc systemtap probes: an example

gnu logoOne feedback I got from my blog post on Understanding malloc behavior using Systemtap userspace probes was that I should have included an example script to explain how this works. Well, better late than never, so here’s an example script. This script prints some diagnostic information during a program run and also logs some information to print out a summary at the end. I’ll go through the script a few related probes at a time.

global sbrk, waits, arenalist, mmap_threshold = 131072, heaplist

Continue reading “Malloc systemtap probes: an example”


Understanding malloc behavior using Systemtap userspace probes

The malloc family of functions are critical for almost every serious application program. Its performance characteristics often have a big impact on the performance of applications. Given that the default malloc implementation needs to have consistent performance for all general cases, it makes available a number of tunables that can help developers tweak its behavior to suit their programs.

About two years ago I had written an article on the Red Hat Customer Portal that described the high level design of the GNU C Library memory allocator and also introduced the reader to various magic environment variables that malloc understands to change its behavior. The behavior documented in that article and the tricks to tweak malloc behavior hold just as true for RHEL-7, which is based on upstream glibc 2.17 as they did for RHEL-6, which is based on upstream glibc 2.12.

Continue reading “Understanding malloc behavior using Systemtap userspace probes”