Carlos O'Donell

Carlos O'Donell

Recent Posts

A platform interface for the GNU C Library

A platform interface for the GNU C Library

codonell
By Carlos O'Donell

Application developers continue to need newer versions of libraries, including core runtimes like GNU C Library (glibc), for their applications. In this article, I’ll look at some issues related to upgrading glibc in an operating system (OS) distribution, and I also encourage you to read Florian Weimer’s excellent blog post on the topic.

The problem

Deciding between a library rebase or continued backporting of commits involves a complex set of risks and rewards. For some customers and users, it is important not to rebase the library (ensuring the lowest risk of impact by change); but for others, the rebase brings valuable bug fixes (lowest risk of impact from known issues). In other cases, the newer library may perform better, even if the interfaces haven’t changed, because it can take advantage of newer hardware or a newer Linux kernel (performance advantage to first mover).

There is no way to simultaneously satisfy all the requirements of slow-moving versus fast-moving development. The recent work in Fedora Modularity is aimed at solving the root of this problem, but there is a limit to this work. The further down the stack you go, the harder the problem becomes. The potential for breakage further up the stack increases. You can’t always arbitrarily change a component’s installed version without consequences, either at build time or at runtime.

Continue reading “A platform interface for the GNU C Library”

Share
C/C++ library upgrades and opaque data types in process shared memory

C/C++ library upgrades and opaque data types in process shared memory

codonell
By Carlos O'Donell

The problem

C/C++ libraries expect to be able to change the internal implementation details of opaque data types from release to release since such a change has no external ABI consequences. If an opaque data type is placed in process-shared memory (when allowed by the standard) and shared with multiple processes, each process must ensure they are using exactly the same version of the library or they could fail in unexpected ways during library upgrades. The placement of opaque data types in process-shared memory is never allowed unless otherwise stated by the library documentation. For the GNU C Library (glibc) you may place pthread_mutex_t, pthread_cond_t, and sem_t in process-shared memory as allowed by POSIX. Failures using these types occur because a process started more recently may have a newer version of the library for the type and that version may have a different understanding of the internal details of the type. The problem has always been one for the developer to solve, but without help, this problem is so intractable as to make it difficult to robustly use opaque data types in process shared memory.

We will cover opaque data types, what they are, why you would use them, and how library upgrades play into the problem, and what might be done by the application developer.

Continue reading “C/C++ library upgrades and opaque data types in process shared memory”

Share
Dirty Tricks: Launching a helper process under memory and latency constraints (pthread_create and vfork)

Dirty Tricks: Launching a helper process under memory and latency constraints (pthread_create and vfork)

codonell
By Carlos O'Donell

You need to launch a helper process, and while Linux’s fork is copy-on-write (COW), the page tables still need to be duplicated, and for a large virtual address space that could result in running out of memory and performance degradation. There are a wide array of solutions available to use, but one of them, namely vfork is mostly avoided due to a few difficult issues. First is that vfork pauses the parent thread while the child executes and eventually calls an exec family function, this is a huge latency problem for applications. Secondly is that there are a great many number of considerations to take into account when using vfork in a threaded application, and missing any one of those considerations can lead to serious problems.

It should be possible for posix_spawn to safely do all of this work via POSIX_SPAWN_USEVFORK, but often there is quite a lot of “work” that needs to be done just before the helper calls an exec family function, and that has lead to ever increasingly complex versions of posix_spawn like posix_spawn_file_actions_addclose, posix_spawn_file_actions_adddup2, posix_spawn_file_actions_destroy, posix_spawnattr_destroy, posix_spawnattr_getsigdefault, posix_spawnattr_getflags, posix_spawnattr_getpgroup, posix_spawnattr_getschedparam, posix_spawnattr_getschedpolicy, and posix_spawnattr_getsigmask. It might be simpler if the GNU C Library documented a small subset of functions you can safely call, which is in fact what the preceding functions are modelling. If you happen to select a set of operations that can’t be supported by posix_spawn with vfork then the implementation falls back to fork and you don’t know why. Therefore it is hard to use posix_spawn robustly.

Continue reading “Dirty Tricks: Launching a helper process under memory and latency constraints (pthread_create and vfork)”

Share
Using System Tap to test the GNU C Library

Using System Tap to test the GNU C Library

codonell
By Carlos O'Donell

gnu logoWhite box testing?

Traditional white box testing verifies the internal implementation details of the software under test. As of today the GNU C Library (glibc) has very little if any white box testing. The general policy has been that we implement standards conforming interfaces and that as such we need to test those interfaces. This is a good start, but we need to test more if we are going to cover all cases and configurations, and this includes more detailed failure path testing.

Continue reading “Using System Tap to test the GNU C Library”

Share

What’s new in the core C, math, and thread libraries for Red Hat Enterprise Linux 7?

codonellredhattoolchainblog
By Carlos O'Donell and Matt Newsome

With the recent release of Red Hat Enterprise Linux 7, we have some great new features to pass along. In this post we walk through the core C, math, and thread libraries and see what is new for developers.

The GNU C Library

The GNU C Library, or “glibc” as we like to call it, is a core component of Red Hat Enterprise Linux 7 and provides several key OS components including the core ISO C functionality (including the math library), and both BSD and POSIX APIs (including the thread library). The project provides the vast majority of the low-level interfaces that developers use day in and day out with the Linux kernel. You might expect that not much would change, but actually Red Hat Enterprise Linux 7 glibc introduces a large number of changes and improvements. In this article we’ll walk through these.

Continue reading “What’s new in the core C, math, and thread libraries for Red Hat Enterprise Linux 7?”

Share