kernel

A platform interface for the GNU C Library

A platform interface for the GNU C Library

Application developers continue to need newer versions of libraries, including core runtimes like GNU C Library (glibc), for their applications. In this article, I’ll look at some issues related to upgrading glibc in an operating system (OS) distribution, and I also encourage you to read Florian Weimer’s excellent blog post on the topic.

The problem

Deciding between a library rebase or continued backporting of commits involves a complex set of risks and rewards. For some customers and users, it is important not to rebase the library (ensuring the lowest risk of impact by change); but for others, the rebase brings valuable bug fixes (lowest risk of impact from known issues). In other cases, the newer library may perform better, even if the interfaces haven’t changed, because it can take advantage of newer hardware or a newer Linux kernel (performance advantage to first mover).

There is no way to simultaneously satisfy all the requirements of slow-moving versus fast-moving development. The recent work in Fedora Modularity is aimed at solving the root of this problem, but there is a limit to this work. The further down the stack you go, the harder the problem becomes. The potential for breakage further up the stack increases. You can’t always arbitrarily change a component’s installed version without consequences, either at build time or at runtime.

Continue reading “A platform interface for the GNU C Library”

Share
Using eXpress Data Path (XDP) maps in RHEL 8: Part 2

Using eXpress Data Path (XDP) maps in RHEL 8: Part 2

Diving into XDP

In the first part of this series on XDP, I introduced XDP and discussed the simplest possible example. Let’s now try to do something less trivial, exploring some more-advanced eBPF features—maps—and some common pitfalls.

XDP is available in Red Hat Enterprise Linux 8, which you can download and run now.

Continue reading “Using eXpress Data Path (XDP) maps in RHEL 8: Part 2”

Share
Achieving high-performance, low-latency networking with XDP: Part I

Achieving high-performance, low-latency networking with XDP: Part I

XDP: From zero to 14 Mpps

In past years, the kernel community has been using different approaches in the quest for ever-increasing networking performance. While improvements have been measurable in several areas, a new wave of architecture-related security issues and related counter-measures has undone most of the gains, and purely in-kernel solutions for some packet-processing intensive workloads still lag behind the bypass solution, namely Data Plane Development Kit (DPDK), by almost an order of magnitude.

But the kernel community never sleeps (almost literally) and the holy grail of kernel-based networking performance has been found under the name of XDP: the eXpress Data Path. XDP is available in Red Hat Enterprise Linux 8, which you can download and run now.

Continue reading “Achieving high-performance, low-latency networking with XDP: Part I”

Share
Analyzing Changes to the Binary Interface Between the Linux Kernel and its Modules

Analyzing Changes to the Binary Interface Between the Linux Kernel and its Modules

This article is for people interested in long-term Linux kernel maintenance. It introduces you to tools that can help keep the binary interfaces between the kernel and its loadable modules stable during the entire lifetime of a supposedly stable kernel, while the code is modified. As these tools are essentially analysis tools, they can be used not only by kernel developers, but also by quality assurance engineers and advanced kernel users (system programmers).

Upstream in-tree kernel modules: the ideal situation

In the canonical development model of the Linux kernel, the source code of all dynamically loaded modules is hosted alongside the source code of the core kernel. In this model, whenever the core kernel changes the interface it exposes to its modules, the compilers detects that the interface changed, making it easy to adjust the code of the modules accordingly.

Continue reading “Analyzing Changes to the Binary Interface Between the Linux Kernel and its Modules”

Share
The need for speed and the kernel datapath – recent improvements in UDP packets processing

The need for speed and the kernel datapath – recent improvements in UDP packets processing

Networking hardware is becoming crazily fast, 10Gbs NICs are entry-level for server h/w, 100Gbs cards are increasingly popular and 200Gbs are already surfacing. While the Linux kernel is striving to cope with such speeds with large packets and all kind of aggregation, ISPs are requesting much more demanding workload with NFV and line rate packet processing even for 64 bytes packets.

Is everything lost and are we all doomed to rely on some kernel bypass solution? Possibly, but let’s first inspect what is really the current status for packet processing in the kernel data path, with a perspective look at the relevant history and the recent improvements.

We will focus on UDP packets reception: UDP flood is a common tool to stress the networking stack allowing arbitrary small packets and defeating packet aggregation (GRO), in place for other protocols.

Continue reading “The need for speed and the kernel datapath – recent improvements in UDP packets processing”

Share
Direct Kernel Open vSwitch Flow Programming

Direct Kernel Open vSwitch Flow Programming

Typically, users will interact with the Open vSwitch kernel datapath by way of the ‘ovs-ofctl’ utility to program OpenFlow rules into the ‘ovs-vswitchd’. However, this isn’t the only mechanism for forwarding packets via the openvswitch kernel module. An additional direct flow-programming interface is available using the ‘ovs-dpctl’ utility to add flows to the kernel. This post will cover influencing the movement of packets through the openvswitch kernel module using the ‘ovs-dpctl’ utility.

Continue reading “Direct Kernel Open vSwitch Flow Programming”

Share

The Benefits of Red Hat Enterprise Linux for Real Time

To deliver the best of predictability on real-time workloads, the Red Hat Enterprise Linux for Real Time provides state-of-art on determinism for the bullet-proof RHEL (Red Hat Enterprise Linux) platform. The availability of this product raises some questions, like: Do I need a real-time operating system? Or What are the benefits and drawbacks of running the RHEL for Real Time? This article aims to clarify how to leverage the success of your business using a real-time operating system, and what kind of workloads or type of industry can benefit from RT.

Introduction

Linux is the best choice for High-Performance Computing (HPC) due to the years of Linux kernel optimization focused on delivering high average throughput for a vast number of different workloads. Being optimized for throughput means that the algorithms for processing data are geared towards processing the most amount of data in the least amount of time. Examples of throughput-oriented operations are transferring megabytes/second over a network connection or the amount of data read from or written to a storage medium. These optimizations are the basis for the success of RHEL on servers and HPC environments.

Nevertheless, these optimizations for high throughput can cause drawbacks on other specific workloads. For example, the RHEL kernel uses a busy loop wait approach to avoid the scheduling overhead on some mutual exclusion methods, like spin locks and read/write locks. While busy waiting to enter in a critical section of code, the task waiting delays the scheduling of other potentially higher priority tasks in the same CPU. As a result, the higher priority task can not be scheduled and executed until the waiting task has completed, causing a delay on the high priority task’s response.

Although this delay is acceptable for the majority of common workloads, it is not acceptable for the class of tasks where correctness depends on meeting timing deadlines. This class of tasks, often classified as real-time tasks, has strict timing constraints where an answer must be delivered within a certain time period, and a late answer means it is wrong or fails.

For example, processing a 30 frames per second video requires the ability to deliver one frame every 33 milliseconds. If the system fails to deliver a frame every 33 milliseconds, the video processing will not be only late, but also wrong. It is natural to think then that real-time means delivering a quick response to an event and assume that real-time can be achieved only by making the system run faster. This assumption is a misconception, however. For instance, if the above-mentioned system can run fast enough to deliver a frame every 16.6 ms (60 frames per seconds), the video will be reproduced twice as fast. A faster response is not the expected behavior for processing this video, so the system will deliver not only early results but also wrong results. Hence, real-time systems are those that deliver a predictable timing behavior instead of just trying to deliver faster results.

To provide an enterprise environment for real-time workloads on Linux, Red Hat provides the RHEL for Real Time product. RHEL for Real Time is composed of the RHEL kernel optimized for determinism, along with a set of integrated tuning tools to provide the state-of-art of determinism on Linux. The deterministic timing behavior depends on both an application’s determinism using its own algorithms and on the Linux kernel determinism in managing the systems shared resources.

Continue reading “The Benefits of Red Hat Enterprise Linux for Real Time”

Share

LTTng Packages now Available for Red Hat Enterprise Linux 7

EfficiOS is pleased to announce it is now providing LTTng packages for Red Hat Enterprise Linux 7, available today as part of its Enterprise Packages portal.

EfficiOS specialises in the research and development of open source performance analysis tools. As part of its activities, EfficiOS develops the Linux Tracing Toolkit: next generation for which it provides enterprise support, training and consulting services.

What is tracing?

Tracing is a technique used to understand the behaviour of a software system. In this regard, it is not far removed from logging. However, tracers and loggers are designed to accommodate very different use cases.

Continue reading “LTTng Packages now Available for Red Hat Enterprise Linux 7”

Share