Jeremy Eder

Recent Posts

Docker project: Can you have overlay2 speed and density with devicemapper? Yep.

It’s been a while since our last deep-dive into the Docker project graph driver performance.  Over two years, in fact!  In that time, Red Hat engineers have made major strides in improving container storage:

All of that, in the name of providing enterprise-class stability, security and supportability to our valued customers.

As discussed in our previous blog, there are a particular set of behaviors and attributes to take into account when choosing a graph driver.  Included in those are page cache sharing, POSIX compliance and SELinux support.

Reviewing the technical differences between a union filesystem and devicemapper graph driver as it relates to performance, standards compliance and density, a union filesystem such as overlay2 is fast because

  • It traverses less kernel and devicemapper code on container creation (devicemapper-backed containers get a unique kernel device allocated at startup).
  • Containers sharing the same base image startup faster because of warm page cache
  • For speed/density benefits, you trade POSIX compliance and SELinux (well, not for long!)

There was no single graph driver that could give you all these attributes at the same time — until now.

How we can make devicemapper as fast as overlay2

With the industry move towards microservices, 12-factor guidelines and dense multi-tenant platforms, many folks both inside Red Hat as well as in the community have been discussing read-only containers.  In fact, there’s been a –read-only option to both the Docker project, and kubernetes for a long time.  What this does is create a mount point as usual for the container, but mount it read-only as opposed to read-write.  Read-only containers are an important security improvement as well as they reduce the container’s attack surface.  More details on this can be found in a blog post from Dan Walsh last year.

When a container is launched in this mode, it can no longer write to locations it may expect to (i.e. /var/log) and may throw errors because of this.  As discussed in the Processes section of, re-architected applications should store stateful information (such as logs or web assets) in a stateful backing service.  Attaching a persistent volume that is read-write fulfills this design aspect:  the container can be restarted anywhere in the cluster, and its persistent volume can follow it.

In other words, for applications that are not completely stateless an ideal deployment would be to couple read-only containers with read-write persistent volumes.  This gets us to a place in the container world that the HPC (high performance/scientific computing) world has been at for decades:  thousands of diskless, read-only NFS-root booted nodes that mount their necessary applications and storage over the network at boot time.  No matter if a node dies…boot another.  No matter if a container dies…start another.

Continue reading “Docker project: Can you have overlay2 speed and density with devicemapper? Yep.”


Tuned: the tuning profile delivery mechanism for RHEL

What is “Tune-D” ?

Tuned is a tuning profile delivery mechanism included in Red Hat Enterprise Linux.  As demonstrated by D. John Shakshober (aka Shak) at Red Hat Summit, tuned improves performance for most workloads by quite a bit.  What’s a tuning profile, you ask?  Using the throughput-performance profile (enabled by default in RHEL7) as an example:



These settings tune RHEL for the datacenter, whether public cloud, or private.  You can easily create your own profiles, too!

Red Hat delivers tuned profiles for most of our product portfolio:

Continue reading “Tuned: the tuning profile delivery mechanism for RHEL”


Can you run Intel’s Data-plane Development Kit (DPDK) in a Docker container? Yep.

nicubunu_PackageAs part of our participation in hundreds of open source communities, Red Hat engineers are often involved in research and development efforts that may or may not become a part of Red Hat’s supported offerings.

Intel’s Data-plane Development Kit (DPDK) is a set of libraries and drivers for Linux and BSD built for fast packet processing, for the burgeoning “Network Function Virtualization“, or NFV discipline.  Typical verticals interested in turning Linux boxes into packet-processing machines are telecom, financial services, military, energy research, datacenter operators, internet service providers and many more.

Continue reading “Can you run Intel’s Data-plane Development Kit (DPDK) in a Docker container? Yep.”


Accelerating Red Hat Enterprise Linux 7-based Linux Containers with Solarflare OpenOnload

RH_Icon_Container_with_App_FlatLinux Containers combine well-established Linux kernel technologies such as namespaces, SELinux, cgroups and iptables with incredible ease of use and exceptional performance.

For customers looking for the lowest possible network latencies and reduced CPU overhead coupled with the deployment advantages of Linux containers, Red Hat’s new Accelerating Red Hat Enterprise Linux 7-based Linux Containers with Solarflare OpenOnload whitepaper provides installation, configuration and tuning guidance for Docker containers running on Red Hat Enterprise Linux with Solarflare OpenOnload network acceleration.

Continue reading “Accelerating Red Hat Enterprise Linux 7-based Linux Containers with Solarflare OpenOnload”

Introducing the "rhel-tools" for RHEL Atomic Host

Introducing the "rhel-tools" for RHEL Atomic Host

RH_Icon_Container_with_App_FlatThe rise of the purpose-built Linux distribution

Recently, several purpose-built distributions have been created specifically to run Linux containers.  There seem to be more popping up every day.  For our part, in April 2014 at the Red Hat Summit, Red Hat announced its intention to deliver a purpose-built, container-optimized version of Red Hat Enterprise Linux 7 called RHEL Atomic Host.  After over a year in the making, we are excited that launch day has finally come!

What’s important to know about Red Hat Enterprise Linux Atomic Host, you ask?  Well, plenty…but for the sake of this blog, I’ll stick to areas I know as a performance engineer:

  • RHEL Atomic leverages years of engineering effort that went into RHEL7.
  • It uses the same exact kernel as RHEL7.
  • Significantly reduced on-disk and in-memory footprint.
  • Utilizes OSTree technology for upgrades and rollbacks.
  • Optimized device-mapper container storage performance out of the box.
  • Optimized container scalability out of the box.
  • Includes purpose-built rhel-tools container for system administration tasks

Continue reading “Introducing the "rhel-tools" for RHEL Atomic Host”

Low Latency Performance Tuning for Red Hat Enterprise Linux 7

Low Latency Performance Tuning for Red Hat Enterprise Linux 7

velocimetroCounting micro-nanoseconds?  We are, because we know our customers are.  Some of the world’s largest stock exchanges including the Chicago Mercantile Exchange (CME), New York Stock Exchange (NYSE), E*TRADE, Union Bank, countless hedge funds and high-frequency trading shops run on Red Hat’s products.  In fact, the majority of the world’s financial transactions are executed with Red Hat Enterprise Linux in the critical path.

Continue reading “Low Latency Performance Tuning for Red Hat Enterprise Linux 7”


Beyond Microbenchmarks: breakthrough container performance with Tesla efficiency

Back story

As virtualization was beginning it’s march to prominence, we saw a phased approach to adoption.  This is common with any sort of game changing technology….let’s take electric cars as an example.  Early adopters are willing to make certain trade-offs (short range) to gain new capabilities (saving money at the gas station).

teslaIn the meantime, engineers are off in the lab working hard to increase the possible consumer-base for electric cars by increasing range, decreasing charging cycle times, and improving performance.  Taken in aggregate, those changes are meant to address objections to the first-cut of the technology.

Virtualization is to Linux containers is to…

Continue reading “Beyond Microbenchmarks: breakthrough container performance with Tesla efficiency”

Comprehensive Overview of Storage Scalability in Docker

Comprehensive Overview of Storage Scalability in Docker


First, a brief backstory on the storage situation for Docker since it was open-sourced in early 2013.  At that time, Docker relied on a filesystem called AUFS (advanced multi layered unification filesystem).  This Union filesystem provided the necessary features to support several of Docker’s main selling points:


Continue reading “Comprehensive Overview of Storage Scalability in Docker”


Performance Analysis of Docker on Red Hat Enterprise Linux 7

Containers introduce some intriguing usability, packaging and deployment patterns. These new patterns offer the potential to effect massive improvements to the enterprise application development and operations specialties. Containers also offer the promise of bare metal performance while offering some amount of isolation as well.

But can they deliver on that promise ?

Continue reading “Performance Analysis of Docker on Red Hat Enterprise Linux 7”