Article

Tuned: the tuning profile delivery mechanism for RHEL

 

July 15, 2015
Jeremy Eder

What is "Tune-D" ?

Tuned is a tuning profile delivery mechanism included in Red Hat Enterprise Linux.  As demonstrated by D. John Shakshober (aka Shak) at Red Hat Summit, tuned improves performance for most workloads by quite a bit.  What's a tuning profile, you ask?  Using the throughput-performance profile (enabled by default in RHEL7) as an example:

tuned-throughput-performance

 

These settings tune RHEL for the datacenter, whether public cloud, or private.  You can easily create your own profiles, too!

Red Hat delivers tuned profiles for most of our product portfolio:

  • Red Hat Enterprise Linux 6 and 7, Workstation and Server
  • Red Hat Enterprise Linux KVM
  • Red Hat Enterprise Virtualization
  • Red Hat Storage (Gluster)
  • Red Hat Enterprise Linux OpenStack Platform
  • Red Hat Enterprise Linux Atomic Host
  • Red Hat OpenShift Enterprise v3
  • Red Hat Enterprise Linux for Realtime

Profile Inheritance

Profiles are applied by automatically based on the product and infrastructure, and inherit settings from each other, ensuring that "RHEL feels like RHEL".  Each profile customizes the operating system for expected workloads.  For example, the Atomic profiles tune the OS for container scaling.

tuned-profile-inheritance-atomic

The new whizbang...

Over the past few years, tuned has gained traction and features rapidly.  With the help of maintainer Jaroslav Skarvada (and others), the latest release of tuned-2.5.0 is no different!  Look at all the great new features:

We have several new profiles:

By customer request, features have been added to:

  • Support running tuned in a daemonless mode, a long-desired feature that was requested by the field to:
    • Reduce the need to schedule the tuned process.
    • Reduce the memory footprint of tuned (~15MB).
  • You can now set ethtool coalescing and NIC offloads with tuned.  This will help when we have use-cases that for example, need a particular offload enabled in order to perform best...*cough* vxlan *cough*.
  • Function plugins (this provides ultimate flexibility, analogous to RHEV hooks).
  • A "verify" feature which ensures that tuned profiles settings are actually applied.
  • Enablement for RT and NFV use-cases, including:
    • cpumask support
    • inverse-cpu-mask support
    • Got rid of the need for mce=ignore_ce on the cmdline by adding sysfs glob support.
    • We can automatically calculate kernel cmdline options based on variables.conf content set by user (for example, isolated_cores=2-23 means that grub should be updated with isolcpus=2-23, and nohz_full 2-23 etc etc)...
  • Extended the scheduler plugin to affine new processes.  This works by watching exec() calls using perf hooks to provide immediate tuning of new processes that do not have affinity set.  This helps us guard against latency spikes caused by (for example) someone ssh'ing in and running top.  That bash and top processes would be automatically affined by tuned.  This is again, a long-requested feature.  Thanks go to Clark Williams for the perf-hook concept and to Arnaldo Carvalho de Melo (aka acme) for letting us borrow some code from tuna ;)

To enable some of these profiles, Red Hat has also extended irqbalance to automatically respect isolcpus (thanks to Rik van Riel).

Continuing towards the goal of completely isolating CPUs from kernel interference, we've extended tuna to handle nohz_full cores, and create a "kthread and irq sweeper" mode.  This mode will traverse the process table and IRQ list and apply the desired affinity/isolation ensuring that the kernel provides maximum CPU cycles to userspace.  Go acme, go!

 

 

cyclictest

Committed to Top Performance

Tuned demonstrates Red Hat's ongoing commitment to engineer our product stack to be the most performant platforms to run your business.  We hope that you enjoy it!

Last updated: March 15, 2023

Comments