Optimize RHEL for edge and IoT deployments

Deploying a full-featured Linux distribution such as Red Hat Enterprise Linux (RHEL) on resource-constrained edge or Internet of Things (IoT) devices presents unique challenges. While image mode for RHEL offers immutability and customization, further optimization is often required to meet strict memory and storage limitations.

Initial considerations for system minimization

The first step in building a minimal system involves reducing both memory and storage footprints. The methods employed will vary depending on the desired level of optimization. However, it is important to highlight that many of these methods involve altering system components in ways that can complicate debugging and impact supportability, especially when modifying files installed via RPM packages, as this compromises package integrity.

Identifying major space consumers

Typical targets for reduction in a RHEL environment include:

1. Installation-level customizations
   • Custom partitioning strategies for more efficient disk use.
   • Minimal package selection.
   • Elimination of unnecessary dependencies.
2. Post-installation optimizations
   • Limiting the number of retained kernel versions.
   • Removing leftover temporary files.
   • Configuring log rotation and compression.
   • Removing unused locales and documentation (/usr/share/doc, /usr/share/man, etc.)
3. Enforcing clean state across updates
   • Use of custom RPMs with %post scripts to reapply optimizations.
   • Automation with Ansible Playbooks for post-deployment configuration.

Additional considerations for RAM optimization

To further reduce memory usage:

• Disable unneeded services and daemons.
• Replace full desktop environments like GNOME with lighter alternatives or minimal UI frameworks.
• Disable autostart for device hotplug daemons.
• Use static network configurations to eliminate the need for NetworkManager.

Navigating trade-offs in optimization

While standard RHEL deployments aim for flexibility and compatibility, edge use cases often benefit from eliminating unnecessary components. However, many applications are compiled with additional features, pulling in unused dependencies. Since recompiling the entire software stack is impractical and unsupported, the focus should remain on minimizing the package set and reducing the attack surface at the same time due to this reduction of installed software.

BaseOS installation and kickstart automation

To streamline deployments, begin with the @Core group in the kickstart configuration, adding only essential utilities, such as SSH. Including certain tools may inadvertently pull additional dependencies (e.g., Kerberos with OpenSSH). Avoiding these requires custom-built binaries, which complicates maintenance and supportability.

After finalizing a minimal package list, you can use it for kickstart and container image creation. Container base images may include more tooling, requiring additional cleanup post-deployment.

Package and service reduction

A non-exhaustive list of packages that may be safely removed include:

• fwupd: Firmware update daemon

• tracker*: File indexing services

• nano: Text editor

• wireplumber: Audio support

• sssd*: Security Services Daemon

• ntpd: NTP service

• NetworkManager-*: Network management daemon

• langpacks-*: Extra language support

• pipewire: Audio stack

• man-db: Manual pages database

• kexec-tools: Kernel crash tools

• firmware: Hardware firmware blobs

• insights-client: System Insights client

• *devel*: Development headers and libraries

Removing documentation and locales

To further reduce disk usage:

rm -Rfv /usr/share/info/ /usr/share/man/ /usr/share/doc/
/usr/share/locale/

Disable unnecessary services, such as:

systemctl disable rhsmcertd

However, you must be aware of trade-offs. Disabling RHSM will block access to official repositories, removing chronyd will stop time synchronization, and disabling NetworkManager will prevent the system from responding to network changes.

GUI alternatives

For graphical environments, you can replace or omit GNOME, depending on application requirements. Alternatives like Weston (available via EPEL) may be appropriate in locked-down scenarios.

Kickstart scripting and automation

Kickstart supports automation via %pre and %post sections and is useful for tasks, such as static IP configuration or system naming. However, debugging can be complex. Configuration management tools like Ansible are often preferable for ongoing configuration.

Here is a sample Kickstart snippet:

%packages
@^minimal-environment
%end
firstboot --disable
timezone Europe/Madrid --utc
rootpw --iscrypted --allow-ssh <your-password-hash>

You can handle the post-installation configuration via Ansible for flexibility and version control.

Persistent cleanup via RPM triggers

To prevent RPM upgrades from restoring removed files, use RPM triggers. For example, deploy a cleanup script:

/usr/bin/systemtrimdown.sh:

#!/bin/bash
rm -Rfv /usr/share/info/ /usr/share/man/ /usr/share/doc/
/usr/share/locale/

Create an RPM spec file:

Name:           systemtrimdown
Version:        1.0
Release:        1%{?dist}
Summary:        Executes cleanup after selected package installations
License:        GPL
BuildArch:      noarch
%files
%triggerin -- linux-firmware
sh /usr/bin/systemtrimdown.sh

Build the package:

rpmbuild -bb systemtrimdown.spec

Configuration via Ansible

Ansible Playbooks simplify reapplication and tracking of changes. A sample playbook might include the following:

- hosts: all
  tasks:
    - name: Remove rescue images
      file: path={{ item }} state=absent
      loop: "{{ query('fileglob', '/boot/*rescue*') }}"
    - name: Remove unneeded packages
      package:
        name: "{{ item }}"
        state: absent
      with_items:
        - abrt*
        - langpacks-*
        - pipewire
    - name: Configure journald
      ini_file:
        path: /etc/systemd/journald.conf
        section: Journal
        option: SystemMaxUse
        value: 200M

Results

After deploying RHEL 9.4 and applying the described optimizations, disk usage reduced significantly, as shown in the following table:

This 33% reduction in disk usage allows for better use of limited storage resources, such as fitting an entire OS into a 2 GB SD card, making it suitable for remote deployments or IoT applications.