Linux Unified Key Setup (LUKS) and FIPS are essential tools for system administrators managing secure environments. However, when it is time to upgrade the operating system, these security features can become significant obstacles. Red Hat Enterprise Linux (RHEL) operates on a predictable 3-year major release cycle with each major version supported for ten years. When the time comes to transition to the next major version, leapp serves as the officially supported tool to perform in-place system upgrades, allowing organizations to retain their original subscriptions, system configurations, and custom repositories without having to completely reinstall machines.
Limitations
However, leapp has a known limitation: it officially will not work if the system uses disk encryption for the root filesystem. This hurdle can leave environments stranded, overly reliant on extended life cycle offerings just to keep systems active and supported.
Fortunately, a clever workaround exists. Administrators can utilize the system's TPM2 module to temporarily bypass the lock while Leapp executes the upgrade. The following outlines a basic workflow for upgrading a RHEL 8 system to RHEL 9 using leapp when FIPS and LUKS are both configured.
Caution: This process design is for partition-level encryption. If you utilize individually encrypted logical volumes, this method could fail to proceed past the reboot during the leapp upgrade process.
LUKS and LVM strategy
Before diving into the upgrade process, it is helpful to understand the role of Linux Unified Key Setup (LUKS) and the optimal strategy for deploying it. LUKS is the standard encryption specification for Linux, designed to protect data at rest. By encrypting the storage medium, LUKS ensures that if a physical drive is ever compromised or stolen, the contents remain completely inaccessible without the correct passphrase or cryptographic key.
When configuring encrypted storage with Logical Volume Management (LVM), system administrators must choose whether to apply LUKS to the underlying physical partition (with LVM built on top) or to encrypt individual logical volumes separately. Applying LUKS directly to the raw partition or drive is widely recognized as the best practice. This configuration encrypts the entire LVM physical volume, ensuring that all subsequent logical volumes, filesystem metadata, and even swap space are uniformly protected under a single cryptographic umbrella.
Furthermore, encrypting the raw partition simplifies the boot process by requiring only a single unlock operation for the entire storage stack. Conversely, encrypting individual logical volumes requires the unlocking of each volume independently. This drastically complicates automation, increases administrative overhead, and, most importantly for this scenario, breaks compatibility with the automated reboot sequences required by the leapp upgrade tool.
Set the stage
First, it is crucial to back up all data and equip the system (either a 40GB VM or a physical box) with a TPM2 chip. The hard drive must be fully LUKS encrypted. You can configure FIPS can if required by local security policies. You can verify the environment as follows.
Check the OS version:
# cat /etc/redhat-release
Red Hat Enterprise Linux release 8.10 (Ootpa)Check FIPS status by running the following command to ensure it returns enabled:
# fips-mode-setup --check
FIPS mode is enabled.Confirm TPM2 is present by running the following and looking for an output of 2:
# cat /sys/class/tpm/tpm0/tpm_version_majorCheck the storage layout. In this example, /dev/sda2 is the encrypted partition providing protection across all logical volumes.
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 40G 0 disk
├─sda1 8:1 0 1G 0 part /boot
└─sda2 8:2 0 39G 0 part
└─luks-95ef6b14-df5c-4400-af81-621f6271adf4 253:0 0 39G 0 crypt
├─rhel-root 253:1 0 25G 0 lvm /
├─rhel-swap 253:2 0 4G 0 lvm [SWAP]
└─rhel-var 253:3 0 10G 0 lvm /varThe pre-upgrade reality check
Using leapp is a two-phase process: an upgradability analysis and the actual upgrade. The following steps outline how to install the tools and perform the initial checks.
Install the necessary packages:
# yum install -y leapp leapp-upgrade-el8toel9Run the pre-upgrade check:
# leapp preupgradeThe system remains unmodified, but leapp creates important report files (e.g., /var/log/leapp/leapp-report.txt) which plot the upgrade path. A review of the report overview will likely reveal a few inhibitors preventing the upgrade, most notably the presence of LUKS devices.
The magic bind: Solving the LUKS inhibitor
To bypass the LUKS inhibitor, the TPM2 module must temporarily handle the unlock duties, so the system can reboot and unlock itself without requiring manual password entry.
First, install the Clevis packages:
# yum install -y clevis-luks clevis-dracutBind a new LUKS key to the TPM module by running:
# clevis luks bind -d /dev/sda2 tpm2 / '{"pcr_bank":"sha256","pcr_ids":"7"}'When prompted, you must enter the existing LUKS password created during installation.
Verify the key linked successfully:
# clevis luks list -d /dev/sda2
1: tpm2 '{"hash":"sha256","key":"ecc","pcr_bank":"sha256","...Regenerate the initramfs file so the boot process registers this new setup:
# dracut -fv --regenerate-allFinally, reboot the system:
# rebootNote: If the LUKS password box appears during bootup, you should not enter the password. The screen will eventually advance to the login screen on its own.
Squash remaining inhibitors
With the LUKS hurdle cleared, running the preupgrade check one more time should confirm that the LUKS inhibitor is completely gone from the report. A leapp upgrade will absolutely not proceed if any inhibitors remain on the system.
# leapp preupgradeThe output will list blocking items, as well as in the /var/log/leapp/leapp-report.txt file. Before proceeding, any remaining inhibitors need to be resolved. You should review the leapp report carefully, and follow the recommended remediation steps for any remaining flags to ensure the system is fully prepped for the jump to RHEL 9.
Taking the leapp
Once the preupgrade report returns completely clean, the upgrade can commence. Run the upgrade script with the reboot flag as follows:
# leapp upgrade --rebootThis command triggers the upgrade process. The bootloader automatically initializes a special upgrade environment using the menu entry Upgrade-Initramfs.
This step requires patience since it swaps out packages and updates configurations. During the reboot, the LUKS drive will unlock automatically via the TPM2 bound key without requiring a passphrase. Once the system is back online, you can verify the fresh RHEL 9 installation.
Check the OS version:
# cat /etc/redhat-release
Red Hat Enterprise Linux release 9.8 (Plow)Ensure FIPS is still active:
# fips-mode-setup --checkFIPS mode is enabled.Verify the storage is intact:
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 40G 0 disk
├─sda1 8:1 0 1G 0 part /boot
└─sda2 8:2 0 39G 0 part
└─luks-95ef6b14-df5c-4400-af81-621f6271adf4 253:0 0 39G 0 crypt
├─rhel-root 253:1 0 25G 0 lvm /
├─rhel-swap 253:2 0 4G 0 lvm [SWAP]
└─rhel-var 253:3 0 10G 0 lvm /varPost-flight cleanup
Do not leave the TPM2 key indefinitely. Once the upgrade is complete, you should remove the bound key so the original passphrase becomes the sole key once again.
Find the slot number for the TPM2 key:
# clevis luks list -d /dev/sda2
1: tpm2 '{"hash":"sha256","key":"ecc","pcr_bank":"sha256","...If the key is in slot 1, run the following to erase it:
# clevis luks unbind -d /dev/sda2 -s 1Confirm the unrecoverable wipe by typing y and entering the original pre-upgrade passphrase when prompted. On the next reboot, the system will prompt for the original LUKS passphrase, reverting to the standard behavior. These steps ensure a successful and secure upgrade.
