Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat OpenShift AI
      Red Hat OpenShift AI
    • Red Hat Enterprise Linux AI
      Linux icon inside of a brain
    • Image mode for Red Hat Enterprise Linux
      RHEL image mode
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • Red Hat Developer Hub
      Developer Hub
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Image mode for Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of OpenJDK
    • Kubernetes

      • Red Hat OpenShift
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Virtualization
      • Red Hat OpenShift Lightspeed
    • Integration & App Connectivity

      • Red Hat Build of Apache Camel
      • Red Hat Service Interconnect
      • Red Hat Connectivity Link
    • AI/ML

      • Red Hat OpenShift AI
      • Red Hat Enterprise Linux AI
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Lightspeed
    • Developer tools

      • Red Hat Trusted Software Supply Chain
      • Podman Desktop
      • Red Hat OpenShift Dev Spaces
    • Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • API Catalog
    • Product Documentation
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

Image-based installation is easier and faster

How single node OpenShift image-based installation works

February 14, 2025
Michail Resvanis
Related topics:
ContainersDisconnected EnvironmentsEdge computingKubernetes
Related products:
Red Hat OpenShift

Share:

    Image-based installation is a process that allows you to provision a single node OpenShift cluster using a pre-generated container image and configure the details per cluster in the second step. This contrasts with traditional, dynamic installation processes where configuration happens in one step during installation. 

    Image-based installation complements existing tools like GitOps Zero Touch Provisioning (ZTP) and the Agent-based Installer (ABI) by significantly boosting speed and the reproducibility of single node OpenShift cluster provisioning.

    Image-based installation features

    The motivation behind the image-based installation is the need for speed. In edge environments, the ability to add a new server or replace an old one much faster is critical. It reduces downtime, maintenance windows, and resource utilization. Image-based installation also enables repeatable installations at scale with consistency by leveraging a single container image to capture common configuration across all hosts.

    The key features of image-based installation include:

    • One installation image for multiple hosts: The installation image contains the common configuration across all hosts. This removes the need to install each host with a separate image, significantly shortens the installation time, and ensures consistent, repeatable installations at scale.
    • Speed: It is 84% faster than other installation methods by shifting installation processes earlier in the workflow.
    • Disconnected environments: By separating the installation and configuration steps, image-based installation is designed for environments without an active internet connection.

    Faster single node OpenShift cluster installation

    Our latest scale tests showed that image-based installation of a single node OpenShift cluster is 84% faster than ZTP. During these scale tests, over 3,500 single node OpenShift clusters were provisioned with nearly 100% success rate. An image-based installation of a single node OpenShift took 12 minutes, while ZTP took 74 minutes for the 99th percentile, using the specific hardware and network setup.

    Image-based installation achieves its speed by shifting installation processes earlier in the workflow. It introduces an upfront preparation step shared across all hosts, making the process highly efficient as this effort is amortized across multiple installations. Image-based installation uses static, common configuration for the installation image, and it allows for dynamic host configuration during the deployment step. This contrasts with other installation methods where configuration is dynamically set up during the installation, offering a different balance between consistency and flexibility.

    Under the hood: core components and concepts 

    Image-based installation leverages the following key technologies:

    • Open Container Initiative (OCI) container image: The seed image, the common configuration across all hosts, is captured via a container image of approximately 1.2G in size.
    • Lifecycle Agent (LCA) operator and command-line utility lca-cli: The software that creates the seed image from an existing single node OpenShift cluster deploys the seed image on a new host and also applies all the distinct per-host configuration.
    • rpm-ostree and OSTree: The tools that manage snapshots of the operating system are bundled into the seed image and deploy the desired snapshot on the target host.
    • recert: A tool that regenerates all cryptographic objects in a single node OpenShift cluster and customizes  it according to user provided configuration.
    • OpenShift Container Platform install options: The Red Hat OpenShift standard installation command-line tool, to which we have added a new image-based subcommand that creates the image-based installation artifacts (i.e., the installation and the configuration ISO images).

    The end-to-end flow installation

    Image-based installation consists of one preparation step that is common across all single node OpenShift provisionings and two steps per host (i.e., installation to disk and deployment), as shown in Figure 1. The preparation step captures the common configuration across all hosts and stores it to an OCI container image, which we call the seed image. The two steps per host are as follows:

    Step 1: Installation to disk 

    1. Boot a temporary Red Hat Enterprise Linux CoreOS (RHCOS).

    2. Extract the seed image to the filesystem.

    3. Install the configuration found in the seed image to the target host.

    Step 2: Deployment 

    Configure the distinct per single node OpenShift cluster details: 

    1. Cluster name
    2. Base domain
    3. The default OpenShift kubeadmin user password.
    4. The kubeconfig file of the single node OpenShift cluster and others.
    Image-Based Installation: End-to-End Flow.
    Figure 1: The end-to-end image-based installation flow.

    Provisioning single node OpenShift clusters

    Users create the following artifacts when using the image-based installation to provision single node OpenShift clusters:

    • Seed OCI image: The seed image is a container image that includes the common configuration and operating system files across all hosts and it is generated by the LCA operator and lca-cli on an existing single node OpenShift cluster.
    • Installation ISO: A customized RHCOS live ISO that contains the seed image and software that knows how to unpack it and provision the common configuration across all hosts. This is created via the OpenShift installer.
    • Configuration ISO: An ISO image that includes the distinct per-host configuration. It is created via the installer or the image-based install operator when Red Hat Advanced Cluster Management for Kubernetes (RHACM) is used to scale your infrastructure. To simplify the demonstration of the image-based installation internals, we will not use the image-based install operator in this article.

    How image-based installation works

    Let's explore the phases of the installation.

    Preparation

    The goal of this step is to capture the common configuration across all hosts. This includes the target single node OpenShift environment and its configuration, both of which can be found in the filesystem. What if we could take a snapshot of a healthy cluster that's already running single node OpenShift with the desired configuration? We could then use this snapshot to rapidly provision multiple single node OpenShift clusters with identical configurations. That's exactly what the LCA operator and its lca-cli do. What better way to package the snapshot of an open source container orchestration platform than to use an OCI container image. 

    Generating the seed image

    The seed image is an OCI container image built from scratch with a single layer containing a few files and is approximately 1.2G in size. The LCA operator, using its lca-cli, generates the seed image after ensuring that the cluster is healthy, OpenShift is stopped and all its TLS certificates are forcefully expired using recert. The contents of the seed image look like this:

    $ ls -F1 /seed-image/
    containers.list
    etc.deletions
    etc.tgz
    lca-cli*
    manifest.json
    mco-currentconfig.json
    ostree-a0c720aa9af355377ee4ee311a1cad89cd2827df7acec2f971a9e87995386828.0.origin
    ostree.tgz
    recert_config.json
    rpm-ostree.json
    var.tgz

    These files play their part during an image-based installation, but what we basically have in there is an operating system along with a recording of its changes (ostree.tgz, etc.tgz, var.tgz), its last known status (rpm-ostree.json), and a command-line tool that knows how to deploy that operating system (lca-cli). That’s all we need to set up a copy of the seed single node OpenShift cluster on a new host.

    Installation to disk

    The installation to disk step sets up a copy of the seed single node OpenShift cluster provisioned on the target host using the seed image. But first things first: how do we get from a container image to a single node OpenShift cluster on an empty host? It would be useful to have a temporary environment from which we could mount the seed image and deploy the seed single node OpenShift onto the host’s disk. That’s what the installation ISO provides.

    Installation ISO

    An ISO image is a single file that contains an exact copy of an optical disc that can be mounted virtually as if it were a physical disk. The best part: it can be bootable, too. We already have such an ISO in the OpenShift ecosystem: the RHCOS live ISO. All we have to do is customize it via Ignition, and after it boots, it executes our installation flow.

    You can create the installation ISO with the openshift-install image-based create image command, a feature released in the OpenShift installer with Red Hat OpenShift Container Platform 4.17.0. Take a look into the respective documentation on how to create an installation ISO with the installer. Keep in mind that you only need to create the installation ISO once for all the hosts with the same configuration.

    Installation to disk flow

    Let’s boot the installation ISO on one of the hosts and see what it actually does. Once the host is booted the image-based installation systemd unit will execute a bash script that does the following: 

    • Runs a container from the seed image.
    • Copies the lca-cli binary from that container.
    • Runs the lca-cli ibi subcommand with a configuration file that was also included in the installation ISO.

    The most important steps lca-cli ibi follows to install the seed cluster snapshot onto the new host are shown in Figure 2.

    Installation to disk flow.
    Figure 2: Installation to disk flow.
    1. Prepare the disk:
      • Install the RHCOS version found in the RHCOS Live ISO on the disk.
      • Mount the seed image to the filesystem to use its contents.
      • Extract the OSTree repository (ostree.tgz) from the seed image and use OSTree’s pull-local command to bring the seed cluster’s RHCOS OSTree repository’s data onto the current system’s OSTree repository.
    2. Configure OSTree:
      • Create a new OSTree stateroot using the OSTree os-init command—that’s where it will deploy the seed cluster’s RHCOS snapshot.
      • Deploy the OSTree commit indicated by rpm-ostree.json. This is the output of the rpm-ostree system status command in JSON format. Since the OSTree repository contains multiple commits, we need to know which one to actually deploy.
      • Copy the ostree-<SHA256 hash>.origin file from the seed image to the proper filesystem location. OSTree uses that file to identify where its default root filesystem tree comes from. It can track the container image defined in the origin file for its future upgrades.
      • Extract var.tgz and etc.tgz onto the filesystem, as those two are the only writable directories in an OSTree managed operating system. OSTree does not manage /var, as it contains runtime data, logs, caches, and other persistent data. On the other hand, /etc is managed by OSTree, and it contains a 3-way merge of the default read-only configuration under /usr/etc, the previous configuration, and the active system’s /etc.
    3. Final setup:
      • Pull all the container images found in the seed image container.list file.
      • Execute an optional user-defined post installation script.
      • Remove the default RHCOS OSTree deployment applied when installing RHCOS from the live ISO.

    A reboot is required for the OSTree staged deployment to take effect. After the reboot, early in the boot process, after the network comes online, another systemd unit, which we call post-pivot, takes over. We snuck the post-pivot systemd unit in the seed image (under /etc/systemd/system). It is responsible for the image-based installation final step, the deployment step.

    Deployment

    The goal of the deployment step is to have a new single node OpenShift cluster with most of its configuration matching the seed single node OpenShift cluster’s configuration. The remaining configuration is specific to the new cluster: 

    • The cluster ID

    • Name and base domain

    • The host IP address and hostname

    • Any SSH keys and pull secrets to be added

    • New TLS certificates

    • The default OpenShift kubeadmin user password and kubeconfig file.

    As in the installation-to-disk step, we can provide this configuration to the host using the configuration ISO image.

    Configuration ISO

    The configuration ISO is not a live ISO. The contents look like the following:

    $ tree /mnt/image-based-config-iso                                                                                         
    /mnt/image-based-config-iso
    ├── cluster-configuration
    │   ├── manifest.json
    └── extra-manifests

    The manifest.json is a JSON-formatted file that contains the configuration applied by lca-cli and recert. You can create the configuration ISO with the openshift-install image-based create config-image command, the previously mentioned feature released in the installer with OpenShift Container Platform 4.17.0. Take a look at the documentation of how to create an image-based configuration ISO with the installer.

    Authentication

    The openshift-install image-based create config-image creates, alongside the configuration ISO, the default OpenShift kubeadmin user password and the kubeconfig file for the new cluster. Those can be found in the auth/{kubeadmin-password,kubeconfig} files respectively, just like in a regular OpenShift installation.

    Deployment flow

    The lca-cli post-pivot command, executed via the post-pivot systemd service, waits until the configuration ISO is mounted. It then configures the single node OpenShift cluster with its distinct details early during boot, before the kubelet starts. The most important steps it follows are depicted in Figure 3 alongside the recert flow.

    Deployment and recert flows.
    Figure 3: Deployment and recert flows.
    1. Set up the host credentials and network:
      • Set the SSH key(s).
      • Set the pull secret(s).
      • Configure the network.
    2. recert
      • Set up a temporary etcd server, and recert will use that server to scan and update the etcd database.
      • Run recert.
    3. Start OpenShift Container Platform:
      • Start kubelet to execute all OpenShift containers.
      • Disable the systemd post-pivot service from which this flow runs.

    The recert tool

    The recert command-line interface tool is responsible for transforming a copy of the seed cluster to a new one by providing the following:

    • Cryptographic object regeneration: recert regenerates all the certificates, private keys, and JSON Web Tokens (JWTs) of the cluster.
    • Single node OpenShift cluster reconfiguration: recert configures the copy of the seed cluster with a new cluster ID, new cluster name, new base domain, etc.

    You can find cryptographic objects and Red Hat OpenShift configuration in the filesystem and in the etcd database. That’s why recert scans and writes into both. Learn more about how recert works in this documentation. 

    Note: The recert tool makes an effort to be FIPS 140-3 compliant by using the OpenSSL library of the underlying RHCOS for all its cryptographic operations. As long as that OpenSSL version is FIPS compliant, recert will also be FIPS compliant.

    The other critical task of recert is reconfiguring the Red Hat OpenShift control-plane components in such a way that they don’t even realize that something has changed. This saves more than 8 minutes of trying to reconcile them during boot. 

    After recert, we have basically what we asked for, a new single node OpenShift cluster configured and ready to go in a matter of minutes.

    Try image-based installation

    Image-based installation drastically reduces the single node OpenShift installation time. By leveraging a single container image, image-based installation consistently deploys single node OpenShift clusters at scale. Try it yourself by following the documentation and tell us what you think.

    Related Posts

    • Scale testing image-based upgrades for single node OpenShift

    • How to install single node OpenShift on bare metal

    • How to install single node OpenShift on AWS

    • Network observability with eBPF on single node OpenShift

    • RHEL 9 and single node OpenShift as VMs on macOS Ventura

    Recent Posts

    • How Trilio secures OpenShift virtual machines and containers

    • How to implement observability with Node.js and Llama Stack

    • How to encrypt RHEL images for Azure confidential VMs

    • How to manage RHEL virtual machines with Podman Desktop

    • Speech-to-text with Whisper and Red Hat AI Inference Server

    What’s up next?

    Learn how to install the oc command-line interface (CLI) in order to work with your Red Hat OpenShift cluster from a command line.

    Start the activity
    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue