How to build an image mode pipeline with GitLab

This guide provides a step-by-step process for creating a robust, automated CI/CD pipeline for image mode for Red Hat Enterprise Linux (RHEL) using GitLab. Image mode, utilizing the power of bootc (bootable containers), fundamentally shifts how we manage and deploy operating systems. Instead of traditional package management, we treat the OS as a container image, making deployments immutable, verifiable, and highly consistent.

Set up the pipeline

Before setting up the pipeline, ensure you have the following prerequisites:

1. GitLab project: You must have a repository containing your image mode configuration. For the purpose of this example, include a Containerfile and an example systemd service. This is the layout of the project:

├── Containerfile
└── etc
    └── systemd
        └── system
            └── my-app.service

2. my-app.service: This is a simple custom systemd service.

[Unit]
Description=My app
[Service]
Type=oneshot
ExecStart=/usr/sbin/echo "My app"
[Install]
WantedBy=default.target

3. Containerfile: This file defines your image mode base image and any custom packages or configurations.

# Example starting point (adjust version as necessary)
FROM registry.redhat.io/rhel10/rhel-bootc@sha256:3cf9522137f3b643a780d303ee6a823cc04df847c24a4103d5563cd44785df04
# Copy the systemd service into the bootc image
COPY etc etc
# Install custom packages
RUN <<EOF
    set -euxo pipefail
    dnf install -y cockpit-ws httpd
    dnf clean all
EOF

4. GitLab variables: Securely store your Red Hat container registry credentials (e.g., CI_REGISTRY_USER, CI_REGISTRY_PASSWORD) and subscription secrets (RH_USER, RH_PASSWORD) as GitLab CI/CD variables.

The image mode CI/CD pipeline flow process involves three primary stages: build, package, and deploy.

Stage 1: Dependency management

While not strictly a CI job, automatic dependency management is crucial for maintaining an up-to-date and secure base OS image. The goal is to use a dependency bot (i.e., Renovate Bot or Dependabot) to automatically detect new versions of the parent image (registry.redhat.io/rhel10/rhel-bootc:10.1).

The key concept is to guarantee reproducible builds and enhanced security. You should reference your parent image using both a static tag and its content digest (SHA256). The bot will automatically update the digest in a merge request when the underlying image content changes.

Stage 2: Building and pushing the bootc container image

This is the first true CI stage, where the Containerfile is executed to produce the final, bootable container image. Use this container image to create the installation disk and to upgrade existing systems. Create .gitlab-ci.yml in the root of your project with the following contents.

Tools: Podman or Buildah are ideal for this stage. They are often available in standard GitLab runners, or you can run them within a specialized runner image.

GitLab CI job (.gitlab-ci.yml):

stages:
  - build
  - package
  - deploy
# Use semantic versioning for release tags (e.g., 1.0.0)
variables:
  IMAGE_NAME: $CI_REGISTRY_IMAGE/my-rhel-os
  IMAGE_TAG: $CI_COMMIT_SHORT_SHA
build_bootc_image:
  stage: build
  image: registry.access.redhat.com/ubi10/podman:latest
  script:
    - |
      set -euxo pipefail
      # Login to the gitlab registry where this image will be pushed to
      podman login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
      # Login to registry.redhat.com so we can pull the rhel-bootc image
      podman login -u $RH_REGISTRY_USER -p $RH_REGISTRY_PASSWORD registry.redhat.io
      subscription-manager register --username $RHN_USER --password $RHN_PASSWORD
      # Set up entitlement secrets for podman build
      # Podman expects these at /run/secrets/ per /etc/containers/mounts.conf
      # This is a hack to enable rhsm for demonstration purposes
      # In a production environment the runner should be registered with RHSM.
      mkdir -p /run/secrets/etc-pki-entitlement
      cp /etc/pki/entitlement/*.pem /run/secrets/etc-pki-entitlement/
      mkdir -p /run/secrets/rhsm
      cp -r /etc/rhsm/* /run/secrets/rhsm/
      podman build -t $IMAGE_NAME:$IMAGE_TAG .
      podman push $IMAGE_NAME:$IMAGE_TAG

Stage 3: Packaging the disk image (OS artifact)

The bootc image is not directly deployable to bare metal or VMs. You must convert it into a disk image format (e.g., .qcow2, .ami, .vmdk). This is where the bootc-image-builder (BIB) tool comes in. Update .gitlab-ci.yml with a new build_disk_image job.

Tools: Run the bootc-image-builder as a container, making it easy to integrate.

GitLab CI job (.gitlab-ci.yml):

# Convert the bootc image to a disk image (qcow2 example)
build_disk_image:
  stage: package
  tags:
    - saas-linux-large-amd64
  image:
    name: quay.io/centos-bootc/bootc-image-builder:latest
    entrypoint: [""]
  needs:
    - build_bootc_image
  script:
    - |
      set -euxo pipefail
      mkdir output
      podman login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
      podman pull $IMAGE_NAME:$IMAGE_TAG
      bootc-image-builder $IMAGE_NAME:$IMAGE_TAG --type qcow2 --output ./output
  artifacts:
    paths:
      - output/*.qcow2

Note: The bootc-image-builder requires --privileged mode to handle the necessary disk partitioning and filesystem creation, so your runner must support this. The shared instance runners on Gitlab are not supported, so you will need a custom runner for this step.

Stage 4: Deployment and storage

The final stage pushes the compiled OS artifact (.qcow2 file) to an accessible location, typically an object storage service like Amazon S3, Azure Blob Storage, or a private server.

Tools: Use cloud provider CLIs (e.g., AWS or AZ) or standard Linux tools (curl, rsync) depending on your target.

GitLab CI job (.gitlab-ci.yml):

# Upload the disk image to object storage for deployment
deploy_image_artifact:
  stage: deploy
  image: python:latest # Use a common image that can install cloud CLIs
  needs:
    - build_disk_image
  script:
    - echo "--- Installing AWS CLI (Example) ---"
    - pip install awscli
    # Ensure AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are set as CI variables
    
    - echo "--- Uploading qcow2 file to S3 ---"
    - FILE_PATH=$(find output -name "*.qcow2")
    - aws s3 cp $FILE_PATH s3://your-os-artifacts-bucket/rhel-image-mode/$IMAGE_TAG.qcow2
    
    - echo "Deployment artifact available at: s3://your-os-artifacts-bucket/rhel-image-mode/$IMAGE_TAG.qcow2"
  # Only run this job when pushing to the main branch
  only:
    - main 

Wrap up

By structuring your GitLab CI/CD pipeline around the three stages—building the bootc container, packaging the disk image using bootc-image-builder, and deploying the artifact—you can fully automate your image mode operating system updates. You can easily provision new systems with the disk image and update existing systems with the new container image. This approach guarantees consistency, leverages container security features, and dramatically speeds up your OS deployment and patching cycles.