Red Hat UBI vs. Red Hat Hardened Images: How to choose

There are many similarities between Red Hat Universal Base Image (UBI) and Red Hat Hardened Images. Both commercial and open source developers can use either foundation to produce container images for their applications and redistribute the results either freely or for a fee, without constraints from Red Hat, while retaining the option to obtain Red Hat support.

To determine whether Red Hat UBI or Hardened Images is better for your needs, the main question is: Do you value closeness to upstream projects—to maintain a near-zero CVE state—more than development stability? Red Hat Hardened Images delivers the former, while UBI delivers the latter. Commercial developers might also require formal certifications, which only UBI delivers.

Notice that, by stability, I don't mean software reliability or mean time between failures: I mean the rate at which backward compatibility might break. This affects how frequently you have to change your application code to work with a new release of a dependency, and how frequently your users might have to change their operational processes and automation.

This article provides more detail about the similarities and differences between UBI and Hardened Images, helping developers make informed decisions about which one best matches their needs.

What are Red Hat Universal Base Image (UBI) and Red Hat Hardened Images?

Red Hat Universal Base Image (UBI) is a component of Red Hat Enterprise Linux that helps build containers. The Red Hat UBI is a traditional container base image that extends easily to run specific workloads. Freely distributable and using Red Hat Enterprise Linux (RHEL) packages, UBI images prioritize shipping runtimes and complementary packages that offer ease of use across multiple deployment targets.

Red Hat Hardened Images provides a minimal, security-focused foundation for container-based workloads. These lightweight images include only the libraries and runtime components required to run an application, effectively minimizing the attack surface. Most images from the Red Hat Hardened Images catalog are distroless and exclude common utilities, such as package managers, GNU utils, and shells.

While UBI images are built from RHEL packages, images from the Red Hat Hardened Images catalog are built from upstream sources. This approach reduces the time it takes to incorporate an upstream security fix into base container images and your final application images.

If you need a more detailed introduction to Hardened Images and UBI, refer to my previous articles:

• Exploring distroless containers with Project Hummingbird
• Build trusted Python containers with Project Hummingbird and Calunga
• Red Hat Universal Base Images for Docker users

Similarities and differences between image types

The Red Hat product documentation provides the following table comparing UBI with Hardened Images.

Red Hat's product documentation also provides bullet points for “When to choose Red Hat Hardened Images” and “When to choose Red Hat UBI” directly following that table. However, I don't think the table and subsequent bullet points tell the whole story. As a developer, they raise many questions in my mind that I attempt to answer in this article, such as:

• If UBI is for general application development, then is Hardened Images only suitable for specific use cases? No, Hardened Images is also suitable for general application development.
• If Red Hat Hardened Images is for production-ready microservices, then was I doing it wrong by using UBI for my production services for years? No, UBI images are also production-ready.
• If Red Hat Hardened Images is for highly secure services, are UBI images insecure? No, UBI images also significantly strengthen your security posture, despite having a higher CVE count and a larger attack surface than images from the Red Hat Hardened Images catalog.

The answers to these and other questions are nuanced, and this article shares a practical perspective to help you decide.

What both container foundations have in common

Despite serving distinct development workflows, both options share foundational traits that provide deployment flexibility for any container environment.

Who can use UBI and Hardened Images?

Everyone can use Hardened Images and UBI. You do not need to be a Red Hat customer, and you do not have to buy a Red Hat product. You don't even need to use Linux. If you are a developer using a Docker product on macOS or Windows, or a cloud provider's Kubernetes service, you can use both options. Either choice provides distinct advantages over other alternatives.

You can also publish the container images you build using Hardened Images and UBI as your base layer to any registry, such as Docker Hub, a public Quay.io repository, the GitHub container registry, or a private Artifactory instance. Your container images won't be constrained, and you won't violate an end-user agreement if you work for a Red Hat customer.

Are UBI and Hardened Images supported with an SLA?

When you run containers on certain Red Hat products, notably RHEL and Red Hat OpenShift, you receive full support under the service-level agreement (SLA) terms of your subscriptions. That support doesn't cover your custom code and build tools, but it does include fixing bugs found within the images themselves.

If you are a Red Hat partner, you also get support for UBI and Hardened Images under the terms of your partner agreement.

If you are a commercial developer and one of your customers finds an issue that you suspect originates in UBI or Hardened Images rather than your code, you can easily get help. Just reproduce the issue on a supported Red Hat product, then open a Red Hat support ticket to receive assistance.

Are UBI images built using a secure software development lifecycle?

Red Hat products have always been built using a security-focused software development lifecycle, and UBI is no exception. UBI is built from the same packages and pipelines that produce RHEL.

Red Hat has always built its products from source code, tracking provenance and dependencies within the same process before digitally signing the results for distribution.

Hardened Images pioneered a build process based on Konflux that makes it easier for third parties and customers to verify provenance and reproduce builds. While UBI uses a different, legacy RHEL build process, it is just as security-focused as the pipeline for Hardened Images. Because Red Hat is migrating all its products to Konflux, this difference between UBI and Hardened Images is expected to disappear over time.

Does Red Hat Hardened Images provide builders for different programming languages?

Yes, the catalog provides these options, but using them is different from using similar images from UBI.

Both UBI and Red Hat Hardened Images support single-stage and multi-stage Containerfiles. They include dependency managers for a given runtime, such as npm for Node.js, and allow you to include native code dependencies. Each option provides base images that you can use as a starting point to add your required tooling if the prebuilt choices don't fit your application.

While UBI integrates easily with other Red Hat products, such as OpenShift and its Source-to-Image feature, Hardened Images is compatible with popular runtimes and builders from the open source community.

This compatibility goal sometimes conflicts with the security goal, and Hardened Images might require changes to a Containerfile originally designed for popular community images. On the other hand, a Containerfile originally designed for UBI frequently requires multiple changes beyond its FROM clause to use Hardened Images.

Key security and deployment differences

The primary operational distinctions between these options center on your requirements for long-term stability, release frequency, and upstream package alignment.

Does Hardened Images provide ABI and API stability? Does Red Hat backport features to older versions?

It depends on the upstream project of the main software in a given image and its dependencies. You have to assess backward compatibility changes case-by-case. In any case, container images from the Red Hat Hardened Images collection do not receive backports; they only receive the fixes applied by upstream projects.

Some open source projects have large enough communities that they are able to maintain multiple releases in parallel, but most only apply patches, including security fixes, to their latest release.

So, to receive those patches, you must also adapt to whatever other changes in behavior, APIs, and configuration syntax come with their latest release. These changes can affect not only your application code but also your operational processes. This is a reason so much software ends up staying behind: it's not that their developers don't care about CVEs, but incorporating those fixes requires a substantial effort.

UBI shields developers and users from that trouble by inheriting from RHEL backports of fixes from upstream. Red Hat performs backporting for its products, making sure they contain fixes to critical CVEs that were applied upstream to their latest release but not to older releases. Backports are required to preserve the API and ABIs of older releases, which helps prevent breaking backward compatibility. Trust me, performing backports is harder than it looks.

Indeed, Red Hat engineers are active participants in many upstream communities and are frequently the authors of the original security fixes. This participation makes sure Red Hat has the knowledge and people power to perform backporting consistently and support its customers well. In the end, your UBI images, as well as your RHEL servers, might be running a somewhat older release of a library or service, but that release actually has the latest critical security fixes.

Historically, security scanners have always had trouble reconciling the CVE information from upstream with the backported fixes by Red Hat and the few other vendors and open source Linux distributions that also perform backports. That is, they trigger a lot of false positives, flagging CVEs that are actually fixed in Red Hat products, including UBI images. Nowadays, a number of vendors are certified for using security data from Red Hat and shouldn't produce those false positives when configured correctly.

Backporting and validating fixes takes time, so UBI images take longer to get fixes than images from the Red Hat Hardened Images catalog. Is that more important than backward compatibility to your application and your users? Is the effort of dealing with incompatible changes from upstream larger than the effort of dealing with false positives from security scanners and assessing which CVEs have a real impact on your application? Only you can tell.

Can I use RHEL packages with Hardened Images?

No, you cannot.

Because images from the Red Hat Hardened Images catalog are not built from RHEL packages, RHEL repositories will not work with them. These packages can follow different structural and naming conventions, meaning dependencies will not align. Even if they did work together, mixing these components would void any certifications inherited from RHEL.

You can use Fedora packages with Hardened Images by selecting Fedora Rawhide. This serves as a temporary workaround if you need a package that is not yet onboarded, though you cannot use Fedora packages with standard RHEL and UBI images.

UBI images, on the other hand, are built from a subset of RHEL packages. You can enable actual RHEL repositories when needed, but your application containers will cease to be UBI containers and become RHEL containers. This switch affects your ability to redistribute content—subjecting you to the standard RHEL end-user license agreement (EULA)—but it might be necessary for certain infrastructure components and system services.

UBI images inherit all certifications from RHEL; if a hardware or software vendor supports RHEL (or declares compatibility with it), that support automatically extends to UBI. Container images from the Red Hat Hardened Images catalog do not share these automatic benefits. Instead, vendors must validate and support images from the Red Hat Hardened Images catalog as distinct entities separate from RHEL.

Can I use Red Hat Trusted Libraries with UBI?

While you can, if you need Red Hat Trusted Libraries, you likely need Red Hat Hardened Images as well.

Red Hat Trusted Libraries follows a similar software development lifecycle (SDLC) to Red Hat Hardened Images, including digital signatures, software bills of materials (SBOMs), and attestations based on Konflux. Like container images from the Red Hardened Images collection, these libraries are built from upstream source code to minimize vulnerabilities.

RHEL and UBI alone might not provide the dependencies you need to build your application. In that case, getting your dependencies from Red Hat Trusted Libraries strengthens your security posture compared to using unvetted community repositories.

Using Red Hat Trusted Libraries with UBI can introduce operational hurdles because of differences in their release cadences, CVE streams, and metadata formats. It might be more efficient to switch to Hardened Images directly.

Does Red Hat Hardened Images provide databases, tools, and images besides runtimes?

Yes, it does! The Hardened Images repository is growing and already includes container images that you can run as-is, such as MariaDB and Valkey databases or utilities like cosign and jq. This variety makes Red Hat Hardened Images useful for both your inner and outer loops, such as powering tasks in your continuous integration and continuous delivery (CI/CD) pipelines.

Conversely, UBI only provides base images and runtimes for programming languages such as Python and Java. Developers using UBI for their inner loop must add container images from community repositories or switch to standard RHEL images, which are subject to full RHEL terms instead of the more permissive UBI terms.

You might argue that developers don't require these images because they can use managed cloud databases or run local services on their machines. Production environments will likely need vendor support for databases and other middleware anyway. While that is true, being able to set up a complete development and testbed environment using only Red Hat Hardened Images is highly convenient.

What about bootc images and image mode for RHEL?

bootc images are another kind of container image that Red Hat supports as part of image mode for Red Hat Enterprise Linux. The bootc technology is open source, and you can find bootc images for Fedora Linux and CentOS, among others.

bootc images are complete operating system images that include a kernel, boot loader, device drivers, and many system services. While you can build a bootc image that “just works” as a regular container in Docker, Podman, and Kubernetes, consider whether the larger image size and attack surface are worth the trade-off.

If you must provide a bootc image for your application, I recommend starting with a regular, lean application container image and embedding it within a bootc image as a bounded container. This way, you can run your application as a container without extra baggage, or deploy it as a physical or virtual machine appliance. Each usage scenario gets the kind of container image (regular or bootc) that best fits it.

If you need an SDLC with the attestation you get from Red Hat Trusted Libraries at the operating system level of your bare-metal machines and cloud instances—and you don't require third-party vendor support or certifications—you can consider Fedora Hummingbird and its bootc container image. Fedora Hummingbird originates from the same package repositories and SDLC used to build Red Hat Trusted Libraries.

Choosing your container image foundation

Both UBI and Red Hat Hardened Images provide base images and runtimes suitable for general application development and production workloads. Both options deliver the benefits of a security-focused software development lifecycle managed by Red Hat, which produces verifiable signed artifacts and allows you to freely redistribute your application containers to community members and paying customers.

UBI provides the same certifications, compatibility, long-term stability, and third-party support found in Red Hat Enterprise Linux, at the expense of a longer delay to incorporate lower-severity security and bug fixes. It also features a larger attack surface due to the shells, package managers, and other components included in its images.

Red Hat Hardened Images, on the other hand, provides attestation of provenance you can extend to your own dependencies and code. They offer a smaller attack surface and fewer CVEs by tracking upstream projects closely. However, this strategy comes at the expense of reduced long-term stability and might require extra effort to adapt to upstream changes in APIs, configuration syntax, and operational procedures.

Which option is better for you? It depends on which characteristics you prioritize and which trade-offs you can accept. UBI is not going away because of Red Hat Hardened Images. Instead, many developers who use UBI today might switch to Hardened Images, while others who previously avoided UBI in favor of community images can now make the transition to Hardened Images.

Thanks a lot to Matt Micene and Scott McCarty for their reviews and suggestions.