Building unique images for various container orchestrators can be a maintenance and testing headache. A better idea is to build a single image that takes full advantage of the vendor support and security built into Red Hat OpenShift, and that also runs well in Kubernetes.
A universal application image (UAI) is an image that uses Red Hat Universal Base Image (UBI) from Red Hat Enterprise Linux as its foundation. The UAI also includes the application being deployed, adds extra elements that make it more secure and scalable in Kubernetes and OpenShift, and can pass Red Hat Container Certification.
This article introduces you to nine best practices you should incorporate into your Dockerfile when building a UAI. Each section in this article explains a practice, shows you how to implement the practice, and includes Red Hat certification requirements related to the topic.
Best practice #1: Choose a Universal Base Image
The base image for your application provides the Linux libraries required by the application. The base image that you choose affects the versatility, security, and efficiency of your container.
A Red Hat UBI enables your universal application image to run well in both Kubernetes and OpenShift, is compliant with the Open Container Initiative (OCI), is freely redistributable, and receives official Red Hat support when run in OpenShift.
Building from a UBI
In your Dockerfile, the FROM command should create your image from a ubi8 base image. If your build machine has an internet connection, it can download the base image directly from Red Hat's registry like this:
FROM registry.access.redhat.com/ubi8/openjdk-11:1.3-15
Red Hat Container Certification requirement
The base image's Linux libraries must come from Red Hat Enterprise Linux. Red Hat Enterprise Linux's base images and Universal Base Images meet this requirement. See base image options in the Red Hat documentation for more information.
Red Hat's base images are available from the Red Hat certified container images registry. The images in the ubi8 namespace incorporate libraries from a newer version of Red Hat Enterprise Linux than the ubi7 images. There are UBIs for several different language runtimes.
If Red Hat doesn't offer a UBI for the language runtime you need, start from the smallest ubi8 base image (registry.access.redhat.com/ubi8/ubi-minimal) and run the commands to install the language's runtime.
The Containers and the Red Hat Universal Base Images (UBI) partner guide offers more information about UBI and related topics. You can also check out the Red Hat Universal Base Images (UBI) ebook.
Best practice #2: Make the image run as a non-root user
If a process running as root breaks out of the container, it gains root (privileged) access to the host machine. Therefore, run the container as a non-root user so that, if the process breaks out of the container, its access to the host machine is much more limited.
By default, Docker builds and runs an image as root (that is, UID=0). To avoid this, the Dockerfile for building the image should specify a user ID other than 0.
When Kubernetes runs the container, its processes run as the user ID specified in the Dockerfile.
Running the image as a non-root user
To specify a user in a Dockerfile, add the USER command, such as USER 1001.
The Dockerfile best practices page points out that if you specify the user as its UID instead of a username, you don't need to add the user or group name to the system's passwd or group file. However, if the base image sets a good non-root user name, you should specify that user's name. For example, a UBI defines a user named default.
Red Hat Container Certification requirement
Red Hat recommends that the image specify a non-root user. When its container is run in OpenShift, the container orchestrator will definitely run its processes as an arbitrary non-root user.
When you build an image on a Red Hat UBI that includes a language runtime, the user is already switched to a non-root user named default.
Best practice #3: Set group ownership and file permissions
If a process needs access to files in the container's local file system, the process's user and group should own those files so they are accessible. For OpenShift, the user that runs a container is assigned arbitrarily, but that arbitrary user is always a member of the root group, so you should assign the root group ownership of the local files so that the arbitrary user has access.
Setting group ownership and file permissions
In the Dockerfile, set permissions on the directories and files that the process uses. The root group must own those files and be able to read and write them as needed. The Dockerfile code looks like the following, where /some/directory is the directory with the files that the process needs access to:
RUN chown -R 0 /some/directory && \
chmod -R g=u /some/directory
For compatibility with Kubernetes, the Dockerfile should specify a non-root user ID, then set file ownership to that user ID and the root group:
USER 1001
RUN chown -R 1001:0 /some/directory
These two approaches combined work for both Kubernetes and OpenShift:
USER 1001
RUN chown -R 1001:0 /some/directory && \
chmod -R g=u /some/directory
For example, if the Cassandra database is configured to store its data in the /etc/cassandra directory, the Dockerfile to build the image for OpenShift needs the following statements:
USER 1001
RUN chown -R 1001:0 /etc/cassandra && \
chmod -R g=u /etc/cassandra
Red Hat Container Certification requirement
Red Hat Container Certification does not require or exclude setting group ownership and file permissions.
Pods run in an OpenShift cluster as arbitrary user IDs that are members of the root group. OpenShift Container Platform specific guidelines specify the following:
For an image to support running as an arbitrary user, directories and files that are written to by processes in the image must be owned by the root group and be read/writable by that group. Files to be executed must also have group execute permissions.
If a process shares files with other processes and therefore needs to run as the specific user or group that owns those files, its pod must define a security context that species the user and group. Also, the cluster must define a set of security context constraints that allow that user and group to be specified. For details, see Getting started with security context constraints on Red Hat OpenShift.
Best practice #4: Build images in multiple stages
While the deployment image must contain the application and its language runtime, it should not add any tools that are used to build the application or any libraries that are not needed by the running application. Instead, create a two-stage Dockerfile that uses separate images: one image to build artifacts and another to host the application.
Building an image in multiple stages
To build an image in multiple stages, the Dockerfile specifies multiple FROM lines, one at the beginning of each stage. The last stage produces the resulting image file, and the build process discards the images from the earlier stages. Typically, every stage but the last is given a name that makes it easier for a later stage to refer to an earlier stage's artifacts.
For example, a multi-stage build typically consists of two stages, one that builds application artifacts and another that builds the application image. The first stage is typically named builder. The Docker Use multi-stage builds pages shows this example:
FROM golang:1.7.3 AS builder
WORKDIR /go/src/github.com/alexellis/href-counter/
RUN go get -d -v golang.org/x/net/html
COPY app.go .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .
FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /go/src/github.com/alexellis/href-counter/app .
CMD ["./app"]
The stages work as follows:
- The first stage is named
builderand starts from a Go image that already has the Go build tools installed. - The second stage, which builds the deployment image, starts from an Alpine image and does not need a name.
- The first stage builds the app named
appin the directory named/go/src/github.com/alexellis/href-counter/. - The second stage copies the file
/go/src/github.com/alexellis/href-counter/appfrom thebuilderstage into the deployment stage's current directory.
Red Hat Container Certification requirement
Red Hat Container Certification does not require or exclude the use of a multi-stage Dockerfile.
Best practice #5: Include the latest security updates in your image
The Linux libraries in an image should contain the latest security patches that are available when the image is built. To get the patches:
- Use the latest release of a base image. This release should contain the latest security patches available when the base image is built. When a new release of the base image is available, rebuild the application image to incorporate the base image's latest release, because that release contains the latest fixes.
- Conduct vulnerability scanning. Scan a base or application image to confirm that it doesn't contain any known security vulnerabilities. Commonly used scanning tools include Trivy, Clair, and Vulnerability Advisor.
- Apply patches. Update the Linux components in an image using the operating system's package manager. The package managers for Red Hat Linux are
yumanddnf. The Dockerfile can run the package manager as part of building the image.
Building an image with the latest security updates
Build your application image from the latest release of a UBI, which should include components with the latest security patches.
If a UBI needs newer components because they contain newer security patches, use the RUN command to update the UBI with the latest security updates like this:
FROM registry.access.redhat.com/ubi8/openjdk-11:1.3-15
USER root
RUN dnf -y update-minimal --security --sec-severity=Important --sec-severity=Critical && \
dnf clean all
USER default
The UBI already contains the latest security patches that are available at the time the image was built, but this installs any updates that are newer than the image.
Red Hat Container Certification requirement
To pass Red Hat Container Certification, Red Hat components in the container image cannot contain any critical or important security vulnerabilities at the time that it is certified.
Understanding Red Hat security ratings explains these different security levels. To update the Red Hat components with security fixes that are not already installed, use this command in the Dockerfile for your image:
RUN yum -y update-minimal --security --sec-severity=Important --sec-severity=Critical
Best practice #6: Embed identifying information inside your image
You should build images that are clearly identifiable, to make it easy for the user to determine the name of the image, who built it, and what it does. This information should be an immutable part of the image that cannot be separated.
Labeling your image
Labels are set in the Dockerfile using the LABEL command. For example, here's how to set the labels required for Red Hat image certification:
LABEL name="my-namespace/my-image-name" \
vendor="My Company, Inc." \
version="1.2.3" \
release="45" \
summary="Web search application" \
description="This application searches the web for interesting stuff."
Red Hat Container Certification requirement
Red Hat Container Certification requires the following labels in your image:
name: The name of the image.vendor: The company name.version: The version of the image.release: A number that's used to identify the specific build for this image.summary: A short overview of the application or component in this image.description: A longer description of the application or component in this image.
Set these labels in the Dockerfile using the LABEL command. If you build your image on a UBI, Red Hat already sets these labels with Red Hat values for the UBI, but you should override those with values for your image.
Best practice #7: Embed license information inside your image
No industry standard exists for bundling the licensing information with software. However, you can easily store the text files for licenses in an image, so it's a good idea to do so. This makes the image self-documenting, so users can immediately know about the software licenses associated with their image.
GitHub can display the license for a repository if you follow these conventions:
- Licensing a repository gives information about license types and encourages the owner of any open source, public repository to specify a license.
- Adding a license to a repository explains that GitHub can detect and display the license for a repository if the license file is stored in the repository's home directory and named
LICENSEorLICENSE.md(with all caps).
Adding license information to an image
The source code directory that contains the Dockerfile should also include a licenses directory that contains these licensing files. Typically, it contains at least one file with a name like LICENSE.txt. The directory looks like this:
$ ls -ld licenses Dockerfile -rw-r--r-- 1 bwoolf staff 774 May 5 15:07 Dockerfile drwxr-xr-x 3 bwoolf staff 96 May 5 15:09 licenses $ ls -l licenses total 8 -rw-r--r-- 1 bwoolf staff 17 May 5 15:10 LICENSE.txt
The following code in the Dockerfile adds this licenses directory to the image:
COPY licenses /licenses
Red Hat Container Certification requirement
Red Hat requires that the image store the license file(s) in the /licenses directory. It's convenient to create a corresponding licenses directory in the repository's home directory that the Dockerfile will copy as-is into the image.
To accommodate GitHub and Red Hat's different approaches, you can store two copies of your license file, one in LICENSE for GitHub and another in licenses/LICENSE.txt for Red Hat.
Best practice #8: Maintain the original base image layers
When building an application image, do not modify, replace, or combine the packages or layers in the base image. However, there is one exception: The build process can and should update the security packages in the Linux libraries with the latest updates.
A container image's metadata should clearly show that the image includes the layers of the base image, has not altered them, and has only added to them.
Maintaining the original image layers in your Dockerfile
A Dockerfile normally builds from a base image and adds new layers to it. An application should run on top of its operating system, but not replace any of it.
Red Hat Container Certification requirement
Red Hat Container Certification prohibits modifications to the layers in the Red Hat base image. When the base layer uses a Red Hat UBI, Red Hat does support the use and extension of the UBI layer.
See the Red Hat Container Support Policy for more information.
Best practice #9: Limit the number of layers in your images
Layers in an image are good, but having too many adds complexity and hurts efficiency. Limit the images you build to about 5-20 layers (including the base image's layers). Up to 30 layers are acceptable, but 40 or more layers become too many to manage easily.
Limiting the number of layers
The number of layers in an image depends on how the image is built. To list the layers in an image, use the Docker command-line interface:
docker history <container_image_name>
Alternatively, use Podman:
podman history <container_image_name>
For example, the ubi8/openjdk-11:1.3-15 image has three layers:
$ docker pull registry.access.redhat.com/ubi8/openjdk-11:1.3-15
$ docker history registry.access.redhat.com/ubi8/openjdk-11:1.3-15
IMAGE CREATED CREATED BY SIZE COMMENT
a9937ea40626 7 days ago 509MB
<missing> 13 days ago 4.7kB
<missing> 13 days ago 103MB Imported from -
Red Hat Container Certification requirement
Red Hat Container Certification requires that the image contain fewer than 40 layers. Red Hat's UBIs have very few layers, enabling your build process to add many more layers without exceeding 40.
Where to learn more
By following the nine best practices in this article, you can build an image that is high-quality and efficient, and runs well in both Kubernetes and OpenShift. Here are a few additional resources:
- Explore Red Hat Container Certification.
- Get a quick overview of Red Hat Universal Base Images.
- Download the free e-book: Red Hat Universal Base Images (UBI)