Typically, building containers and images from a standard Dockerfile requires root access and permissions. This can create a challenge when working with public or shared clusters. For example, cluster admins don't often allow permissions to run this type of workload, as it might compromise the security of the entire cluster.
In these situations, many developers use a build tool such as kaniko to simplify the effort. Kaniko can build your images without requiring root access. This capability makes kaniko a feasible alternative for building containers and images in any kind of environment; for example, standard Kubernetes clusters, Google Kubernetes Engine, and public or shared clusters. Kaniko can also automatically push your images to a specified image registry.
This article shows you how to use kaniko to build a container image in a Red Hat OpenShift cluster and push the image to a registry.
To perform a kaniko build on a Red Hat OpenShift cluster, ensure that the following prerequisites are in place:
- Access to an active OpenShift cluster (with admin access).
- Access to a source code repository that is either local or hosted somewhere, such as GitHub.
- A valid Dockerfile for your target source directory. The Dockerfile can exist anywhere, as long as a fully-qualified URL is available.
Note: All the
oc commands in this article also work with
kubectl, whether you are working with an OpenShift cluster or a Kubernetes cluster, or without a cluster.
Setup and configuration for kaniko on OpenShift
Once the prerequisites are set up, configured, and active, you can perform a kaniko build on an OpenShift cluster and push the image to a registry.
Log in to the OpenShift cluster
To start, log in to your OpenShift cluster as follows:
$ oc login --token=token --server=server-url
Create a new project
Create your own project using:
$ oc new-project project-name
Create a secret using the credentials to your registry
To push your image to an external registry (such as Docker Hub), create a secret named
regcred using the following
$ oc create secret docker-registry regcred \
Replace the italicized values in this command with the following information:
your-registry-server: the fully qualified domain name (FQDN) of your private Docker registry (https://index.docker.io/v1/ for Docker Hub)
your-name: your Docker username
your-pword: your Docker password
your-email: your Docker email address
Note: Push your image to an internal registry through a pod on a cluster using a service account. For example, you can acquire login credentials for your service account, such as for Builder, through the cluster's console.
From the list of available secrets in your namespace, pick a
builder-dockercfg secret, and expose the base64 credentials using the Reveal Values button on the OpenShift console.
Locate the URL for your target image registry and copy the authorization token. Use it to prepare a new
config.json file by replacing image-registry-url and auth-token with the appropriate values. For example:
"User-Agent": "Docker-Client/19.03.8 (darwin)"
config.json file is ready, create a secret as follows, naming it
$ oc create secret generic regcred --from-file=.dockerconfigjson=path/to/config.json --type=kubernetes.io/dockerconfigjson
Clone a source code repository
In the local file system,
git clone your source code repository. For example, in an empty directory enter the following:
Next, download the corresponding Dockerfile from its location and place it at the root of this directory, if it doesn't already exist there.
If a specific subdirectory within your cloned repo hosts the code used to build an image (as opposed to the entire cloned directory), place the Dockerfile at the root of that subdirectory. Together the directory containing your source code and Dockerfile now represent your build context.
Note: When using the Dockerfile present in the mentioned repository, drop the line that says
USER nobody, to avoid permission issues.
Make sure that the paths mentioned after
/kaniko/build-context against the
--context parameters in the
openshift-pod.yaml file accurately represents the directory structure present inside
kaniko-build-context.tar.gz. The paths must be an exact match.
Compress the build context into a tar.gz file
Once the build context is ready, compress it into a
tar.gz file as follows:
$ tar -czvf kaniko-build-context.tar.gz path/to/folder
Create an openshift-pod.yaml file with two containers
openshift-pod.yaml file that has two containers, as shown in Figure 1.
If you are pushing to Docker Hub, you could set the destination as follows:
If you are pushing to the internal registry, set the destination as shown here:
Apply the pod to the cluster
Use the following command to apply the pod to your cluster:
$ oc apply -f path/to/openshift-pod.yaml
The command should return:
Check the status of the cluster
To check the cluster's status, run the following command:
$ oc get pods
Here is an example of what it displays:
NAME READY STATUS RESTARTS AGE
kaniko 0/1 Init:0/1 0 50s
Copy tar.gz from the local file system to the kaniko-init container
tar.gz file that you created earlier from the local file system to the
kaniko-init container currently running in the pod:
$ oc cp path/to/kaniko-build-context.tar.gz kaniko:/tmp/kaniko-build-context.tar.gz -c kaniko-init
Extract the copied tar.gz file on the mounted path to the shared volume
From inside your
kaniko-init container, extract the copied
tar.gz file into the mounted path pointing to the shared volume inside of the kaniko pod. This allows it to be accessed by other containers with access to this shared volume.
Check your work
You should see the pushed image reflected in your target registry. Additionally, you can take a closer look inside the container at any time. (I found this to be quite useful while attempting to debug the process.) To begin, start a bash session inside your
kaniko-init container and take a look:
$ oc exec kaniko -c kaniko-init -it /bin/bash
Once the extraction process is complete, you can shut down the init container, at which point the kaniko container takes over. Then create a file that serves as a trigger:
$ oc exec kaniko -c kaniko-init -- touch /tmp/complete
When you run
oc get pods again, the output displays whether everything is working well:
NAME READY STATUS RESTARTS AGE
kaniko 1/1 Running 0 6m57s
Next, run the following
oc command to get a more detailed look at what's going on inside the kaniko container:
$ oc describe pod kaniko
Alternatively, you can look at the pod logs within the OpenShift console.
After the pod has reached a completed state, if you pushed it to an external registry, you should be able to log into your registry and find the newly pushed image there. If you pushed to the internal registry, you should be able to navigate to Builds —> ImageStreams (within the OpenShift console's Administrator view) to find the newly pushed image there.
You can delete the pod if needed using
oc delete pod kaniko.