Adapting Docker and Kubernetes containers to run on Red Hat OpenShift Container Platform

More and more companies are migrating their applications to the Red Hat OpenShift Container Platform (RHOCP). This enterprise-grade container platform is secure and comprehensive, based on industry standards including those related to Docker and Kubernetes. However, due to the tightened security restrictions, containers that run on Docker and Kubernetes might not run successfully on Red Hat OpenShift without modification.

Red Hat OpenShift Container Platform is a fully managed Red Hat OpenShift service that takes advantage of enterprise-ready scaling and security. It is directly integrated with Kubernetes and provides several models for application deployment. For example, OpenShift can mitigate the risk that processes running in a container might be given escalated privileges on the host machine, due to security vulnerabilities in the container engine. For this reason, containers are run using an arbitrarily assigned user ID.

In contrast, in Docker and Kubernetes containers are run either as the user specified by the USER directive in the Dockerfile, or as the root user if a USER directive is not specified. Containerized applications designed to run as the root user might not run as expected on OpenShift.

This article reviews the common issues I found when adapting containers from Docker and Kubernetes to run on Red Hat OpenShift. First, I describe potential areas to address so that containers can run on OpenShift securely without mandating a non-restricted Security Context Constraint (SCC). Then, I provide tips on how to create images that can run both on Kubernetes and OpenShift without modification. In addition, I provide techniques for debugging issues with applications that do not run as expected.

Group ownership and file permission

Although OpenShift runs containers using an arbitrarily assigned user ID, the group ID must always be set to the root group (0). Therefore, the directories and files that the processes running in the image need to access should have their group ownership set to the root group. They also need to be read/writable by that group as recommended by the OpenShift Container Platform-specific guidelines.

Adding the following to your Dockerfile sets the directory and file permissions to allow users in the root group to access them with the same authorization as the directory and file owner:

RUN chgrp -R 0 /some/directory && \
    chmod -R g=u /some/directory

Runtime user compatibility with Kubernetes

For this step, I suggest that you set the runtime user for Kubernetes to a non-root user for backward compatibility. You can perform this action by adding the following to the Dockerfile and then updating the file and directory permissions accordingly:

USER 1001
RUN chown -R 1001:0 /some/directory

The result, the specified user is ignored when the image is run on OpenShift because the user is set to an arbitrary ID. In contrast, when the image runs on Kubernetes, many of the OpenShift restrictions take effect as the container is run as a non-root user.

Good work. Runtime user compatibility helps to ensure that a single Dockerfile can be used to create an image that functions correctly, both on OpenShift and on Kubernetes.

Executable permissions

When containers run as the root user on Kubernetes, file permissions are ignored. In contrast, when an arbitrary user ID is used on OpenShift, set the permission bit in order to execute files.

For example, you add the following to set the execute permission bits to run for the owner and for the group:

RUN chmod 775 /some/directory/script

Volume mounts

In OpenShift, volume mounts are owned by user/group root:root and each is assigned the following permissions:

drwxrwx---

Note that Linux commands, such as chown(1), chgrp(1), chmod(1), cannot be performed on the volume mount point itself. However, you can create files or directories within the volume mount as the root group with full access permissions.

Privileged ports

TCP/IP port numbers below 1024 are privileged port numbers that enable only the root user to bind to these ports. When running a container on OpenShift, server applications need to be assigned port numbers greater than 1023.

Applications requiring the user's name

Applications sometimes fail on OpenShift when you attempt to look up the username for the currently running user ID. This problem occurs when there is no /etc/passwd entry for the arbitrarily assigned user ID. A workaround for this issue is to add an /etc/passwd entry for the arbitrary assigned user ID when the container starts.

For demonstration purposes, in the same directory as your Dockerfile, you need to create a file named, uid_entrypoint, with the following contents.

Remember to replace myuser with the name of the user you choose:

#!/bin/sh
if ! whoami &> /dev/null; then
  if [ -w /etc/passwd ]; then
    echo "myuser:x:$(id -u):0:My User:${HOME}:/sbin/nologin" >> /etc/passwd
  fi
fi
exec "$@"

Add the following to the Dockerfile, after replacing runcmd with the main script or program that runs in the container:

COPY uid_entrypoint /
RUN chmod g=u /etc/passwd && chmod 775 /uid_entrypoint
ENTRYPOINT ["uid_entrypoint"]
CMD ["runcmd"]

Note that, when the container starts running, you need to add a password entry for the specified username, if it does not already exist, and then the main process starts.

Deployments

Deployments to OpenShift can sometimes fail due to different behaviors between Kubernetes and OpenShift.

InitContainers won't help resolve permission issues

When you run in Kubernetes, initContainers are sometimes used to set the permissions of files and directories used by other containers in the pod. This advantage relies on Kubernetes running initContainers as the root user and running other containers as the user specified in the Docker directive USER.

When you run on OpenShift, both initContainers and regular containers use the OpenShift-assigned user ID. Therefore, the permissions in initContainers are exactly the same as the permissions in regular containers running in the same pod.

SecurityContext directives

Importantly, because OpenShift assigns an arbitrary user ID and a group ID of zero (0), SecurityContext directives, such as runAsUser and runAsGroup, must not appear in the deployment specifications (or Helm charts) when you run on OpenShift. Enabling SecurityContext directives causes the deployment to fail.

Avoid the OpenShift project default

Applications running in the OpenShift project default receive permissions similar to the permissions used when running on Kubernetes. OpenShift security restrictions are not applied to this project. Therefore, do not use this namespace for testing containers.

In addition, I recommend that you do not use the following OpenShift namespaces for running pods or services: default, kube-system, kube-public, openshift-node, openshift-infra, openshift.

How to debug issues

When migrating an image from Docker or Kubernetes to OpenShift, the image might not run out-of-the-box. For this reason, I recommend that you use the following tools and methods when debugging to find the root cause or the error:

• Check for errors logged in the system events for the current namespace by running:

  $ oc get events -w

• Check for errors in the pod logs by running:

  $ oc logs -f <podname> --all-containers

• Log into a container in the pod to check the file and permissions and other issues using the command:

  $ oc rsh <podname>

• When the container keeps starting and crashing, in the Dockerfile set:

  ENTRYPOINT ["sleep", "100000000"]

  Rerun the pod and log in to determine why issues are occurring.

• Install the strace(1) command in the container and trace the system calls of the running program to see which one fails.

Conclusion

In this article, you learned how to review common issues found when adapting containers from Docker and Kubernetes to OpenShift. I've also demonstrated how you can easily resolve these issues and create images that can be run on Docker, Kubernetes, and OpenShift.