In Red Hat OpenShift, a build is the process of transforming source code and binaries into a container image that can run securely on a container platform. In this article, we'll introduce Project Shipwright, a framework for building container images on Kubernetes, and explore the future of builds in Red Hat OpenShift.
Note: Be sure to check out our session at Red Hat Summit 2021 (June 15 and 16), where we'll cover the topics in this article in more depth.
A history of OpenShift builds
Kubernetes changed the unit of delivery for software developers from compiled binaries and scripts to container images. Assembling a container image is not a simple task, requiring deep knowledge of new tools and technologies. Red Hat OpenShift 3 greatly simplified this task with the
Build API and Source-to-Image (S2I), allowing developers to build container images directly from code. However, much has changed since then, including the proliferation of container-based image-building tools.
The BuildConfig API
In Red Hat OpenShift 3, we introduced the
BuildConfig API, enabling developers to define Kubernetes-styled resources that execute image build operations on a Red Hat OpenShift cluster, as shown in Figure 1.
Red Hat OpenShift supports the following build strategies:
We introduced the custom build strategy with flexibility as a design goal. This gives users the option to define their own DIY build strategy if their use case requires customizations that aren't satisfied by existing build strategies.
Evolving from Docker to Buildah
Not only has the Kubernetes community evolved with the introduction of new tools to build container images, but OpenShift builds have also gone through their share of transformation. The underlying implementation of the build process was changed from Docker in Red Hat OpenShift 3 to Buildah in Red Hat OpenShift 4. The power of a good API abstraction lies in shielding users from changes in the underlying implementation. We've been reasonably successful in doing so.
The case for an open framework
While OpenShift builds have been a huge success, we're aware of new strategies and tools for building container images that have emerged in the Kubernetes community (see Figure 2).
Now Kubernetes users and operators have options for the tools and strategies that convert source code and binaries into a container image. To address this, we wanted to accomplish the following design goals:
- Allow users to give a minimalistic definition of what they wish to build.
- Empower cluster operators to declaratively define and tinker with the how of one or more image-building processes in Kubernetes.
This is where Project Shipwright comes in.
Shipwright is an open framework powered by Tekton. It lets developers use a single Kubernetes API to assemble source code and binaries into a container image without being exposed to the underlying tooling details needed to build images. The project also offers a streamlined Kubernetes API that platform operators and build tool authors can use to define how an application image may be assembled.
At the heart of every build are the following components:
- Source code: What you are trying to build.
- Output image: Where you are trying to deliver your application.
- Build strategy: How your application is assembled.
- Invocation: When you want to build your application.
Developers who use Docker or Podman are familiar with this process:
- Clone the source from a Git-based repository (the “what”).
- Build the container image (the “when” and “how”):
$ podman build -t registry.mycompany.com/myorg/myapp:latest .
- Push the container image to your registry (the “where”):
$ podman push registry.mycompany.com/myorg/myapp:latest
The Shipwright project introduces Kubernetes-native APIs that drive image builds on Kubernetes—namely,
The BuildStrategy API
The Shipwright community maintains the following ready-to-use build strategies that may be used as-is:
- Cloud Native Buildpacks
Alternatively, users can create new build strategies or extend existing ones.
ClusterBuildStrategy are related APIs that define how a given tool should be used to assemble an application. They are distinguished by their scope:
BuildStrategy objects are namespace-scoped, whereas
ClusterBuildStrategy objects are cluster-scoped.
The spec consists of a
buildSteps object, which looks and feels like a Kubernetes container specification. The following is an example spec for Kaniko, which can build an image from a Dockerfile within a container:
--- apiVersion: shipwright.io/v1alpha1 kind: ClusterBuildStrategy metadata: name: kaniko spec: buildSteps: - name: build-and-push image: gcr.io/kaniko-project/executor:v1.6.0 workingDir: $(params.shp-source-root) securityContext: runAsUser: 0 capabilities: add: - CHOWN - DAC_OVERRIDE - FOWNER - SETGID - SETUID - SETFCAP - KILL env: - name: DOCKER_CONFIG value: /tekton/home/.docker - name: AWS_ACCESS_KEY_ID value: NOT_SET - name: AWS_SECRET_KEY value: NOT_SET command: - /kaniko/executor args: - --skip-tls-verify=true - --dockerfile=$(build.dockerfile) - --context=$(params.shp-source-context) - --destination=$(params.shp-output-image) - --oci-layout-path=/kaniko/oci-image-layout - --snapshotMode=redo - --push-retry=3 resources: limits: cpu: 500m memory: 1Gi requests: cpu: 250m memory: 65Mi volumeMounts: - name: layout mountPath: /kaniko/oci-image-layout - name: results image: registry.access.redhat.com/ubi8/ubi-minimal command: - /bin/bash args: - -c - | set -euo pipefail # Store the image digest grep digest /kaniko/oci-image-layout/index.json | sed -E 's/.*sha256([^"]*).*/sha256\1/' | tr -d '\n' > "$(results.shp-image-digest.path)" # Store the image size du -b -c /kaniko/oci-image-layout/blobs/sha256/* | tail -1 | sed 's/\s*total//' | tr -d '\n' > "$(results.shp-image-size.path)" resources: limits: cpu: 100m memory: 128Mi requests: cpu: 100m memory: 128Mi volumeMounts: - name: layout mountPath: /kaniko/oci-image-layout
The Build API
Build object provides a playbook on how to assemble your specific application. The most straightforward build consists of a Git source, a build strategy, and an output image:
apiVersion: shipwright.io/v1alpha1 kind: Build metadata: name: kaniko-golang-build annotations: build.build.dev/build-run-deletion: "true" spec: source: url: https://github.com/sbose78/taxi strategy: name: kaniko kind: ClusterBuildStrategy output: image: registry.mycompany.com/my-org/taxi-app:latest
BuildRun object invokes a build on your cluster. You can think of these as Kubernetes jobs or Tekton TaskRuns—they represent a workload on your cluster, ultimately resulting in a running pod.
apiVersion: shipwright.io/v1alpha1 kind: BuildRun metadata: name: kaniko-golang-buildrun spec: buildRef: name: kaniko-golang-build
As the build runs, the status of the
BuildRun object is updated with the current progress.
What's new in OpenShift builds
Shipwright forms the basis of OpenShift builds version 2, which will introduce support for popular build strategies such as Cloud Native Buildpacks on Red Hat OpenShift Container Platform.
Using the flexibility offered by the Shipwright APIs, we will offer supported ready-to-use build strategies on OpenShift Container Platform, namely:
- Cloud Native Buildpacks
Additionally, cluster operators will be able to define enterprise build strategies and offer them to users on the cluster.
How to install Shipwright
You can install Shipwright in one of two ways:
If your cluster has Operator Lifecycle Manager (OLM) installed, option 2 is the recommended approach. OLM will take care of installing Tekton as well as Shipwright.
Get involved with Project Shipwright
Project Shipwright, an open source project, is being actively developed by engineers from Red Hat and IBM CodeEngine. Please visit github.com/shipwright-io for more information. We are actively looking for contribution in the following areas:
- Shipwright controllers.
- Command-line interface.
- Installation via OperatorHub or Helm chart.
- Documentation website.
- Event-driven builds and integration with Tekton triggers.