How to use OpenShift GitOps to deploy applications

This article discusses how to use Red Hat OpenShift GitOps to automate application deployment. With a specific pattern, you will be able to deploy multiple applications to different environments once the app is committed to a Git repo. Follow this semi-guided demo to set up an automated method to deploy your application to Red Hat OpenShift clusters using the GitOps approach.

In this article, we will demonstrate the following processes:

1. Automating continuous delivery with Red Hat OpenShift Pipelines and continuous deployment with OpenShift GitOps.
2. Deploying an application to different environments.
3. Promoting from lower to upper environments.

Prerequisites

The following prerequisites are required to replicate this demo:

• Red Hat OpenShift 4.9 or later
• OpenShift GitOps Operator
• OpenShift Pipelines Operator
• Basic knowledge of OpenShift GitOps (Argo CD) applications and application sets.
• Basic knowledge of OpenShift Pipelines (Tekton).

Our advice for using a GitOps approach

The primary goal of the GitOps approach is to simplify the deployment of application workloads across multiple environments. This is accomplished using a single Git repository (monorepo) and organizing environments with a directory structure that supports flexibility and reuse. We also recommend using a mainline development approach with a single branch representing the source of truth for all environments. Use branches and merge requests to enable a testing, review, and approval workflow when propagating changes to various environments.

The Git repos for this demo:

• Application source code
• GitOps repo

GitOps architecture

The diagram in Figure 1 illustrates the architecture of the GitOps approach. To demonstrate the concepts, the different environments are represented as unique namespaces within a single cluster, not separate clusters. These namespaces mimic separate clusters from a conceptual perspective. GitHub, OpenShift Pipelines, Red Hat Quay, Kustomize, and OpenShift GitOps are the tools used in this approach.

OpenShift Pipelines

In this demonstration, we will model a simplistic continuous integration pipeline that clones an application source code repository, builds an application container image, pushes the resulting container image to an enterprise container registry (Quay), and updates a deployment manifest located in Git. We will implement these pipeline stages using common tasks that are available with OpenShift Pipelines. Finally, we will deploy the latest image to the cluster with continuous delivery using OpenShift GitOps.

The directory structure provides flexibility

This example GitOps approach utilizes a directory structure that allows for re-use across environments while also providing flexibility for customization. The directories are organized as follows.

Application resources

All of the applications resources such as deployments, ConfigMaps, routes, and services are located under the org-services directory.

Common directory:

common
├── config-base
│   ├── configmap.yaml
│   └── kustomization.yaml
├── manifest-base
│   ├── base-deployment.yaml
│   ├── base-deploymentorg.yaml
│   ├── base-route.yaml
│   ├── base-service.yaml
│   └── kustomization.yaml
├── namespaces
│   ├── kustomization.yaml
│   ├── namespace1.yaml
│   └── namespace2.yaml
└── overlays
    ├── dev
    │   ├── configmap-1.yaml
    │   └── kustomization.yaml
    └── test
        ├── endpoints-common.yaml
        ├── kustomization.yaml
        └── test-configMap.yaml

6 directories, 15 files

The common resources across all of the applications on all of the environments:

• config-base: Contains the manifests for shared ConfigMaps across applications.
• manifest-base: Contains the base of the other resources such as deployment, routes, and services.
• namespaces: Contains the manifests the namespaces to be created.
• overlays: Contains subdirectory for each environment.

From each of these directories, OpenShift GitOps will create the shared ConfigMaps for each environment.

It’s important to note that we have separate directories for the ConfigMaps and the namespaces to avoid duplicate creation and conflicts since the name of these resources are the same across all of the application.

Under the services directory, create a subdirectory for each namespace. Then create another subdirectory for every service to be deployed under that specific namespace.

Each service has the following layout:

Base:

php-hello/base
├── configmap.yaml
└── kustomization.yaml

0 directories, 2 files

This contains the shared resource for each specific service across all environments for a specific namespace. The kustomization.yaml file under this directory patches the resources from the common directory by adding or updating the service specific fields and generates the YAML files.

  1 apiVersion: kustomize.config.k8s.io/v1beta1
  2 kind: Kustomization
  3 
  4 resources:
  5 - configmap.yaml
  6 -../../../../common/manifest-base/
  7 
  8 patches:
  9 - patch: |-
 10     - op: replace
 11       path: /metadata/labels/app
 12       value: php-hello
 13     - op: replace
 14       path: /metadata/name
 15       value: php-hello
 16     - op: replace
 17       path: /spec/selector/matchLabels/app
 18       value: php-hello
 19     - op: replace
 20       path: /spec/template/metadata/labels/app
 21       value: php-hello
 22     - op: replace
 23       path: /spec/template/spec/containers/0/name
 24       value: php-hello
 25   target:
 26     kind: Deployment
 27 - patch: |-
 28     - op: replace
 29       path: /metadata/labels/app
 30       value: php-hello
 31     - op: replace
 32       path: /metadata/name
 33       value: php-hello
 34     - op: replace
 35       path: /spec/to/name
 36       value: php-hello
 37   target:
 38     kind: Route
 39 - patch: |-
 40     - op: replace
 41       path: /metadata/labels/app
 42       value: php-hello
 43     - op: replace
 44       path: /metadata/name
 45       value: php-hello
 46     - op: replace
 47       path: /spec/selector/app
 48       value: php-hello
 49   target:
 50     kind: Service

Overlay:

overlays
├── dev-stable
│   ├── configmap.yaml
│   ├── kustomization.yaml
│   ├── replicas.yaml
│   └── version.yaml
├── dev-unstable
│   ├── configmap.yaml
│   ├── kustomization.yaml
│   ├── replicas\ copy.yaml
│   ├── replicas.yaml
│   ├── version\ copy.yaml
│   └── version.yaml
├── prod
│   ├── configmap.yaml
│   ├── kustomization.yaml
│   ├── replicas.yaml
│   └── version.yaml
└── test
    ├── configmap.yaml
    ├── kustomization.yaml
    ├── replicas.yaml
    └── version.yaml

4 directories, 18 files

The overlay contains a directory for each environment, which contains the patched resources for that specific environment. The Argo CD application points to this directory to deploy the application to a specific environment. The list of files and the contents could vary from one environment to another.

Let us take a look at the list of files under the dev-stable environment directory to better understand this.

• Configmap.yaml

  1 apiVersion: v1
  2 data:
  3   ENVIRONMENT: 'Development'
  4 kind: ConfigMap
  5 metadata:
  6   name: php-hello

This original file exists under the base directory. However, it is listed here since it needs another value for this specific environment. Kustomize patches and combines them to create a full environment specific ConfigMap.

• Replicas.yaml

To patch the deployment with the desired replicas for each environment:

  1 apiVersion: apps/v1
  2 kind: Deployment
  3 metadata:
  4   name: php-hello
  5 spec:
  6   replicas: 1

• Version.yaml

This file gets updated by the continuous integration pipeline when a new container application image is created.

  1 apiVersion: apps/v1
  2 kind: Deployment
  3 metadata:
  4   name: php-hello
  5 spec:
  6   template:
  7     spec:
  8       containers:
  9       - image: quay.io/tkhussein/php-hello:2485c7e15c7ee4fe0ee15e4f71922d38144fd2e6
 10         name: php-hello

ApplicationSet resources

The ApplicationSet is the main driver for deploying the services to the cluster through Argo CD. The following are the directory structure and the ApplicationSets.

1. kustomize/applicationsets/kustomize-appset-global

kustomize-appset-global
├── dev
│   └── kustomization.yaml
├── stage
│   └── kustomization.yaml
└── test
    └── kustomization.yaml

3 directories, 3 files

Each subdirectory in this directory is the entry point and the main driver for deploying all of the applications. To start deploying to each environment, you need to create a single Argo CD application that points to the directory corresponding to the environment. Once the Argo CD application is created, all of the services that belong to that environment will be automatically deployed.

This is a sample of the Kustomize YAML file that drives all of the deployments:

  1 apiVersion: kustomize.config.k8s.io/v1beta1
  2 kind: Kustomization
  3 
  4 resources:
  5 -../../org-services/namespace1/overlays/dev
  6 -../../org-services/namespace2/overlays/dev
  7 
  8 # create namespaces appset.
  9 -../../namespaces
 10 # Create sealed secret appsets
 11 -../../sealed-secrets/dev

Lines 5 and 6 point to the ApplicationSets that creates applications for all services listed in that specific directory.

Line 9 is used to create the namespaces.

Line 11 creates sealed secrets.

2. The applicationsets/org-services directory contains a subdirectory for each namespace.

Within each namespace directory there are two ApplicationSets that are responsible for deploying shared resources as well as patching and deploying environment specific resources.

The Kustomize file within the overlays directories patches the ApplicationSet to include the details for that specific environment.

org-services
├── namespace1
│   ├── base
│   │   ├── common-appset.yaml
│   │   ├── kustomization.yaml
│   │   ├── php-hello-appset.yaml
│   │   └── service2-appset.yaml
│   └── overlays
│       ├── dev
│       │   └── kustomization.yaml
│       └── stage
│           └── kustomization.yaml
└── namespace2
    ├── base
    │   ├── babboon-appset.yaml
    │   ├── common-appset.yaml
    │   └── kustomization.yaml
    └── overlays
        ├── dev
        │   └── kustomization.yaml
        └── stage
            └── kustomization.yaml

10 directories, 11 files

The following is a sample Kustomize file under the overlays directories:

  1 apiVersion: kustomize.config.k8s.io/v1beta1
  2 kind: Kustomization
  3 
  4 resources:
  5 -../../base
  6 
  7 patches:
  8 - patch: |-
  9     - op: add
 10       path: /spec/generators/0/list
 11       value:
 12         elements:
 13         - cluster: dev-stable
 14           url: https://kubernetes.default.svc
 15           namespace: namespace1
 16   target:
 17     kind: ApplicationSet
 18 - patch: |-
 19     - op: replace
 20       path: /spec/template/spec/source/path
 21       value: kustomize/org-services/common/overlays/dev
 22   target:
 23       name: php-hello-common

This configuration is similar for the creation of services, namespaces, and sealed secrets.

Sealed secrets

Sealed secrets is a secure method to store encrypted secrets in Git and utilize a Kubernetes controller to deploy to a cluster. For more information about sealed secrets, check out the source readme.

Onboarding a new application

Once the directories and configurations for the service have been created, the following instructions describe how to deploy your service to the DEV cluster.

• Log in to DEV cluster’s Argo CD instance. Then click on + NEW APP as shown in Figure 2.

• Fill in the details, as shown in Figures 3 and 4.

Note: The GitOps repo is the path to the Kustomize file that contains the starting point to deploy to DEV environment.

• Verify the configuration you entered, then click CREATE, as shown in Figure 5.

• Once the application is successfully created, it will automatically deploy the new services to DEV cluster. Figure 6 shows these apps.

Updating the application

If the changes are only on the source code, then the developers make their changes and commit them to their branch. This will trigger the pipeline line to build the new image and push it to Quay. Then the pipeline updates the following YAML file in the GitOps repository with the new image tag:

kustomize/org-services/services/namespace1/php-hello/overlays/dev-stable/version.yaml

Argo CD detects the changes, then it syncs the application and deploys new pods.

If the changes are on the app resources, such as number of replicas or ConfigMaps, then changes occur on the GitOps repo. Start with DEV, then move on into the upper environment. All the changes should be picked up by Argo CD automatically.

Promoting across environments

1. To promote a new application version, copy the version file from DEV to the higher environment and commit the changes.
2. To promote resources in the common area, use an incremental process. For example, if there is a change in one of the ConfigMap shared across the environment, we make the changes on the overlay level. Once the test has been verified, it should be moved to the common area.

GitOps simplifies application deployment

The primary goal of the GitOps approach we demonstrated in this article is to simplify the deployment of application workloads across multiple environments. If you have questions, comment below. We welcome your feedback.