How to easily deploy OpenShift on Azure via GitOps

March 17, 2023
Ignacio Lago Pablo Castelo
Related topics:
ContainersGitOpsKubernetes
Related products:
Microsoft Azure Red Hat OpenShiftRed Hat Advanced Cluster Management for KubernetesRed Hat OpenShiftRed Hat OpenShift Container Platform

Deploying Red Hat OpenShift in Microsoft Azure utilizing OpenShift GitOps makes it not only easier but enables you to achieve faster time to market, better collaboration, more efficient workflows, and ensures access to the cloud's scalability, flexibility, and reliability.

In this article, we are going to follow the same process from our previous article but use a GitOps approach for this.

[ Learning path: Getting started with Microsoft Azure Red Hat OpenShift (ARO) ] 

Step 1: Setting up the credentials for Azure in GitOps

First of all, we have to fork the repository, so we can make changes to it since we are going to use the integration from the previous article. In between Red Hat Advanced Cluster Management for Kubernetes and Argo CD to deploy the changes (Argo CD reads a Git repository). Now we can take a look at the repo:

tree -L 2
.
├── README.md
├── bootstrap
│   ├── argocd
│   └── deploy
└── resources
    ├── 01-acm-operator
    ├── 02-azure-credentials
    └── 03-azure-cluster

7 directories, 1 file

We need to edit the folders for the 02-azure-credentials and 03-azure-cluster.

Let us take a look at what is inside the folders. The ./resources/02-azure-credentials are shown in the following snippet:

tree -L 2./resources/02-azure-credentials 
./resources/02-azure-credentials
├── 00-azure-credentials-namespace copy.yaml
├── 01-azure-credentials-secret.yaml
└── kustomization.yaml

0 directories, 3 files

Here we need to edit the file 01-azure-credentials-secret.yaml. Let us take a look at that file and what we have to replace. In this case, the baseDomainResourceGroupName¹, PullSecret², the baseDomain³, the SSH private⁵⁴ and public⁵ keys:

cat resources/02-azure-credentials/01-azure-credentials-secret.yaml 
---
apiVersion: v1
kind: Secret
type: Opaque
metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "1"
  name: azure-credentials
  namespace: azure-credentials
  labels:
    cluster.open-cluster-management.io/type: azr
    cluster.open-cluster-management.io/credentials: ""
stringData:
  # 1 resource group created for the DNS to be used as base in the installation
baseDomainResourceGroupName: openenv-jrh4l
  cloudName: AzurePublicCloud
 # 2 ClientId + ClientSecret + TenantId + SubscriptionId in json format
  osServicePrincipal.json: '{"clientId":"client_id_here","clientSecret":"client_secret_here","tenantId":"tenant_id_here","subscriptionId":"subscription_id_here"}'
  # 3 base DNS to be used as base in the installation
   baseDomain: jrh4l.azure.something.io
  # Your pull secret for pulling the images
  pullSecret: >
    {pull_secret_here}
  # Your Private Key for accessing the machines
4 ssh-privatekey: |
    -----BEGIN OPENSSH PRIVATE KEY-----
            private_key_here
    -----END OPENSSH PRIVATE KEY-----
  # Your Public Key for accessing the machines
5 ssh-publickey: >
    ssh-rsa
    public_key_here
  httpProxy: ""
  httpsProxy: ""
  noProxy: ""
  additionalTrustBundle: ""

Step 2: Install the cluster in Azure via GitOps

Let us take a look at what is inside the folders. The ./resources/02-azure-credentials are shown in the following snippet:

tree -L 2./resources/03-azure-cluster    
./resources/03-azure-cluster
├── 00-azure-cluster-namespace.yaml
├── 01-azure-managedclusterset copy.yaml
├── 02-azure-cluster.yaml
└── kustomization.yaml

0 directories, 4 files

Here we need to edit the file 02-azure-cluster.yaml. Let us take a look at that file and what we have to replace. In this case, the Cluster Name, Cluster Namespace, the baseDomain, the baseDomainResourceGroupName, Base64 encoding of the pull secret, install-config.yaml in base64 encoding, ssh-private key, osServicePrincipal.json:

cat resources/03-azure-cluster/02-azure-cluster.yaml 
---
apiVersion: v1
kind: Namespace
metadata:
  name: cluster01
---
apiVersion: hive.openshift.io/v1
kind: ClusterDeployment
metadata:
# 1 Cluster name in this case is cluster01
    name: 'cluster01'
# 2 Cluster namespace in this case is cluster01
   namespace: 'cluster01'
  labels:
    cloud: 'Azure'
    region: 'westeurope'
    vendor: OpenShift
    cluster.open-cluster-management.io/clusterset: 'azure-clusters'
spec:
# 3 base DNS to be used as base in the installation
   baseDomain: jrh4l.azure.something.io
  clusterName: 'cluster01'
  controlPlaneConfig:
    servingCertificates: {}
  installAttemptsLimit: 1
  installed: false
  platform:
    azure:
# 4 resource group created for the DNS to be used in the installation
       baseDomainResourceGroupName: openenv-jrh4l
      credentialsSecretRef:
        name: cluster01-azure-creds
      region: westeurope
      cloudName: AzurePublicCloud
  provisioning:
    installConfigSecretRef:
      name: cluster01-install-config
    sshPrivateKeySecretRef:
      name: cluster01-ssh-private-key
    imageSetRef:
       #quay.io/openshift-release-dev/ocp-release:4.10.53-x86_64
      name: img4.10.53-x86-64-appsub
  pullSecretRef:
    name: cluster01-pull-secret
---
apiVersion: cluster.open-cluster-management.io/v1
kind: ManagedCluster
metadata:
  labels:
    cloud: Azure
    region: westeurope
    name: 'cluster01'
    vendor: OpenShift
    cluster.open-cluster-management.io/clusterset: 'azure-clusters'
  name: 'cluster01'
spec:
  hubAcceptsClient: true
---
apiVersion: hive.openshift.io/v1
kind: MachinePool
metadata:
  name: cluster01-worker
  namespace: 'cluster01'
spec:
  clusterDeploymentRef:
    name: 'cluster01'
  name: worker
  platform:
    azure:
      osDisk:
        diskSizeGB: 128
      type: Standard_D2s_v3
      zones:
      - "1"
      - "2"
      - "3"
  replicas: 3
---
apiVersion: v1
kind: Secret
metadata:
  name: cluster01-pull-secret
  namespace: 'cluster01'
data:
# 5 Base64 encoding of the pull secret
   .dockerconfigjson: >-
    pull_secret_in_base64
type: kubernetes.io/dockerconfigjson
---
apiVersion: v1
kind: Secret
metadata:
  name: cluster01-install-config
  namespace: 'cluster01'
type: Opaque
data:
# 6 Base64 encoding of install-config yaml
   install-config.yaml: YXBpVmVyc2lvbjogdjENCm1ldGFkYXRhOg0KICBuYW1lOiAnY2x1c3RlcjAxJw0KIyBiYXNlIEROUyB0byBiZSB1c2VkIGFzIGJhc2UgaW4gdGhlIGluc3RhbGxhdGlvbg0KYmFzZURvbWFpbjoganJoNGwuYXp1cmUuc29tZXRoaW5nLmlvDQpjb250cm9sUGxhbmU6DQogIGFyY2hpdGVjdHVyZTogYW1kNjQNCiAgaHlwZXJ0aHJlYWRpbmc6IEVuYWJsZWQNCiAgbmFtZTogbWFzdGVyDQogIHJlcGxpY2FzOiAzDQogIHBsYXRmb3JtOg0KICAgIGF6dXJlOg0KICAgICAgb3NEaXNrOg0KICAgICAgICBkaXNrU2l6ZUdCOiAxMjgNCiAgICAgIHR5cGU6ICBTdGFuZGFyZF9ENHNfdjMNCmNvbXB1dGU6DQotIGh5cGVydGhyZWFkaW5nOiBFbmFibGVkDQogIGFyY2hpdGVjdHVyZTogYW1kNjQNCiAgbmFtZTogJ3dvcmtlcicNCiAgcmVwbGljYXM6IDMNCiAgcGxhdGZvcm06DQogICAgYXp1cmU6DQogICAgICB0eXBlOiAgU3RhbmRhcmRfRDJzX3YzDQogICAgICBvc0Rpc2s6DQogICAgICAgIGRpc2tTaXplR0I6IDEyOA0KICAgICAgem9uZXM6DQogICAgICAtICIxIg0KICAgICAgLSAiMiINCiAgICAgIC0gIjMiDQpuZXR3b3JraW5nOg0KICBuZXR3b3JrVHlwZTogT1ZOS3ViZXJuZXRlcw0KICBjbHVzdGVyTmV0d29yazoNCiAgLSBjaWRyOiAxMC4xMjguMC4wLzE0DQogICAgaG9zdFByZWZpeDogMjMNCiAgbWFjaGluZU5ldHdvcms6DQogIC0gY2lkcjogMTAuMC4wLjAvMTYNCiAgc2VydmljZU5ldHdvcms6DQogIC0gMTcyLjMwLjAuMC8xNg0KcGxhdGZvcm06DQogIGF6dXJlOg0KICAgICMgcmVzb3VyY2UgZ3JvdXAgY3JlYXRlZCBmb3IgdGhlIEROUyB0byBiZSB1c2VkIGFzIGJhc2UgaW4gdGhlIGluc3RhbGxhdGlvbg0KICAgIGJhc2VEb21haW5SZXNvdXJjZUdyb3VwTmFtZTogb3BlbmVudi1qcmg0bA0KICAgIGNsb3VkTmFtZTogQXp1cmVQdWJsaWNDbG91ZA0KICAgIHJlZ2lvbjogd2VzdGV1cm9wZQ0KcHVsbFNlY3JldDogIiIgIyBza2lwLCBoaXZlIHdpbGwgaW5qZWN0IGJhc2VkIG9uIGl0J3Mgc2VjcmV0cw0Kc3NoS2V5OiB8LQ0KICAgIHNzaC1yc2ENCiAgICBwdWJsaWNfa2V5X2hlcmUNCg==
---
apiVersion: v1
kind: Secret
metadata:
  name: cluster01-ssh-private-key
  namespace: 'cluster01'
stringData:
# 7  we have to put our private ssh key in base64 here
    ssh-privatekey: >-
    private_ssh_key_in_base64
type: Opaque
---
apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: cluster01-azure-creds
  namespace: 'cluster01'
stringData:
# 8 we have to put the azure credentials in base 64 here 
   osServicePrincipal.json: >-
    azure_credentials_in_base64
---
apiVersion: agent.open-cluster-management.io/v1
kind: KlusterletAddonConfig
metadata:
  name: 'cluster01'
  namespace: 'cluster01'
spec:
  clusterName: 'cluster01'
  clusterNamespace: 'cluster01'
  clusterLabels:
    cloud: Azure
    vendor: OpenShift
  applicationManager:
    enabled: true
  policyController:
    enabled: true
  searchCollector:
    enabled: true
  certPolicyController:
    enabled: true
  iamPolicyController:
    enabled: true

Before deploying it, we need to change the application sets so it points to our newly forked repository. Take a look at the deploy folder in the following snippets:

tree -L 2 bootstrap/deploy 
bootstrap/deploy
├── 00-applicationset-acm
│   ├── 00-acm-appproject.yaml
│   ├── 01-acm-appset.yaml
│   └── kustomization.yaml
├── 01-applicationset-azure-credentials
│   ├── 00-azure-credentials-appproject.yaml
│   ├── 01-azure-credentials-appset.yaml
│   └── kustomization.yaml
└── 02-applicationset-azure-cluster
    ├── 01-azure-cluster-appset.yaml
    └── kustomization.yaml

3 directories, 8 files
cat bootstrap/deploy/01-applicationset-azure-credentials/01-azure-credentials-appset.yaml

---
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: azure-credentials
  namespace: openshift-gitops
spec:
  generators:
  - clusters: {} 
  template:
    metadata:
      name: azure-credentials-{{name}}
    spec:
      project: azure
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
      source:
# we have to replace this with our forked repository
        repoURL: https://github.com/ignaciolago/acm-ocp-azure.git
        targetRevision: main
        path: resources/02-azure-credentials
      destination:
        server: https://kubernetes.default.svc


cat bootstrap/deploy/02-applicationset-azure-cluster/01-azure-cluster-appset.yaml 


---
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: azure-cluster
  namespace: openshift-gitops
spec:
  generators:
  - clusters: {} 
  template:
    metadata:
      name: azure-cluster-{{name}}
    spec:
      project: azure
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
      source:
# we have to replace this with our forked repository
        repoURL: https://github.com/ignaciolago/acm-ocp-azure.git
        targetRevision: main
        path: resources/03-azure-cluster
      destination:
        server: https://kubernetes.default.svc

Now we are ready to deploy. Run the following commands:

oc apply -k bootstrap/deploy/01-applicationset-azure-credentials
oc apply -k bootstrap/deploy/02-applicationset-azure-cluster

We can track the process in the web GUI or get the logs from the pod in the namespace cluster01.

oc get pods -n cluster01

oc get logs pod-name -n cluster01

Quick and easy deployment OpenShift on Azure

Congrats on deploying OpenShift on Azure in only a few steps! In these two articles, we have shown two easy ways to deploy OpenShift on Azure: GUI and GitOps. Comment below if you have questions. As always, we welcome your feedback.

Last updated: October 26, 2023