Enhance Kubernetes deployment efficiency with Argo CD and ApplicationSet

June 6, 2024
Luis Falero Otiniano
Related topics:
AutomationCI/CDGitOpsKubernetes
Related products:
Red Hat OpenShift

Using Argo CD to manage your Kubernetes ecosystem is an excellent GitOps approach to continuous deployment (CD). However, it can pose challenges when trying to manage multiple Kubernetes manifests for deployments, services, secrets, configurations, and other files within your Git repository.

Unlike an Argo CD Application resource that deploys resources from a single Git repository in a single cluster or namespace, ApplicationSet expands upon this use case. An ApplicationSet utilizes template-based automation to create, modify, and manage multiple Argo CD applications simultaneously, targeting multiple clusters and namespaces.

In what situations is it more convenient to use an ApplicationSet?

It is convenient to use an ApplicationSet when a user or team has too many applications to manage and needs to do the following:

  • Deploy to multiple Kubernetes clusters: Allows centralized management of distributed deployments across various environments.
  • Deploy to different namespaces: Facilitates organization and segregation of resources within the same cluster, enabling more orderly administration. 
  • Deploy to different namespaces in a single Kubernetes cluster: Provides flexibility to handle different development, testing, and production environments in a single cluster. 
  • Deploy from different Git repositories / folders / branches: Enables teams to work on multiple projects or versions simultaneously, ensuring that updates and changes are deployed efficiently and controlled.

Generators in Argo CD

Generators in Argo CD serve to automate the creation of Argo CD applications by providing dynamic parameters that integrate into templates. These parameters are used to define the applications and their configurations, making it easier to deploy and manage applications in multiple environments.

ApplicationSet includes a variety of generators, detailed below:

  • List generator: Allows targeting Argo CD applications to clusters based on a fixed list of cluster name/URL values.
  • Cluster generator: Targets Argo CD applications to clusters based on the list of clusters defined and managed by Argo CD.
  • Git generator: Creates applications based on files within a Git repository, or based on the directory structure of a Git repository.
  • Matrix generator: Used to combine the generated parameters of two separate generators.
  • Merge generator: Merges the generated parameters of two or more generators. Additional generators can override the values of the base generator.
  • SCM Provider generator: Uses the API of an SCM provider (e.g., GitHub) to automatically discover repositories within an organization.
  • Pull Request generator: Utilizes the API of an SCMaaS provider (e.g., GitHub) to automatically discover open pull requests within a repository.
  • Cluster Decision Resource generator: Used to interact with Kubernetes custom resources that use custom resource-specific logic to decide which set of Argo CD clusters to deploy to.

You can view all the supported generators for ApplicationSet in the Argo CD documentation.

List generator

You can configure ApplicationSet as follows:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: code-with-quarkus-application-set
  namespace: openshift-gitops
spec:
  generators:
    - list:
        elements:
          - namespace: redhat-dev
          - namespace: redhat-uat
          - namespace: redhat-prd
  template:
    metadata:
      name: "{{namespace}}-code-with-quarkus"
    spec:
      project: default
      source:
        repoURL: https://github.com/luisfalero/code-with-quarkus-manifest.git
        targetRevision: HEAD
        path: ./list-generator/manifests
      destination:
        server: https://api.cluster-zrgpw.dynamic.redhatworkshops.io:6443
        namespace: "{{namespace}}"
      syncPolicy:
        syncOptions:
          - CreateNamespace=true
        automated:
          prune: true
          selfHeal: true

Create the following application with the parameters:

We verify that a single application was deployed in three different namespaces within a single cluster with a single procedure. See Figure 1.

Figura 01: Application with List Generator
Figura 01: Application with List Generator
Figure 1: Application with List Generator.

We reviewed the details of the created ApplicationSet, as shown in Figure 2.

Figura 02: List Generator Detail
Figura 02: List Generator Detail
Figure 2: List Generator details.

We verified the application deployed in the three different namespaces, as shown in Figure 3.

Figura 03: List Generator Pods
Figura 03: List Generator Pods
Figure 3: List Generator pods.

Cluster generator

We confirmed that three OpenShift clusters have been added to Argo CD. See Figure 4.

Figure 04: Clusters in Argo CD
Figure 04: Clusters in Argo CD
Figure 4: Clusters in Argo CD.

These three clusters are reflected in three secrets within the openshift-gitops namespace. Figure 5 depicts this.

Figure 05: Secrets of Argo CD
Figure 05: Secrets of Argo CD
Figure 5: Secrets of Argo CD.

For this test case, we will add the labels dev:"true", uat:"true", and prd:"true" to the corresponding clusters.

kind: Secret
data:
  name: cluster-api.cluster-zrgpw.dynamic.redhatworkshops.io-2950750467
  namespace: openshift-gitops
metadata:
  labels:
    argocd.argoproj.io/secret-type: cluster
    dev: "true"
# ...output omitted...
kind: Secret
data:
  name: cluster-api.cluster-zzl8c.dynamic.redhatworkshops.io-688123982
  namespace: openshift-gitops
metadata:
  labels:
    argocd.argoproj.io/secret-type: cluster
    uat: "true"
# ...output omitted...
kind: Secret
data:
  name: cluster-api.cluster-294xw.dynamic.redhatworkshops.io-2256184837
  namespace: openshift-gitops
metadata:
  labels:
    argocd.argoproj.io/secret-type: cluster
    prd: "true"
# ...output omitted...

You can configure ApplicationSet as follows:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: code-with-quarkus-application-set
  namespace: openshift-gitops
spec:
  generators:
    - clusters:
        selector:
          matchLabels:
            dev: "true"
    - clusters:
        selector:
          matchLabels:
            uat: "true"
    - clusters:
        selector:
          matchLabels:
            prd: "true"
  template:
    metadata:
      name: "{{name}}-code-with-quarkus"
    spec:
      project: default
      source:
        repoURL: https://github.com/luisfalero/code-with-quarkus-manifest.git
        targetRevision: HEAD
        path: ./cluster-generator/manifests
      destination:
        server: "{{server}}"
        namespace: redhat
      syncPolicy:
        syncOptions:
          - CreateNamespace=true
        automated:
          prune: true
          selfHeal: true

Create the following application with the parameters:

We verified that a single application was deployed in three different clusters within a single namespace using a single procedure. See Figure 6.

Figura 06: Application with Cluster Generator
Figura 06: Application with Cluster Generator
Figure 6: Application with Cluster Generator.

We verified the application deployed in the three different clusters, as shown in Figure 7.

Figura 07: Cluster Generator Detail
Figura 07: Cluster Generator Detail
Figure 7: Cluster generator detail.

Git generator

You can configure ApplicationSet as follows:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: code-with-quarkus-application-set
  namespace: openshift-gitops
spec:
  generators:
    - git:
        repoURL: https://github.com/luisfalero/code-with-quarkus-manifest.git
        revision: HEAD
        directories:
          - path: git-generator/manifests/overlays/*
  template:
    metadata:
      name: "{{path.basename}}-code-with-quarkus"
    spec:
      project: default
      source:
        repoURL: https://github.com/luisfalero/code-with-quarkus-manifest.git
        targetRevision: HEAD
        path: "{{path}}"
      destination:        
        server: https://api.cluster-zrgpw.dynamic.redhatworkshops.io:6443
        namespace: "{{path.basename}}"
      syncPolicy:
        syncOptions:
          - CreateNamespace=true
        automated:
          prune: true
          selfHeal: true

We verified the contents of the git-generator folder.

git-generator
├── application-set.yaml
└── manifests
    ├── base
    │   ├── deployment.yaml
    │   ├── kustomization.yaml
    │   └── service.yaml
    └── overlays
        ├── development
        │   └── kustomization.yaml
        ├── production
        │   └── kustomization.yaml
        └── testing
            └── kustomization.yaml

Create the following application with the parameters:

We verified that a single application was deployed in three different namespaces within a single cluster using a single procedure. See Figure 8.

Figura 08: Application with Git Generator
Figura 08: Application with Git Generator
Figure 8: Application with Git Generator.

We reviewed the details of the created ApplicationSet (Figure 9).

Figura 09: Git Generator Detail
Figura 09: Git Generator Detail
Figure 9: Git Generator details.

We verified the application deployed in the three different namespaces. See Figure 10.

Figura 10: Git Generator Pods
Figura 10: Git Generator Pods
Figure 10: Git Generator pods.

Matrix generator

For this test case, we will add the labels remote-oc:"true" to the corresponding clusters.

kind: Secret
data:
  name: cluster-api.cluster-zrgpw.dynamic.redhatworkshops.io-2950750467
  namespace: openshift-gitops
metadata:
  labels:
    argocd.argoproj.io/secret-type: cluster
    dev: "true"
    remote-ocp: "true"
# ...output omitted...
kind: Secret
data:
  name: cluster-api.cluster-zzl8c.dynamic.redhatworkshops.io-688123982
  namespace: openshift-gitops
metadata:
  labels:
    argocd.argoproj.io/secret-type: cluster
    uat: "true"
    remote-ocp: "true"
# ...output omitted...
kind: Secret
data:
  name: cluster-api.cluster-294xw.dynamic.redhatworkshops.io-2256184837
  namespace: openshift-gitops
metadata:
  labels:
    argocd.argoproj.io/secret-type: cluster
    prd: "true"
    remote-ocp: "true"
# ...output omitted...

You can configure ApplicationSet as follows:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: code-with-quarkus-application-set
  namespace: openshift-gitops
spec:
  generators:
    - matrix:
        generators:
          - clusters:
              selector:
                matchLabels:
                  remote-ocp: "true"
          - list:
              elements:
                - namespace: redhat-dev
                - namespace: redhat-uat
                - namespace: redhat-prd
  template:
    metadata:
      name: "{{name}}-{{namespace}}-code-with-quarkus"
    spec:
      project: default
      source:
        repoURL: https://github.com/luisfalero/code-with-quarkus-manifest.git
        targetRevision: HEAD
        path: ./matrix-generator/manifests
      destination:
        server: "{{server}}"
        namespace: "{{namespace}}"
      syncPolicy:
        syncOptions:
          - CreateNamespace=true
        automated:
          prune: true
          selfHeal: true

Before creating the application, we added more pods to the controller, as nine applications will be deployed.

This is because it will be one application across three namespaces, distributed in three different clusters (Figure 11).

Figura 11: Argo CD Controller
Figura 11: Argo CD Controller
Figure 11: Argo CD controller.

Create the following application with the parameters:

We verify that a single application was deployed in three different namespaces across three clusters with a single procedure. See Figure 12.

Figura 12: Application with Matrix Generator
Figura 12: Application with Matrix Generator
Figure 12: Application with Matrix Generator.

We reviewed the details of the created ApplicationSet, as shown in Figure 13.

Figura 13: Matrix Generator Detail
Figura 13: Matrix Generator Detail
Figure 13: Matrix Generator details.

Conclusion

Using Argo CD with ApplicationSet and generators provides a robust and flexible solution for managing deployments in complex Kubernetes environments. The ability to deploy and manage multiple applications across various clusters and namespaces, as well as from different Git repositories, greatly simplifies the continuous deployment (CD) process and ensures orderly and efficient management of infrastructure resources. The combination of these tools offers a comprehensive solution for deployment and management needs in Kubernetes environments, enabling development teams to maintain an agile and secure workflow.