Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat OpenShift AI
      Red Hat OpenShift AI
    • Red Hat Enterprise Linux AI
      Linux icon inside of a brain
    • Image mode for Red Hat Enterprise Linux
      RHEL image mode
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • Red Hat Developer Hub
      Developer Hub
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Image mode for Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of OpenJDK
    • Kubernetes

      • Red Hat OpenShift
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Virtualization
      • Red Hat OpenShift Lightspeed
    • Integration & App Connectivity

      • Red Hat Build of Apache Camel
      • Red Hat Service Interconnect
      • Red Hat Connectivity Link
    • AI/ML

      • Red Hat OpenShift AI
      • Red Hat Enterprise Linux AI
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Lightspeed
    • Developer tools

      • Red Hat Trusted Software Supply Chain
      • Podman Desktop
      • Red Hat OpenShift Dev Spaces
    • Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • API Catalog
    • Product Documentation
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

Canary deployment strategy with OpenShift Service Mesh

March 26, 2024
David Severiano
Related topics:
GitOpsHelmKubernetesMicroservices
Related products:
Red Hat OpenShift

Share:

    As we reviewed in my previous articles about blue/green deployment, a critical topic in cloud-native computing is the microservice architecture. We are no longer dealing with one monolithic application. We have several applications that have dependencies on each other and also have other dependencies like brokers or databases.

    Applications have their own life cycle, so we should be able to execute independent canary deployment. All the applications and dependencies will not change their version at the same time.

    Another important topic in the cloud-native space is continuous delivery. If we are going to have several applications doing canary deployment independently, we have to automate it. We will use Helm, Red Hat OpenShift Service Mesh, Red Hat OpenShift GitOps, and (of course) Red Hat OpenShift to help us.

    In the next steps, we will see a real example of how to install, deploy and manage the life cycle of cloud-native applications doing canary deployment using OpenShift Service Mesh.

    Let's start with some theory. After that, we will have a hands-on example.

    Canary deployment

    A canary deployment is a strategy where the operator releases a new version of their application to a small percentage of the production traffic. This small percentage can test the new version and provide feedback. If the new version is working well, the operator can increase the percentage, until all the traffic is using the new version. Unlike blue/green, canary deployments are smoother, and failures have limited impact.

    Shop application

    We are going to use very simple applications to test canary deployment. We have created two Quarkus applications: Products and Discounts. Figure 1 shows the shop applications.

    Figure 1: The two Quarkus shop applications created, products and discounts.
    Figure 1: The two Quarkus shop applications created, products and discounts.

    Products call Discounts to get the product's discount and expose an API with a list of products with its discounts.

    Shop Canary

    To achieve canary deployment with cloud-native applications using OpenShift Service Mesh, we have designed this architecture which you can see in Figure 2 as a simplification.

    Figure 2: Architecture designed to achieve blue/green deployment with cloud native applications.
    Figure 2: Architecture designed to achieve blue/green deployment with cloud native applications.

    OpenShift components - online:

    • Route, Gateway, and Virtual Services.
    • Services mapped to the deployment.

    In blue/green deployment, we always have an offline service to test the version that is not in production. In the case of canary deployment, we do not need it because progressively we will have the new version in production.

    We have defined an active or online service, products-umbrella-online. The final user will always use products-umbrella-online. When a new version is deployed, OpenShift Service Mesh will send the amount of traffic that has been defined in the Virtual Service. We have to take care of the number of replicas in the new release and the old release, based on the amount of traffic that we have defined in the virtual service.

    Shop umbrella Helm chart

    One of the best ways to package cloud-native applications is Helm. In canary deployment, it makes even more sense. We have created a chart for each application that does not know anything about canary deployment. Then we pack everything together in an umbrella Helm chart.

    Demo

    Prerequisites

    • Red Hat OpenShift 4.13 with admin rights
      • You can download Red Hat OpenShift Local for OCP 4.13.
      • Refer to the getting started guide.
    • Git
    • GitHub account
    • oc 4.13 CLI

    We have a GitHub repository for this demo. As part of the demo, you will have to make some changes and commits. So it is important that you fork the repository and clone it in your local.

    git clone https://github.com/your_user/cloud-native-deployment-strategies

    Install OpenShift GitOps

    Go to the folder where you have cloned your forked repository and create a new branch canary-mesh:

    git checkout -b canary-mesh 
    git push origin canary-mesh

    Log in to OpenShift as a cluster admin and install the OpenShift GitOps operator with the following command. This may take a few minutes.

    oc apply -f gitops/gitops-operator.yaml

    Once OpenShift GitOps is installed, an instance of Argo CD is automatically installed on the cluster in the openshift-gitops namespace and a link to this instance is added to the application launcher in OpenShift Web Console.

    Log in to the Argo CD dashboard

    Argo CD upon installation generates an initial admin password which is stored in a Kubernetes secret. To retrieve this password, run the following command to decrypt the admin password:

    oc extract secret/openshift-gitops-cluster -n openshift-gitops --to=-

    Click on Argo CD from the OpenShift web console application launcher and then log in to Argo CD with admin username and the password retrieved from the previous step.

    Configure OpenShift with Argo CD

    We are going to follow, as much as we can, a GitOps methodology in this demo. So we will have everything in our Git repository and use Argo CD to deploy it in the cluster.

    In the current Git repository, the gitops/cluster-config directory contains OpenShift cluster configurations such as:

    • Namespaces gitops.
    • Role binding for Argo CD to the namespace gitops.
    • OpenShift Service Mesh
    • Kiali Operator
    • OpenShift Elasticsearch Operator
    • Red Hat OpenShift distributed tracing platform

    Let's configure Argo CD to recursively sync the content of the gitops/cluster-config directory into the OpenShift cluster.

    Execute this command to add a new Argo CD application that syncs a Git repository containing cluster configurations with the OpenShift cluster.

    oc apply -f canary-service-mesh/application-cluster-config.yaml

    Looking at the Argo CD dashboard, you will notice that an application has been created.

    You can click on the cluster-configuration application to check the details of sync resources and their status on the cluster.

    Create the shop application

    We are going to create the application shop, which we will use to test canary deployment. Because we will make changes in the application's GitHub repository, we have to use the repository that you have just forked. Edit the file canary-service-mesh/application-shop-mesh.yaml and set your own GitHub repository in the reportURL and the OCP cluster domain in change_domain.

    apiVersion: argoproj.io/v1alpha1
    kind: Application
    metadata:
      name: shop
      namespace: openshift-gitops
    spec:
      destination:
        name: ''
        namespace: gitops
        server: 'https://kubernetes.default.svc'
      source:
        path: helm/quarkus-helm-umbrella/chart
        repoURL:  https://github.com/change_me/cloud-native-deployment-strategies.git
        targetRevision: canary-mesh
        helm:
          valueFiles:
            - values/values-mesh.yaml
          parameters:
          - name: "global.namespace"
            value: gitops
          - name: "domain"
            value: "change_domain"
      project: default
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
    oc apply -f canary-service-mesh/application-shop-mesh.yaml

    Looking at the Argo CD dashboard, you will notice that we have a new shop application.

    Test the shop application

    We have deployed the shop with Argo CD. We can test that it is up and running.

    We have to get the route that we have created.

    oc get routes shop-umbrella-products-route -n istio-system --template='https://{{.spec.host}}/products'

    Notice that in each microservice response, we have added metadata information to better see the version of each application. This will help us to see the changes while we do the canary deployment. We can see that the current version is v1.0.1:

    {
       "products":[
          {
             ...
             "name":"TV 4K",
             "price":"1500€"
          }
       ],
       "metadata":{
          "version":"v1.0.1", <--
          "colour":"none",
          "mode":"online"
       }
    }

    Products canary deployment

    We have already deployed the products version v1.0.1 with 2 replicas, and we are ready to use a new products version, v1.1.1, that has a new description attribute.

    Figure 3 has the current status.

    Figure 3: has the current status
    Figure 3: has the current status
    Figure 3: The current status.

    We have split a cloud-native canary deployment into three automatic steps:

    1. Deploy canary version for 10%.
    2. Scale canary version to 50%.
    3. Scale canary version to 100%.

    This is just an example. The key point here is that it's very easy for us to use the canary deployment that best fits our needs.

    Step 1: Deploy canary version for 10%

    We will deploy a new version, v1.1.1. To do that, we have already configured products-green with the new version, v1.1.1. And we have to edit the file helm/quarkus-helm-umbrella/chart/values/values-mesh.yaml and make some changes:

    1. In global.istio change the weight to send 10% of the traffic to the new version:

      global:
        istio:
          productsblueWeight: 90
          productsgreenWeight: 10
    2. Increase the number of replicas to be able to support 10% of the traffic in the new version:

      products-green:
        quarkus-base:
          replicaCount: 1
    3. Push the changes to start the deployment:

      git add .
      git commit -m "Deploy products v1.1.1 with 10% traffic"
      git push origin canary-mesh

    Argo CD will refresh the status after some minutes. If you don't want to wait, you can refresh it manually from Argo CD UI or configure the Argo CD Git webhook.

    Figure 4 has the current status.

    Figure 4: has the current status
    Figure 4: has the current status
    Figure 4: The current status from Step 1.

    In the products URL's response, you will have the new version in 10% of the requests.

    New revision:

    {
      "products":[
         {
            "discountInfo":{...},
            "name":"TV 4K",
            "price":"1500€",
            "description":"The best TV" <--
         }
      ],
      "metadata":{
         "version":"v1.1.1", <--
      }
    }

    Old revision:

    {
      "products":[
         {
            "discountInfo":{...},
            "name":"TV 4K",
            "price":"1500€"
         }
      ],
      "metadata":{
         "version":"v1.0.1", <--
      }
    }

    Step 2: Scale canary version to 50%

    Now we have to make the changes to send 50% of the traffic to the new version v1.1.1. We have to edit the file helm/quarkus-helm-umbrella/chart/values/values-mesh.yaml.

    1. In global.istio, change the weight to send 50% of the traffic to the new version:

      global:
        istio:
          productsblueWeight: 50
          productsgreenWeight: 50
    2. Increase the number of replicas to be able to support 50% of the traffic in the new version:

      products-green:
        quarkus-base:
          replicaCount: 2
    3. Push the changes to start the deployment:

      git add .
      git commit -m "Deploy products v1.1.1 with 50% traffic"
      git push origin canary-mesh

    Figure 5 has the current status.

    Figure 5: has the current status
    Figure 5: has the current status
    Figure 5: The current status from Step 2.

    In the products URL's response, you will have the new version in 50% of the requests.

    Step 3: Scale canary version to 50%

    Now we have to make the changes to send 100% of the traffic to the new version v1.1.1. We have to edit the file helm/quarkus-helm-umbrella/chart/values/values-mesh.yaml.

    1. In global.istio change the weight to send 50% of the traffic to the new version.

      global:
        istio:
          productsblueWeight: 0
          productsgreenWeight: 100
    2. We can decrease the number of replicas in the old version because it will not receive traffic.

      products-blue:
        quarkus-base:
          replicaCount: 0
    3. Push the changes to start the deployment.

      git add .
      git commit -m "Delete product v1.0.1"
      git push origin canary-mesh

    Figure 6 has the current status.

    Figure 5: has the current status
    Figure 5: has the current status
    Figure 5: The current status from Step 3.

    In the products URL's response, you will only have the new version v1.1.1.

    {
      "products":[
         {
            "discountInfo":{...},
            "name":"TV 4K",
            "price":"1500€",
            "description":"The best TV" <--
         }
      ],
      "metadata":{
         "version":"v1.1.1", <--
      }
    }

    Delete environment

    To delete all the things that we have done for the demo, you have to:

    • In GitHub, delete the branch canary-mesh.
    • In Argo CD, delete the applications cluster-configuration and shop.
    • In OpenShift, go to project openshift-operators and delete the installed operators OpenShift GitOps, OpenShift Service Mesh, Kiali Operator, OpenShift Elasticsearch Operator, Red Hat OpenShift distributed tracing platform.
    Last updated: May 7, 2024

    Related Posts

    • Blue/green deployment strategy with OpenShift Pipelines

    • Blue/green deployment strategy with Argo Rollouts

    • Comparing OpenShift Service Mesh and Service Interconnect

    • 3 patterns for deploying Helm charts with Argo CD

    • Troubleshooting "no healthy upstream" errors in Istio service mesh

    • Monolith to microservices: Breaking down apps the useful way

    Recent Posts

    • Our top 10 articles of 2025 (so far)

    • The benefits of auto-merging GitHub and GitLab repositories

    • Supercharging AI isolation: microVMs with RamaLama & libkrun

    • Simplify multi-VPC connectivity with amazon.aws 9.0.0

    • How HaProxy router settings affect middleware applications

    What’s up next?

    Download the GitOps Cookbook for useful recipes and examples for successful hands-on applications development and deployment with GitOps.

    Get the e-book
    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit
    © 2025 Red Hat

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue