Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat OpenShift AI
      Red Hat OpenShift AI
    • Red Hat Enterprise Linux AI
      Linux icon inside of a brain
    • Image mode for Red Hat Enterprise Linux
      RHEL image mode
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • Red Hat Developer Hub
      Developer Hub
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Image mode for Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of OpenJDK
    • Kubernetes

      • Red Hat OpenShift
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Virtualization
      • Red Hat OpenShift Lightspeed
    • Integration & App Connectivity

      • Red Hat Build of Apache Camel
      • Red Hat Service Interconnect
      • Red Hat Connectivity Link
    • AI/ML

      • Red Hat OpenShift AI
      • Red Hat Enterprise Linux AI
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Lightspeed
    • Developer tools

      • Red Hat Trusted Software Supply Chain
      • Podman Desktop
      • Red Hat OpenShift Dev Spaces
    • Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • API Catalog
    • Product Documentation
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

Create a Kubernetes Operator in Golang to automatically manage a simple, stateful application

December 16, 2020
Priyanka Jiandani
Related topics:
CI/CDGoKubernetesOperators
Related products:
Red Hat OpenShift

Share:

    A Kubernetes Operator acts as an automated site reliability engineer for its application, encoding the skills of an expert administrator in software. For example, an operator can manage a cluster of database servers and configure and manage its application. It can also install a database cluster of a declared software version and a designated number of members. The operator continues to monitor its application while it runs, and can automatically back up data, recover from failures, and upgrade the application over time.

    Cluster users employ kubectl and other standard tools to work with operators and their applications, thereby extending Kubernetes services. Operators use custom resources (CRs) to manage applications and their components. They follow Kubernetes principles, notably by using controllers (control loops).

    In this article, you will learn how to deploy a stateful application using a Kubernetes Operator. In this case, the operator uses the operator-sdk and a custom resource to deploy a WordPress site on MySQL.

    Note: Are you new to operators and operator patterns? Check out the Kubernetes documentation and the Kubernetes Operators e-Book to learn more.

    Example prerequisites

    To create Kubernetes Operator and use this demo, first install the following:

    • Golang v1.12+
    • The operator-sdk command-line interface (version 15)
    • A minikube start
    • The kubectl client

    Build and initialize the Kubernetes Operator

    To build the operator, start in $GOPATH/src directory. To initialize it, run the command:

    operator-sdk new wordpress-operator --type go --repo github.com/<github-user-name>/<github-repo-name>
    

    The output is represented as follows:

    INFO[0000] Creating new Go operator 'wordpress-operator’.
    INFO[0000] Created go.mod
    INFO[0000] Created tools.go
    INFO[0000] Created cmd/manager/main.go
    INFO[0000] Created build/Dockerfile
    INFO[0000] Created build/bin/entrypoint
    INFO[0000] Created build/bin/user_setup
    INFO[0000] Created deploy/service_account.yaml
    INFO[0000] Created deploy/role.yaml
    INFO[0000] Created deploy/role_binding.yaml
    INFO[0000] Created deploy/operator.yaml
    INFO[0000] Created pkg/apis/apis.go
    INFO[0000] Created pkg/controller/controller.go
    INFO[0000] Created version/version.go
    INFO[0000] Created .gitignore
    INFO[0000] Validating project

    After the operator is initialized, run the cd wordpress-operator command.

    Create custom resource definitions

    Custom resource definitions (CRDs) define our resources for interacting with the Kubernetes API. This is similar to how existing resources are defined, such as pods, deployments, services, persistent volume claims (PVCs), and so on. In this case, we specify the api-version of the format <group>/<version>, and create the kind custom resource. To create a CRD for a WordPress Operator, run the command:

    operator-sdk add api --kind Wordpress --api-version example.com/v1

    Now examine the file:

    deploy/crds/example.com_v1_wordpress_cr.yaml

    This is an example CRD of the generated type. It is prepopulated with the appropriate api-version and kind, and the resource name. Also, ensure that the spec section is completed with values relevant to the CRD we created. Examine the following file:

    deploy/crds/example.com_wordpresses_crd.yaml

    This file is the beginning of a CRD manifest. The SDK generates many of the fields related to the resource type's name. In the pkg/apis/example/v1/*_types.go file, address two struct objects called the spec object and the status object. Examine the following:

    // WordpressSpec defines the desired state of Wordpress  
    type WordpressSpec struct s
    {                                                                   
        SQLRootPassword string `json:"sqlrootpassword"`// the user will provide the root password through CR
    }

    Update the CRD with these changes by running the operator-sdk generate crds and operator-sdk generate k8s commands. This adds the spec specified in *_types.go to *crd.yaml.

    Set the controller

    Set a controller inside the operator pod to watch for changes to the custom resources and react accordingly. To start, generate the controller skeleton code using operator-SDK, for example:

    operator-sdk add controller --api-version=example.com/v1 --kind=Wordpress

    Next, edit the file to include the controller logic:

    pkg/controller/wordpress/wordpress_controller.go

    Creating watches

    The main reconcile function is called each time there are changes to the custom resource. Let's review the code line by line. Initially, the controller needs to add watches for the resources, so that Kubernetes can tell the controller about changes needed for the resources. The initial watch is created for the primary resource, Wordpress (in our case), that is monitored by the controller. For example:

    // Watch for changes to primary resource Wordpress
    err = c.Watch(&source.Kind{Type: &examplev1.Wordpress{}}, &handler.EnqueueRequestForObject{})
    if err != nil {
        return err
    }

    Next, we can create subsequent watches for child resources, such as pod, deployment, service, PVC, and so on. The operator uses this watch to support the primary resource. Create the watch for a child resource by specifying the value of OwnerType as a primary resource. For example:

    err = c.Watch(&source.Kind{Type: &appsv1.Deployment{}}, &handler.EnqueueRequestForOwner{
          IsController: True,
          OwnerType: &examplev1.Wordpress{},
    })
    if err != nil {
          return err
    }
    err = c.Watch(&source.Kind{Type: &corev1.Service{}}, &handler.EnqueueRequestForOwner{
        IsController: true,
        OwnerType: &examplev1.Wordpress{},
    })
    if err != nil {
       return err
    }
    err = c.Watch(&source.Kind{Type: &corev1.PersistentVolumeClaim{}}, &handler.EnqueueRequestForOwner{
        IsController: true,
        OwnerType: &examplev1.Wordpress{},
    })
    if err != nil {
         return err
    }

    Running the reconcile loop

    Now, run the reconcile function, also called the reconcile loop. This is where the actual logic resides. This function returns the reconcile.Result{} which indicates whether or not the reconcile loop needs to execute another pass. The possible outcomes based on the reconcile.Result{} return value are:

    Outcome Description
    return reconcile.Result{}, nil The reconcile process finished with no errors, so another iteration through the reconcile loop is not needed.
    return reconcile.Result{}, err The reconcile failed due to an error and Kubernetes needs to re-queue it and run it again.
    return reconcile.Result{Requeue: true}, nil The reconcile did not encounter an error, however, Kubernetes needs to re-queue it and run another iteration.

    Take the example:

    return reconcile.Result{RequeueAfter: time.Second*5}, nil

    Compare this example to the table's last entry. The watch waits for the specified amount of time—for example, five seconds—before re-running the request. This approach is useful when we are serially running multiple steps, although it might take longer to complete. If a back-end service needs a running database prior to starting, we can use this example to re-queue the reconcile function with a delay to give the database time to start. Once the database is running, the operator does not re-queue the reconcile request, and the rest of the steps continue. I recommend reviewing the Kubernetes API documentation, especially the core/v1 and apps/v1 directories, for more details.

    The reconcile function

    Next, let's consider the code for the reconcile function. Once again, we'll go line by line. Initially, the reconcile function retrieves the primary resource. For example:

    // Fetch the Wordpress instance
    wordpress := &examplev1.Wordpress{}
    err := r.client.Get(context.TODO(), request.NamespacedName, wordpress) -----1
    if err != nil {
          if errors.IsNotFound(err) {
           // Request object not found, could have been deleted after reconcile request.
           // Owned objects are automatically garbage collected. For additional cleanup logic use finalizers.
           // Return and don't requeue
              return reconcile.Result{}, nil
          }
       // Error reading the object - requeue the request.
       return reconcile.Result{}, err
    }
    
    // ensure that the child resources are running (example can be seen in below snippet)
    
    //if everything goes fine
    return reconcile.Result{}, nil

    The function checks whether the Wordpress resource already exists. The variable r is the reconciler object on which the reconcile function is called. client is the client for the Kubernetes API. Next, we create a child resource. Similar to the primary resource, the reconciler checks if the child resource is present by calling Get() for the Kubernetes client. If not, it creates the child resource in the target namespace. For example:

    found := &appsv1.Deployment{}
    err := r.client.Get(context.TODO(), types.NamespacedName{
        Name: dep.Name,
        Namespace: instance.Namespace,
    }, found)
    if err != nil && errors.IsNotFound(err) {
    
    // Create the deployment
    
            log.Info("Creating a new Deployment", Deployment.Namespace", dep.Namespace, "Deployment.Name", dep.Name)
            err = r.client.Create(context.TODO(), dep)   ------------------1
    
           if err != nil {
    
            // Deployment failed
             log.Error(err, "Failed to create new Deployment", "Deployment.Namespace", dep.Namespace, "Deployment.Name", dep.Name)
             return &reconcile.Result{}, err
             }
         // Deployment was created successfully
    
           return nil, nil
    
    }else if err != nil {
        // Error that isn't due to the deployment not existing
         log.Error(err, "Failed to get Deployment")
         return &reconcile.Result{}, err
    }
    // deployment successful
    return nil, nil

    The MySQL deployment instance

    Here is the code snippet for the MySQL deployment instance (dep). Note the bolded lines:

    labels := map[string]string{
    "app": cr.Name,
    }
    matchlabels := map[string]string{
    "app": cr.Name,
    "tier": "mysql",
    }
    
    dep := &appsv1.Deployment{
            ObjectMeta: metav1.ObjectMeta{
                 Name: "wordpress-mysql",
                 Namespace: cr.Namespace,
                 Labels: labels,
              },
    
            Spec: appsv1.DeploymentSpec{
                Selector: &metav1.LabelSelector{
                              MatchLabels: matchlabels,
                        },
                Template: corev1.PodTemplateSpec{
                               ObjectMeta: metav1.ObjectMeta{
                               Labels: matchlabels,
                                  },
                               Spec: corev1.PodSpec{
                                  Containers: []corev1.Container{{
                                  Image: "mysql:5.6",
                                  Name: "mysql",
                                  Env: []corev1.EnvVar{
                                       {
                                         Name: "MYSQL_ROOT_PASSWORD",
                                         Value: cr.Spec.SQLRootPassword,  ------1
                                        },
                                   },
    
                                  Ports: []corev1.ContainerPort{{
                                         ContainerPort: 3306,
                                         Name: "mysql",
                                          }},
                                  VolumeMounts: []corev1.VolumeMount{
                                                {
                                                  Name: "mysql-persistent-storage",
                                                  MountPath: "/var/lib/mysql",
                                               }, 
                                           }, 
                                       }, 
                                  },
    
                Volumes: []corev1.Volume{
                              {
                                  Name: "mysql-persistent-storage",
                                  VolumeSource: corev1.VolumeSource{
                                                PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{
                                                          ClaimName: "mysql-pv-claim",
                                                     },
                                                 },
                                          },
                                  },
                           },
                    },
             },
    }
    
    controllerutil.SetControllerReference(cr, dep, r.scheme) -------2

    Note the bolded lines:

    1. It's important to note the value for MYSQL_ROOT_PASSWORD, as taken from cr.Spec.
    2. This is the most critical line in the definition, as it establishes the parent-child relationship between the primary resource, Wordpress, and the child, deployment. We can also write similar code for child resources as pod, deployment, service, PVC, and so on. I recommend you also review the WordPress-Operator for more details.

    Run the Kubernetes WordPress Operator

    Now to run the WordPress Operator. First, make sure that minikube cluster is running using these commands:

    kubectl create-f ./deploy/crds/example.com_wordpresses_crd.yaml
    operator-sdk run --local

    In the next terminal run, use this command:

    kubectl apply -f ./deploy/crds/example.com_v1_wordpress_cr.yaml

    Once complete, the following logs display:

    INFO[0000] Running the operator locally in namespace default. 
    {"level":"info","ts":1598973876.2819793,"logger":"cmd","msg":"Operator Version: 0.0.1"}
    {"level":"info","ts":1598973876.2820053,"logger":"cmd","msg":"Go Version: go1.13.10"}
    {"level":"info","ts":1598973876.282011,"logger":"cmd","msg":"Go OS/Arch: linux/amd64"}
    {"level":"info","ts":1598973876.2820172,"logger":"cmd","msg":"Version of operator-sdk: v0.15.2"}
    {"level":"info","ts":1598973876.285575,"logger":"leader","msg":"Trying to become the leader."}
    {"level":"info","ts":1598973876.285611,"logger":"leader","msg":"Skipping leader election; not running in a cluster."}
    {"level":"info","ts":1598973876.5921307,"logger":"controller-runtime.metrics","msg":"metrics server is starting to listen","addr":"0.0.0.0:8383"}
    {"level":"info","ts":1598973876.596543,"logger":"cmd","msg":"Registering Components."}
    {"level":"info","ts":1598973876.5967476,"logger":"cmd","msg":"Skipping CR metrics server creation; not running in a cluster."}
    {"level":"info","ts":1598973876.5967603,"logger":"cmd","msg":"Starting the Cmd."}
    {"level":"info","ts":1598973876.5973437,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"wordpress-controller","source":"kind source: /, Kind="}
    {"level":"info","ts":1598973876.5975914,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"wordpress-controller","source":"kind source: /, Kind="}
    {"level":"info","ts":1598973876.5977812,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"wordpress-controller","source":"kind source: /, Kind="}
    {"level":"info","ts":1598973876.5979419,"logger":"controller-runtime.controller","msg":"Starting EventSource","controller":"wordpress-controller","source":"kind source: /, Kind="}
    {"level":"info","ts":1598973876.5980544,"logger":"controller-runtime.controller","msg":"Starting Controller","controller":"wordpress-controller"}
    {"level":"info","ts":1598973876.598183,"logger":"controller-runtime.manager","msg":"starting metrics server","path":"/metrics"}
    {"level":"info","ts":1598973876.6982796,"logger":"controller-runtime.controller","msg":"Starting workers","controller":"wordpress-controller","worker count":1}
    {"level":"info","ts":1598973876.6983802,"logger":"controller_wordpress","msg":"Reconciling Wordpress","Request.Namespace":"default","Request.Name":"example-wordpress"}
    {"level":"info","ts":1598973876.6984997,"logger":"controller_wordpress","msg":"Creating a new PVC","PVC.Namespace":"default","PVC.Name":"wp-pv-claim"}
    {"level":"info","ts":1598973876.7138047,"logger":"controller_wordpress","msg":"Creating a new Deployment","Deployment.Namespace":"default","Deployment.Name":"wordpress"}
    {"level":"info","ts":1598973876.736821,"logger":"controller_wordpress","msg":"Creating a new Service","Service.Namespace":"default","Service.Name":"wordpress"}
    {"level":"info","ts":1598973876.8298655,"logger":"controller_wordpress","msg":"Reconciling Wordpress","Request.Namespace":"default","Request.Name":"example-wordpress"}
    {"level":"info","ts":1598973876.8301716,"logger":"controller_wordpress","msg":"Creating a new Service","Service.Namespace":"default","Service.Name":"wordpress"}

    This example also shows the pod, deployment, service, PVC, and so on, as follows:

    [pjiandan@pjiandan crds]$ kubectl get po
    NAME                               READY   STATUS    RESTARTS   AGE
    wordpress-6d5b4988ff-dcxfj         1/1     Running   0          16h
    wordpress-mysql-59d5d89ff8-qj92r   1/1     Running   0          17h
    [pjiandan@pjiandan crds]$ kubectl get svc
    NAME              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
    kubernetes        ClusterIP   10.96.0.1       <none>        443/TCP        19h
    wordpress         NodePort    10.100.123.86   <none>        80:31881/TCP   16h
    wordpress-mysql   ClusterIP   None            <none>        3306/TCP       17h
    [pjiandan@pjiandan crds]$ kubectl get deploy
    NAME              READY   UP-TO-DATE   AVAILABLE   AGE
    wordpress         1/1     1            1           16h
    wordpress-mysql   1/1     1            1           17h
    [pjiandan@pjiandan crds]$ kubectl get pvc
    NAME             STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
    mysql-pv-claim   Bound    pvc-9ee52dce-b7b7-433d-8596-22392033e55e   10Gi       RWO            standard       17h
    wp-pv-claim      Bound    pvc-8674f3fa-acb3-4cd7-9283-5ecec8305945   10Gi       RWO            standard       16h
    

    Next, run the following command to return the IP address for the WordPress service:

    minikube service wordpress --url

    An example of this IP address response follows:

    http://192.168.99.101:31881

    Verify the site is running

    Last, copy the IP address and load the page in the browser to view the site, as shown in Figure 1.

    Initial WordPress page to load to browser .
    Figure 1: Copy the IP address and load the WordPress page to the browser.

    Conclusion

    This completes my demo and presentation! In this article, I demonstrated how to use a Kubernetes Operator to deploy a stateful application. This operator uses the operator-sdk project to deploy WordPress on MySQL using a custom resource. If you need to deploy a stateful application without an operator, see: Example: Deploying WordPress and MySQL with Persistent Volumes.

    Last updated: August 3, 2021

    Related Posts

    • 5 tips for developing Kubernetes Operators with the new Operator SDK

    • How to use third-party APIs in Operator SDK projects

    • Deploying Kubernetes Operators with Operator Lifecycle Manager bundles

    • Auto-generating news and publishing it to WordPress with Apache Camel

    Recent Posts

    • Storage considerations for OpenShift Virtualization

    • Upgrade from OpenShift Service Mesh 2.6 to 3.0 with Kiali

    • EE Builder with Ansible Automation Platform on OpenShift

    • How to debug confidential containers securely

    • Announcing self-service access to Red Hat Enterprise Linux for Business Developers

    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit
    © 2025 Red Hat

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue