Developing container-based applications with Kubernetes is a complex process. Kubernetes Operators offer a default way to extend Kubernetes by automating the deployment and life-cycle management tasks of the containerized applications that you are building.
This article is an overview of Kubernetes Operators: What they are, what they do, and key features you want to know when working with operators. Topics include the Operator SDK (Software Development Kit), operator capability levels, and the registry for Kubernetes Operators, OperatorHub.io.
Operators extend Kubernetes to automate tasks
Operators are extensions to Kubernetes that use custom resources to manage Kubernetes applications and their components. They automate software configuration and maintenance activities that are typically performed by human operators.
While Kubernetes is great at managing stateless applications, operators are useful when you need more complex configuration details for a stateful application such as a database. A stateful workload is more difficult to manage than a stateless workload. The state in a workload changes how a workload:
- Needs to be installed.
- Upgrades to a new version.
- Recovers from failures.
- Needs to be monitored.
- Scales out and back in again.
The operator takes care of everything needed to make sure the service runs successfully. An operator makes its service more self-managing, so the application team spends less effort managing the service and can spend more effort using the service.
What's in an operator?
Operators extend the Kubernetes control plane with specialized functionality to manage a workload on behalf of a Kubernetes administrator. An operator includes:
- A custom resource definition (CRD) that defines a schema of settings available for configuring the workload. An operator introduces new object types through its custom resource definition. The Kubernetes API handles these objects like native Kubernetes objects, including interaction via Kubernetes client tools and inclusion in role-based access control policies (RBAC).
- A custom resource (CR) that is created by a CRD and is the Kubernetes API extension. A user provides configuration settings within a CR, and then the operator translates the configuration into low-level actions using the operator’s custom controller logic to implement the translation.
- A controller that is customized for the workload and configures the current state of the workload to match the desired state that’s represented by the values in the CR.
The Red Hat article What is a Kubernetes operator? provides more details about operators.
How Kubernetes Operators reconcile state
In Kubernetes, controllers in the control plane run in a control loop that repeatedly compares the cluster's desired state to its current state. If the states don’t match, then the controller takes action to adjust the current state to more closely match the desired state.
Similarly, the controller in an operator watches a specific CR type and takes application-specific actions to make the workload’s current state match the desired state as expressed in the CR.
The diagram in Figure 1 illustrates how the control plane runs the controllers in a loop, where some controllers are built into Kubernetes, and some are part of operators.
The controllers in the control plane are optimized for stateless workloads, and one set of controllers works for all stateless workloads because they’re all similar. The controller in an operator is customized for one particular stateful workload. Each stateful workload has its own operator with its own controller that knows how to manage that workload.
Deploying an operator
To use an operator, you must first deploy the operator as a workload in your cluster. You can deploy it like any other workload via the Kubernetes CLI or using a Helm chart. Additionally, you can make use of the OperatorHub or deploy your operator through the Operator Lifecycle Manager.
Kubernetes Operator technologies
Operators are usually implemented in one of three main technologies:
- Go: Code written in Go is powerful and can do almost anything Kubernetes can do. Kubernetes itself is implemented in Go, so operators implemented in Go are a good fit.
- Ansible: Ansible is a good choice for infrastructure teams that have already written Ansible modules. Ansible is declarative and human-readable and expresses almost as much functionality as Go.
- Helm: Helm operators are simpler to implement, but functionality is limited to Helm features.
You can read more about the pros and cons of each approach in Build Your Kubernetes Operator With the Right Tool. The Operator SDK supports all three technologies.
The Operator SDK
The Operator SDK is a set of open source tools for building, testing, and packaging operators. You can use the SDK CLI to scaffold a project, and it also provides prebuilt commands to generate code. The SDK uses Make, a build automation tool that runs commands configured in a makefile to generate executable code and libraries from source code.
The Operator Lifecycle Manager
The SDK also lets you install the Operator Lifecycle Manager (OLM) using the command line. The OLM is a set of cluster resources that manage an operator's life cycle. Once installed, you can see the status of the OLM to verify whether the SDK can successfully communicate with the OLM components in the cluster.
Additional Operator SDK concepts
In addition to the operator, custom resource, and custom resource definition, you need to understand the Operator SDK’s operand and managed resources:
- Operand: The managed workload provided by the operator as a service.
- Managed resources: The Kubernetes objects or off-cluster services that the operator uses to constitute an operand (also known as secondary resources).
Operator capability levels
Some operators are more sophisticated at managing their operand’s life cycle than others. The Operator capability levels model defines five levels of sophistication, which are illustrated in Figure 2.
This model aims to guide the features that users can expect from a particular operator. As Figure 2 shows, you can only use Ansible and Go to achieve all five capability levels. Helm can only be used to achieve the first two levels.
Note: Capability levels build on top of one another. That means if an operator has Level 3 capabilities, then it should also have all of the capabilities required from Levels 1 and 2.
This article only covers the first capability level, but you can read more about the other four Operator capability levels in the Operator SDK documentation.
OperatorHub.io
OperatorHub.io is a web-based application that developers use to find, install, and publish operators. It is a one-stop-shop for Kubernetes operators, as shown in Figure 3.
Check out How to contribute an Operator for details of how to package, test, preview, and submit your operator for addition to the OperatorHub.
Summary
In this article, you learned how operators can extend the base Kubernetes functionality using custom controllers and custom resources. You also learned that the Operator SDK offers code scaffolding tools you can use to write your operator more easily, and it offers guidelines for the capability levels of an operator. Lastly, we introduced OperatorHub.io, where you can browse existing operators and submit your own.
The second half of this article will take a closer look at the Kubernetes architecture for operators.