Operator pattern: REST API for Kubernetes and Red Hat OpenShift

Operator pattern: REST API for Kubernetes and Red Hat OpenShift

In this article, we will see a similar pattern when writing the REST API in any known framework vs. writing an Operator using Kubernetes’ client libraries. The idea behind this article is not to explain how to write a REST API, but instead to explain the internals of Kubernetes by working with an analogy.

Local setup

To follow along, you will need the following installed:

As a developer, if you have used the REST API with frameworks like Quarkus/Spring (Java), Express (Nodejs), Ruby on Rails, Flask (Python), Golang (mux), etc., understanding and writing an operator will be easier for you. We will use this experience with other languages or frameworks to build our understanding.

By writing an operator, you implement a REST API using Kubernetes client libraries as your framework—i.e., extending the Kubernetes APIs with your custom logic. If you are running a single-cluster Kubernetes instance locally using CRC/Minishift/Minikube, etc., type this command into the terminal:

$ kubectl proxy

The Kubernetes API endpoints are as shown in Figure 1:

Kubernetes API endpoints.

Figure 1: Kubernetes API endpoints.

As you can see, each entity within the Kubernetes domain space is an exposed API endpoint. Whenever you trigger a deployment or scale up a pod, you hit these APIs behind the scenes.

We can start by writing a REST API in the framework that is familiar. I will use Spring Boot for building a simple Customer API. Later, we will write a customer-api-operator that extends Kubernetes’ api-server with our defined Customer API. If you are not familiar with Spring Boot, you can continue. The main idea is just to understand the analogy that writing the API using any language and using the Kubernetes client libraries follows nearly the same pattern.

Everything you need to grow your career.

With your free Red Hat Developer program membership, unlock our library of cheat sheets and ebooks on next-generation application development.

SIGN UP

Example: REST API

Let’s start the REST API side of this analogy by defining our Customer model class, as shown in Figure 2:

Customer model class definition.

Figure 2: Customer model class definition.

Next, we need the controller, which exposes a Customer REST endpoint and handles the request/response process. Let’s define three endpoints (current, desired, and reconcile) and a CustomerReconciler service that handles the business logic, as shown in Figure 3:

CustomerReconciler service definition.

Figure 3: CustomerReconciler service definition.

Start your Spring Boot application with:

$ mvn spring-boot:run

This command starts the application on the default port. Let’s hit our API to get the current state of our customers using values I mocked for this demo. In general, we fetch this information from other APIs or databases, but for this example Figure 4 shows the current state of our Kubernetes pods:

Figure 4: The artificially-created current pod state.

Now, let’s try to update the state of our Customer with a desired state using the POST method and JSON data as shown in Figure 5:

The JSON data.

Figure 5: The JSON state data.

In Kubernetes, updating the state from current to desired is handled by the reconcile endpoint, which shows the difference between the current state and the desired state. We defined our business logic to handle this state in the reconciler loop shown in Figure 6:

The reconciler state data.

Figure 6: The reconciler state data.

Now we have a Customer API that can get the customer’s current state, update the state, and reconcile it to handle the change.

Example: Custom operator

Now let’s see how to implement a similar API by extending the Kubernetes API server using the Operator SDK. First, generate the Operator:

$ operator-sdk new customer-api-operator

This action generates basic boilerplate code for our Customer API Operator, as shown in Figure 7:

Basic boilerplate code for our new API Operator.

Figure 7: Basic boilerplate code for our new API Operator.

Start by adding the model data type (i.e., Customer), where kind is an entity like a pod, node, deployment, etc. Next, add a custom API for the Kubernetes platform, which requires generating a new Custom Resource Definition (CRD) with an API version:

$ operator-sdk add api --api-version=akoserwal.io/v1alpha1 --kind=Customer

This command generates a basic scaffolding for our Customer API. Half of the work for defining a basic model is now already done by the SDK. You can look into the file customer_types.go (the same as your model class, customer.java, defined in the first example).

In Figure 8, you can see that Spec and Status are already pre-defined with the generated Customer struct and that we add the new entities from the Customer model class (i.e., FirstName, LastName, and Email):

The pre-defined Customer struct.

Figure 8: The pre-defined Customer struct.

After modifying customer_types.go, run these commands for deep copy and generating an OpenAPI spec for the Customer API:

$ operator-sdk generate k8s
$ operator-sdk generate openapi

Now, generate the controller:

$ operator-sdk add controller --api-version=akoserwal.io/v1alpha1 --kind=Customer

The generated boilerplate controller code adds our Customer API schema to the Kubernetes schema. Next, we add a couple of watchers to look out for any changes or events, which triggers the reconciler loop for handling the changes as you can see in Figure 9:

The controller boilerplate code.

Figure 9: The controller boilerplate code.

We can relate this result to the Spring Boot example’s CustomerController and reconciler endpoint. Although, this version does more than our Spring Boot controller.

You can see the simplified flow in Figure 10:

The simplified API flow.

Figure 10: The simplified API flow.

So, to reiterate, creating a custom Operator—instead of using a REST API in your chosen framework—flows as follows:

  1. Define a namespace for our operator (i.e.,Customer).
  2. Create the Customer Resource Definition (CRD), so define the “kind” (i.e., the customer as a resource in OpenShift/Kubernetes).
  3. Let the controller analyze the CRD and add that information to the Kubernetes API schema.
  4. Expose a new API endpoint for the CRD.
  5. Add watchers for the namespace to observe.
  6. Run a process/loop (i.e., reconciler loop) for acting based on the desired change.
  7. Store the state in the EtcD.

Now that the custom Operator is defined, let’s continue. Deploy the Operator:

$ crc start

Log into the CRC instance:

$ oc login -u kubeadmin -p <secret>

Create the custom CRD:

$ oc create -f deploy/crds/customer.github.io_customers_crd.yaml

Add the roles, role_binding, and service account for the customer. These settings define the permissions and rules regarding access to our Operator:

$ kubectl create -f deploy/role.yaml
$ kubectl create -f deploy/role_binding.yaml
$ kubectl create -f deploy/service_account.yaml

Run our Operator locally:

$ export NAMESPACE=customer
$ operator-sdk up local

Now our controller is in the running state. It watches the NAMESPACE for any changes.

Create an instance of the CRD by creating a Custom Resource object:

$ oc create -f deploy/crds/customer.github.io_v1alpha1_customer_cr.yaml

Once created, the CR looks like this:

apiVersion: customer.github.io/v1alpha1
kind: Customer
metadata:
  name: customer-api
spec:
  # Add fields here
  size: 3
firstName: Abhishek
lastName: Koserwal
email: ak..@redhat.com

When we create a CR object, an event is triggered that gives control to the reconciler loop. Reconcile reads the state of that cluster for a custom object and makes changes based on the state it reads.

Figure 11 shows the Customer Custom Resource in the OpenShift UI:

The new Custom Resource in the OpenShift API.

Figure 11: The new Custom Resource in the OpenShift API.

Figure 12 shows the Customer Custom Resource in its raw API format:

The new Custom Resource as a raw API.

Figure 12: The new Custom Resource as a raw API.

Based on the logic inside our reconciler loop, this CR object creates a pod for the instance customer-api:

// Define a new Pod object
pod := newPodForCR(instance)

Figure 13 shows such an instance:

Additional update logic.

Figure 13: Additional update logic.

Now, we can add update logic for defining the desired state for our existing Custom Resource. Let’s update the reconciler so it will update our CR (firstName, lastName, and Email) as shown in Figure 14:

customer_controller.go [func (r *ReconcileCustomer) Reconcile]

Additional update logic.

Figure 14: Additional update logic.

Now, re-run the Operator:

$ operator-sdk up local

Finally, in Figure 15, we can see that the reconciler has updated the desired state for the Custom Resource:

The updated desired state.

Figure 15: The updated desired state.

Conclusion

Operator patterns allow you to use Kubernetes platform capabilities to run custom business logic. These patterns are powerful, and they are capable of doing way more than what I have covered here. In this article, I focused on Kubernetes internals and Operator patterns with a relatable approach via the REST API. Writing these types of extensions is not the best use case for Operator patterns. You can run a Spring Boot—or any other language—application in a container for use cases such as “building a REST API application/service.” For actual use cases, visit operatorhub.io.

Thank you for reading, I hope you find this article helpful. You can find the code here. Happy coding.

References

To learn more, visit our Linux containers or microservices pages.

Join Red Hat Developer and get access to handy cheat sheets, free books, and product downloads that can help you with your microservices and container application development.

Share