How to implement Kubernetes operators with Java Operator SDK

This article will demonstrate how to implement a basic operator using Java Operator SDK, Quarkus, and Fabric8 Kubernetes client. Operators are Kubernetes extensions that use custom resources to manage applications and their components. Operators follow Kubernetes principles, such as the control loop.

The operator pattern concept lets you extend the cluster's behavior without modifying the code of Kubernetes by linking controllers to one or more custom resources. Operators are clients of the Kubernetes API that act as controllers for a custom resource. 

The purpose is to code the knowledge of a human operator who is managing a service or set of services. Human operators who look after specific applications and services have a deep knowledge of how the system ought to behave, how to deploy it, and how to react if there are problems.

You can find additional information in the official documentation:

• Kubernetes
• Kubernetes extension patterns
• Kubernetes operator pattern

Why use Java?

While Golang remains the most widely used language for implementing operators and controllers, not everyone is familiar with its concepts or pointers and reference style similar to C.

Java is very common in the software world. It uses a virtual machine to separate the programmer from the hardware and its object-oriented concepts highly human readable. Also, Fabric8, the popular Kubernetes client used in Java, has capabilities resembling Golang's clients.

The Java Operator SDK

The Operator SDK is capable of automatically generating a lot of the boilerplate code needed for operator implementation. This allows the user to focus on modeling and coding the knowledge, without worrying about network interaction with Kubernetes. Plug-ins are supported to extend the SDK's options. We will focus specifically on the Quarkus plug-in.

The Quarkus Java framework offers fast application start-up times, low memory consumption, and lower space requirements for native images. This should get our operator up and handling our custom resources efficiently. Quarkus also uses GraalVM, which supports changing code while the application is running.

Let's get started building our operator

We will create a simple Java operator, using the tools previously mentioned. This hands-on section demonstrates how the tooling helps us get a working extension to K8s API in a relatively quick way.

Prerequisites

• Operator SDK and Maven should be installed on your system. If you are using macOS, these can be installed with brew. For Linux, use the package managers available for your distribution such as dnf or apt, or download them from their websites: Operator SDK, Maven.
• Connection to a Kubernetes or Red Hat OpenShift cluster via kubeconfig.
• An IDE you are comfortable with should be available. In this example we will use VS Code with common Java extensions.
• We recommend familiarizing yourself with the Group Version Kind concept of Kubernetes resources.

The use case

Example.com, the company we work with, has a product named Echo, which repeats a user's input.  Deploying an Echo instance normally requires complex input and some logic.  We would like to code the knowledge needed to repeat the input in a new Kubernetes custom resource (Figure 1).

Step 1: Scaffolding the first Java operator

Create an empty directory named echo-operator. Then cd into it and run:

operator-sdk init --plugins quarkus --domain example.com --project-name echo-operator

Next, we'll follow up with scaffolding our API and open the IDE:

operator-sdk create api --group example --version v1 --kind EchoResource

Step 2: Custom resources in Java code

Focusing on the files created under src/main/java, let's look at the structure bottom-to-top. EchoResourceSpec is the spec inside our custom resource. This is where the users will be providing their input.

Add the following field with a Getter and Setter:

 private String inputMessage;

    public String getInputMessage() {
        return inputMessage;
    }

    public void setInputMessage(String inputMessage) {
        this.inputMessage = inputMessage;
    }

EchoResourceStatus is the output status our operator returns back to the user. Add the following output field:

    private String echoMessage;

    public String getEchoMessage() {
        return echoMessage;
    }

    public void setEchoMessage(String echoMessage) {
        this.echoMessage = echoMessage;
    }

EchoResource is the Java class representing our custom resource. It extends the Fabric8 class CustomResource for our spec and status. No changes required here.

Finally, we'll simulate an Echo Resource input given by the user.

Create a new file named cr-test-echo-resource.yaml under src/test/resources and paste the following content:

apiVersion: example.example.com/v1
kind: EchoResource
metadata:
  name: test-echo-resource
spec:
  inputMessage: "Hello from test-echo-resource"

Step 3: How Operator SDK implements the reconciler

Let's open the file named EchoResourceReconciler.

This class implements Operator SDK's Reconciler method for our echo resource. It is required to implement the method reconcile().

We will implement the control loop here with our knowledge about the Echo product, which repeats the user's input (Figure 2).

Add the following code for convenience:

private static final Logger log = LoggerFactory.getLogger(EchoResourceReconciler.class);

Then the implementation of reconcile:

log.info("This is the control loop of the echo-operator. resource message is {}", resource.getSpec().getInputMessage());
if (reconcileStatus(resource,context)){
      return UpdateControl.updateStatus(resource);
}
return UpdateControl.noUpdate();

And finally the implementation of handling status:

private boolean reconcileStatus(EchoResource resource, Context<EchoResource> context) {
    String desiredMsg = resource.getSpec().getInputMessage();
    if (resource.getStatus() == null){
      // initialize if needed
      resource.setStatus(new EchoResourceStatus());
      resource.getStatus().setEchoMessage("");
    }
    if (!resource.getStatus().getEchoMessage().equalsIgnoreCase(desiredMsg)){
       // the status needs to be updated with a new echo message
       resource.getStatus().setEchoMessage(desiredMsg);
       log.info("Setting echo resource status message to {}", desiredMsg);
       // return true to signal the need to update status in Kubernetes
       return true;
    }
    return false;
  }

Step 4: Testing a custom resource with live coding

For convenience, we will instruct Quarkus to create the custom resource definition on our cluster, in case it doesn't exist. Open src/main/resources/application.properties and change quarkus.operator-sdk.crd.apply to true.

Now, let's run Quarkus via Maven as follows:

mvn clean compile && mvn quarkus:dev

The controller is now running and ready to accept user input. We can follow up with our test resource: kubectl apply -f src/test/resources/cr-test-echo-resource.yaml.

Observe the output and check the status inside the EchoResource on the cluster. You can also change the input spec message again and see it updated in status.

We can also change the code by going back to EchoResourceReconciler and modifying the log message to: This is the reconciler of the echo-operator. Then press r in the Quarkus terminal.

Feel free to change and experiment. When you are done, exit out of Quarkus using q and clean up the test resource with kubectl delete -f src/test/resources/cr-test-echo-resource.yaml.

Wrap up

We have demonstrated how to implement a basic operator using the Java Operator SDK, Quarkus, and the Fabric8 Kubernetes client. You can re-run, modify code, experiment, and look at the files generated. Fabric8 is also capable of creating other resources in Kubernetes, via either a builder pattern or by reading an input template yaml.

Take a look at the following code snippets for additional experimentation:

Building a service with Fabric8's builders:

  private boolean reconcileService(EchoResource resource, Context<EchoResource> context) {
    String desiredName = resource.getMetadata().getName();

    Service echoService = client.services().withName(desiredName).get();
    if (echoService == null){
      log.info("Creating a service {}", desiredName);
      Map<String,String> labels = createLabels(desiredName);

      echoService = new ServiceBuilder()
       .withMetadata(createMetadata(resource, labels))
       .withNewSpec()
           .addNewPort()
               .withName("http")
               .withPort(8080)
           .endPort()
           .withSelector(labels)
           .withType("ClusterIP")
       .endSpec()
       .build();

    client.services().resource(echoService).createOrReplace();
    return true;
    }
    return false;
  }
  
  private Map<String, String> createLabels(String labelValue) {
    Map<String,String> labelsMap = new HashMap<>();
    labelsMap.put("owner", labelValue);
    return labelsMap;
  }
  
  private ObjectMeta createMetadata(EchoResource resource, Map<String, String> labels){
    final var metadata=resource.getMetadata();
    return new ObjectMetaBuilder()
       .withName(metadata.getName())
       .addNewOwnerReference()
           .withUid(metadata.getUid())
           .withApiVersion(resource.getApiVersion())
           .withName(metadata.getName())
           .withKind(resource.getKind())
       .endOwnerReference()
       .withLabels(labels)
   .build();
  }

Parsing and applying a YAML with a Kubernetes resource:

  private void createFromYaml(String pathToYaml) throws FileNotFoundException {
  // Parse a yaml into a list of Kubernetes resources
  List<HasMetadata> result = client.load(new FileInputStream(pathToYaml)).get();
  // Apply Kubernetes Resources
  client.resourceList(result).createOrReplace();
  }