Installing AMQ Streams Operator from OperatorHub

1. Launch your web browser and open a new tab with the Administrator view of the web console for your OpenShift cluster. Log in to the console as an administrator.
2. After logging in, click Operators → OperatorHub in the left side navigation menu.
3. On the Project drop-down menu at the top of the OperatorHub page, select the project in which you want to install the operator.

4. There are quite a few different operators shown on the page. You can use the Filter by keyword box at the top of the page to quickly locate the AMQ Streams Operator. Just input the text “AMQ Streams” and click on the result operator, as shown in Figure 1.

   

   Figure 1: Locating the AMQ Streams Operator in OperatorHub.

5. Click the AMQ Streams operator. In the dialog box that appears, click Install as illustrated in Figure 2.

   

   Figure 2: Installing the AMQ Streams Operator.

   It will bring you to the AMQ Streams Operator configuration page, as illustrated in Figure 3.

   

   Figure 3: The AMQ Streams Operator configuration page.

   The operator supports watching just one namespace or the whole cluster. For this quick setup, we will use the default option and have the operator watch all namespaces.

6. Click Install. When the operator installation is complete, it will show you that the operator is ready to use, as illustrated in Figure 4.

   

   Figure 4: The installed AMQ Streams operator once ready for use.

7. Click View Operator, and you will get the installed AMQ Streams Operator details page.

Creating Apache Kafka® clusters with the AMQ Streams operator

1. On the Project drop-down menu at the top of the page, select the project for which you want to create the cluster.

   If there is no project already created, you can create a new one as illustrated in Figure 5.

   

   Figure 5: The Create Project window.

   Click on the operator name to get back to the details page.

2. On the operator details page, click Create instance under the Kafka box as illustrated in Figure 6. 

   

   Figure 6: The API options, each with an option to create a new instance.

3. Click on YAML View and paste the following manifest, as shown in Figure 7.

   apiVersion: kafka.strimzi.io/v1beta2
                   kind: Kafka
                   metadata: 
                     name: my-cluster
                   spec:
                     kafka:
                       replicas: 3
                       listeners:
                         - name: external
                           port: 9092
                           type: route
                           tls: true
                       storage:
                         type: ephemeral
                     zookeeper:
                       replicas: 3
                       storage:
                         type: ephemeral
                     entityOperator:
                       topicOperator: {}

   

   Figure 7: Defining the YAML for the new project.

   On the creation page, you can change or modify the basic template. For this quickstart, we will go with the defaults. This will deploy a 3-node Kafka cluster with 3 nodes ZooKeeper and enable one external listener for the cluster using ephemeral storage.

4. Click Create at the bottom of the page.

5. The console will show you the created Kafka instance and its status. Click on the Name of the Kafka instance to see more details. 

   

   Figure 8: Viewing the newly created Kafka instance on the Operator details page.

   If you scroll down, the details page shows more information about the instance, as illustrated in Figure 9.

   

   Figure 9: Details about the Kafka instance you just created.

This confirms you have successfully created one Apache Kafka cluster using AMQ Streams in Red Hat OpenShift.

Start sending and receiving from a Topic

Test using an external Red Hat build of Apache Camel application. You will need to have Maven installed and a Java 8 JDK.

1. Clone this Git repo to test access to your new Kafka cluster:

   $ git clone https://github.com/hguerrero/amq-examples.git    

2. Switch to the camel-kafka-demo folder:

   $ cd amq-examples/camel-kafka-demo/

3. Log in to the OpenShift cluster with cluster-admin privileges, for example:

   $ oc login -u system:admin

4. Make sure you are in the correct project namespace:

   $ oc project kafka

5. As we are using Routes for external access to the cluster, we need the cluster CA certificate to enable TLS in the client. Extract the public certificate of the broker certification authority:

   $ oc extract secret/my-cluster-cluster-ca-cert --keys=ca.crt --to=- > src/main/resources/ca.crt       

6. Import the trusted cert to a Keystore:

   $ keytool -import -trustcacerts -alias root -file src/main/resources/ca.crt -keystore src/main/resources/keystore.jks -storepass password -noprompt     

   Now you can run the Apache Camel application to send and receive messages to the Kafka cluster using the following Maven command (This step currently requires Java 8):

   $ BOOTSTRAP_SERVER="`oc get routes my-cluster-kafka-external-bootstrap -o=jsonpath='{.status.ingress[0].host}{"\n"}'`:443" mvn clean package spring-boot:run

   After finishing the clean and package phases, you will see the Spring Boot application start creating a producer and consumer, sending and receiving messages from the my-topic Kafka topic.

   14:36:18.170 [main] INFO  com.redhat.kafkademo.Application - Started Application in 12.051 seconds (JVM running for 12.917)
                   14:36:18.490 [Camel (MyCamel) thread #1 - KafkaConsumer[my-topic]] INFO  o.a.k.c.c.me.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=6de87ffa-c7cf-441b-b1f8-e55daabc8d12] Discovered coordinator my-cluster-kafka-1-myproject.192.168.99.100.nip.io:443 (id: 2147483646 rack: null)
                   14:36:18.498 [Camel (MyCamel) thread #1 - KafkaConsumer[my-topic]] INFO  o.a.k.c.c.me.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=6de87ffa-c7cf-441b-b1f8-e55daabc8d12] Revoking previously assigned partitions []
                   14:36:18.498 [Camel (MyCamel) thread #1 - KafkaConsumer[my-topic]] INFO  o.a.k.c.c.me.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=6de87ffa-c7cf-441b-b1f8-e55daabc8d12] (Re-)joining group
                   14:36:19.070 [Camel (MyCamel) thread #3 - KafkaProducer[my-topic]] INFO  producer-route - producer >>> Hello World from camel-context.xml with ID ID-hguerrer-osx-1540578972584-0-2
                   14:36:19.987 [Camel (MyCamel) thread #4 - KafkaProducer[my-topic]] INFO  producer-route - producer >>> Hello World from camel-context.xml with ID ID-hguerrer-osx-1540578972584-0-4
                   14:36:20.982 [Camel (MyCamel) thread #5 - KafkaProducer[my-topic]] INFO  producer-route - producer >>> Hello World from camel-context.xml with ID ID-hguerrer-osx-1540578972584-0-6
                   14:36:21.620 [Camel (MyCamel) thread #1 - KafkaConsumer[my-topic]] INFO  o.a.k.c.c.me.AbstractCoordinator - [Consumer clientId=consumer-1, groupId=6de87ffa-c7cf-441b-b1f8-e55daabc8d12] Successfully joined group with generation 1
                   14:36:21.621 [Camel (MyCamel) thread #1 - KafkaConsumer[my-topic]] INFO  o.a.k.c.c.me.ConsumerCoordinator - [Consumer clientId=consumer-1, groupId=6de87ffa-c7cf-441b-b1f8-e55daabc8d12] Setting newly assigned partitions [my-topic-0, my-topic-1, my-topic-2]
                   14:36:21.959 [Camel (MyCamel) thread #6 - KafkaProducer[my-topic]] INFO  producer-route - producer >>> Hello World from camel-context.xml with ID ID-hguerrer-osx-1540578972584-0-8
                   14:36:21.973 [Camel (MyCamel) thread #1 - KafkaConsumer[my-topic]] INFO  consumer-route - consumer >>> Hello World from camel-context.xml with ID ID-hguerrer-osx-1540578972584-0-8
                   14:36:22.970 [Camel (MyCamel) thread #7 - KafkaProducer[my-topic]] INFO  producer-route - producer >>> Hello World from camel-context.xml with ID ID-hguerrer-osx-1540578972584-0-11
                   14:36:22.975 [Camel (MyCamel) thread #1 - KafkaConsumer[my-topic]] INFO  consumer-route - consumer >>> Hello World from camel-context.xml with ID ID-hguerrer-osx-1540578972584-0-11
                   14:36:23.968 [Camel (MyCamel) thread #8 - KafkaProducer[my-topic]] INFO  producer-route - producer >>> Hello World from camel-context.xml with ID ID-hguerrer-osx-1540578972584-0-14  
7. You're done! Press Ctrl + C to stop running the program.