Get started with OpenShift Streams for Apache Kafka

Red Hat OpenShift Streams for Apache Kafka is a managed cloud service for architects and developers working on event-driven applications. It provides hosted and managed, high-availability Kafka clusters on-demand, so you can focus on developing your applications while Red Hat takes care of the infrastructure.  In this learning path you’ll sign up for a free Red Hat account, provision a managed Kafka instance, and connect to it using service account credentials via SSL.

Apache Kafka clients are available for most major programming languages to produce and consume messages. The example in this section uses the kcat CLI so that no custom coding is required to produce and consume messages.

Create a file named initial_songs and paste the following content into the file. These are two messages that will be produced and consumed from the topic:

1: {"id": 1, "name": "The Ecstasy of Gold", "author":"Ennio Morricone", "op":"ADD"}

2: {"id": 2, "name": "Still Loving you", "author":"Scorpions", "op":"ADD"}

Open a terminal and run the following command:

$ kcat -b $BOOTSTRAP_SERVER \
-X security.protocol=SASL_SSL \
-X sasl.mechanisms=PLAIN \
-X sasl.username="$CLIENT_ID" \
-X sasl.password="$CLIENT_SECRET" -t songs -C -K:

This instructs kcat to run in consumer mode, which will print incoming messages from the songs topic and their keys separated by a colon from their values. If needed, you can press Ctrl-c to stop the consumer at any moment.

Open a second terminal and run the following command to produce the songs into the songs topic. This command starts a producer (-P option) that writes to the songs topic (-t option), and reads the messages line by line (-l option) from the initial_songs file. Text before the colon (-K: option) on each line in the file is treated as the message’s key.

$ kcat -b $BOOTSTRAP_SERVER \

-X security.protocol=SASL_SSL \

-X sasl.mechanisms=PLAIN \

-X sasl.username="$CLIENT_ID" \

-X sasl.password="$CLIENT_SECRET" -t songs -P -K: -l initial_songs

The kcat consumer running in the first terminal should have consumed and printed the messages like so:

1: {"id": 1, "name": "The Ecstasy of Gold", "author":"Ennio Morricone", "op":"ADD"}

2: {"id": 2, "name": "Still Loving you", "author":"Scorpions", "op":"ADD"}

% Reached end of topic songs [0] at offset 2

Note that the offset of this consumer has been updated to 2, as the two first records have been consumed from the topic.

Previous resource
Configuring account access permissions
Next resource
Changing topic retention