Get started with OpenShift Streams for Apache Kafka

Red Hat OpenShift Streams for Apache Kafka is a managed cloud service for architects and developers working on event-driven applications. It provides hosted and managed, high-availability Kafka clusters on-demand, so you can focus on developing your applications while Red Hat takes care of the infrastructure.  In this learning path you’ll sign up for a free Red Hat account, provision a managed Kafka instance, and connect to it using service account credentials via SSL.

By default, OpenShift Streams for Apache Kafka retains data on a topic for 168 hours (7 days). But retention can be changed, so let’s change it to 10 seconds:

  1. Go to console.redhat.com and log in with your Red Hat account.
  2. On the console.redhat.com landing page, select Application Services from the menu on the left.
  3. On the Kafka Instances page, click the name of the Kafka instance you created earlier.
  4. Select the Topics tab and click on the song topic. In the topic properties tab, click the Edit properties button.
  5. Change the retention time to 10,000 milliseconds or 10 seconds (Figure 19). Leave the other properties as they are.
Edit the properties of your topic from the topics tab of your Kafka instance.
Figure 21: You can edit the properties of your topic from the topics tab of your Kafka instance.
  1. Scroll down to the bottom of the page and click Save.

Now start a consumer in a terminal using kcat:

$ kcat -b $BOOTSTRAP_SERVER \

-X security.protocol=SASL_SSL \

-X sasl.mechanisms=PLAIN \

-X sasl.username="$CLIENT_ID" \

-X sasl.password="$CLIENT_SECRET" -t songs -C -K:

Produce messages in another terminal using kcat:

$  kcat -b $BOOTSTRAP_SERVER \

-X security.protocol=SASL_SSL \

-X sasl.mechanisms=PLAIN \

-X sasl.username="$CLIENT_ID" \

-X sasl.password="$CLIENT_SECRET" -t songs -P -K: -l initial_songs

Verify that the produced messages were printed in the consumer terminal. Stop the consumer using Ctrl+C and wait for at least 10 seconds. Restart the kcat consumer, and note that it does not print any messages this time. The offset will still be stored as a non-zero value. The reason is that the messages are expired and deleted from the topic, but the offset still has the default retention period. This retention period is 1 week, so although there is no data, the current offset for a consumer is 4.

Reset the retention time to the default by repeating the procedure just described. Set the retention time back to 7 days.

Previous resource
Producing and consuming messages