Changing topic retention

By default, OpenShift Streams for Apache Kafka retains data on a topic for 168 hours (7 days). But retention can be changed, so let’s change it to 10 seconds:

  1. Go to console.redhat.com and log in with your Red Hat account.
  2. On the console.redhat.com landing page, select Application Services from the menu on the left.
  3. On the Kafka Instances page, click the name of the Kafka instance you created earlier.
  4. Select the Topics tab and click on the song topic. In the topic properties tab, click the Edit properties button.
  5. Change the retention time to 10,000 milliseconds or 10 seconds (Figure 19). Leave the other properties as they are.

 

Figure 19: You can change defaults for your topic, such as the retention time, on its dashboard page.

 

  1. Scroll down to the bottom of the page and click Save.

 

Now start a consumer in a terminal using kcat:

$ kcat -b $BOOTSTRAP_SERVER \

-X security.protocol=SASL_SSL \

-X sasl.mechanisms=PLAIN \

-X sasl.username="$CLIENT_ID" \

-X sasl.password="$CLIENT_SECRET" -t songs -C -K:

 

Produce messages in another terminal using kcat:

$  kcat -b $BOOTSTRAP_SERVER \

-X security.protocol=SASL_SSL \

-X sasl.mechanisms=PLAIN \

-X sasl.username="$CLIENT_ID" \

-X sasl.password="$CLIENT_SECRET" -t songs -P -K: -l initial_songs

 

Verify that the produced messages were printed in the consumer terminal. Stop the consumer using Ctrl+C and wait for at least 10 seconds. Restart the kcat consumer, and note that it does not print any messages this time. The offset will still be stored as a non-zero value. The reason is that the messages are expired and deleted from the topic, but the offset still has the default retention period. This retention period is 1 week, so although there is no data, the current offset for a consumer is 4.

Reset the retention time to the default by repeating the procedure just described. Set the retention time back to 7 days.