Kafka Monthly Digest: January 2022

February 10, 2022
Mickael Maison
Related topics:
Event-drivenJavaKafkaKubernetes
Related products:
Red Hat OpenShift

This 48th edition of the Kafka Monthly Digest covers what happened in the Apache Kafka community in January 2022. The release of Apache Kafka 3.1.0 is the big news, including updates to Kafka Connect and Kafka Streams. I'll also discuss new KIPs and open source releases in January 2022.

This is the fourth anniversary of this series! If you are interested in the history behind the Kafka Monthly Digest, last year I explained on Twitter how I got started.

For last month's digest, see Kafka Monthly Digest: December 2021.

Kafka 3.1.0 release

David Jacot released Kafka 3.1.0 on January 24 and published an announcement on the Apache blog. As always, you can find the complete list of changes in the release notes or the release plan on the Kafka wiki.

Kafka 3.1 adds support for Java 17. Note that running Kafka in KRaft mode (without ZooKeeper) is still not ready for production.

This new minor release brings a few interesting features, which I'll highlight in the next sections.

Kafka brokers and clients

Updates to the Kafka broker and clients include the following:

  • A production-ready OAuth implementation with OpenID Connect (OIDC) support (KIP-768).
  • Metrics improvements include new metrics for active and fenced brokers (KIP-748) and consistent names for latency metrics (KIP-773).
  • Topic IDs that were introduced in Kafka 2.8 are now used in FetchRequests (KIP-516).

Kafka Connect

The ReplicationPolicy interface allows you to customize the names of MirrorMaker internal topics (KIP-690).

Kafka Streams

Updates to Kafka Streams include the following:

Kafka Improvement Proposals

Last month, the community submitted seven KIPs (KIP-812 to KIP-818). I'll highlight just a few of them:

  • KIP-813: Shareable state stores: This KIP proposes a mechanism for reusing existing Kafka Streams state stores in other Streams applications. In scenarios where multiple applications perform similar processing steps, this could be useful to avoid doing the same processing twice.

  • KIP-814: Static membership protocol should let the leader skip assignment: This KIP's goal is to improve support for static consumer group membership (introduced in Kafka 2.4). Today, in some cases, consumer groups with static membership can end up without a leader and miss metadata updates.

  • KIP-817: Fix inconsistency in dynamic application log levels: There are two methods for changing Kafka's log at runtime: Java Management Extensions (JMX) and the Admin API. However, there are some inconsistencies in the levels they support. For example, it's not possible to set a logger to OFF via the Admin API. This KIP aims at clearing up those inconsistencies.

Community releases for Apache Kafka

This section covers a few notable open source community project releases:

  • A Great Day Out With... Apache Kafka: This is a curated list of resources, projects, tools, and products for people getting started with Kafka. The most recent update added the latest key Kafka resources people should be aware of when working with the platform.

  • Sarama 1.31: Sarama is a pure Go language Kafka client. This new release adds support for IncrementalAlterConfigs in the Admin client, improves request pipelining for the producer, and as usual fixes a few bugs.

Kafka blogs and articles

Here are a few of the most noteworthy Kafka-related blogs and articles published in January 2022:

To learn more about Kafka, visit Red Hat Developer's Apache Kafka topic page.

Last updated: September 20, 2023