Kafka Monthly Digest: June 2022

July 14, 2022
Mickael Maison
Related topics:
KafkaKubernetes
Related products:
Red Hat OpenShift Container Platform

This 53rd edition of the Kafka Monthly Digest covers what happened in the Apache Kafka community in June 2022.

For last month’s digest, see Kafka Monthly Digest: May 2022.

Releases

There is currently one release in progress: 3.3.0.

3.3.0

The release process for 3.3.0 continued. KIP freeze happened on June 15, and feature freeze happened on July 6. You can find the release plan in the wiki.

Kafka Improvement Proposals

Last month, the community submitted 4 KIPs (KIP-847 to KIP-850). I'll highlight a few of them:

  • KIP-847: Add ProducerCount metrics: This KIP proposes adding new metrics to track how many idempotent and transactional producers are connected to each broker. For each idempotent or transactional producer connected, brokers keep in memory some metadata about it. The goal is to improve observability and help diagnose whether there may be too many producers connected.

  • KIP-848: The Next Generation of the Consumer Rebalance Protocol: The Kafka group membership protocol allows distributing resources amongst client instances. It is used by consumers when they are in groups, but also by Connect and Streams. While this protocol is very flexible, over the years a few pain points have been identified. This KIP proposes significantly updating this protocol to make rebalances lightweight and simplify troubleshooting by moving most of the logic to the server side.

  • KIP-849: Expose logdirs total and usable space via kafka-log-dirs.sh: In Kafka 3.2, KIP-827 exposed the size of usable and total space of log directories via the Admin API. This KIP aims at updating kafka-log-dirs.sh to include these values in its output so administrators can easily access them.

Community releases

  • Librdkafka 1.9.0. Librdkafka is a Kafka client in C/C++. This new release adds support for OAuthbearer OIDC and admin APIs for managing ACLs. As always, it also brings in many improvements and bug fixes.

  • kafkajs 2.1: Kafkajs is a pure JavaScript Kafka client for Node.js. Users can now pause and resume consuming partitions directly from from the eachBatch/eachMessage handlers. This release also contains some important bug fixes.

Blogs

I selected some interesting blog posts and articles that were published last month:

To learn more about Kafka, visit Red Hat Developer's Apache Kafka topic page.

Last updated: September 20, 2023