This 66th edition of the Kafka Monthly Digest covers what happened in the Apache Kafka community in July 2023.
For last month’s digest, see Kafka Monthly Digest: June 2023.
Releases
There is one new release, 3.5.1, and 3.6.0 is still in progress.
3.5.1
On July 21, Divij Vaidya published Kafka 3.5.1. The main change in this bugfix release is the upgrade of Snappy to 1.1.10.1 in order to address CVE-2023-34455. It also fixes 10 Jiras including a regression that was introduced in 3.3.0. For all the details you can check the release notes or the announcement on the Kafka blog.
3.6.0
The release process for 3.6.0 continued and KIP freeze happened on July 26. The next milestone is feature freeze and it is currently planned for August 16. You can find all the details in the release plan.
Kafka Improvement Proposals
Last month, the community submitted 14 KIPs (KIP-948 to KIP-962, KIP-961 was skipped). I'll highlight a few of them:
- KIP-951: Leader discovery optimisations for the client: When the leadership of a partition changes, clients get an error response from brokers and have to refresh their metadata to discover the new leader. This process can take hundreds of milliseconds and negatively impacts the latency. This KIP proposes including the new leader information in the error response from the broker to speed up the recovery process.
- KIP-960: Support interactive queries (IQv2) for versioned state stores: Interactive queries allow querying the state of a Kafka Streams application from outside. Versioned state stores were added in Kafka 3.5, however they don't support querying via interactive queries. This KIP's goal is to add support for interactive queries to versioned state stores.
As the initial support for tiered storage is nearing completion, a few KIPs aim at making it production ready:
- KIP-950: Tiered Storage Disablement: As defined in KIP-405, enabling tiered storage is a one way decision. This KIP's goal is to allow users to disable tiered storage. It would help the adoption of tiered storage and make it easier for users to test and adopt this feature.
- KIP-956 Tiered Storage Quotas: This KIP aims at adding configurations to control the rate for uploading and reading records from tiered storage. Both of these operations have an impact on the performance of the cluster so it can be important for users to set limits for their environments.
- KIP-958: Tiered Storage Metrics: This KIP proposes adding a few metrics about the effectiveness of the index cache for improving the observability of tiered storage.
Community Releases
- Sarama 1.40: Sarama is a pure Golang Kafka client. The ownership of the project has transitioned from Shopify to IBM. This new version adds support for Kerberos authentication with a credentials cache, improved memory consumption while decompressing records and includes a number of fixes around retries in the Admin client.
- strimzi-kafka-operator 0.36: Strimzi is a Kubernetes Operator for running Kafka. It adds support for Kafka 3.4.1 and 3.5.0, support for SCRAM-SHA in KRaft mode, and adds a new topic operator that works with KRaft.
- Librdkafka 2.2: Librdkafka is a Kafka client in C/C++. It adds support for rack-aware partition assignment for the consumer, support for periodic re-authentication, support for IncrementalAlterConfigs and fixes a few bugs.
Blogs
I selected some interesting blog articles that were published last month:
To learn more about Kafka, visit Red Hat Developer's Apache Kafka topic page.
