Kafka Monthly Digest: April 2026

Releases

4.3.0

4.2.1

Kafka Improvement Proposals

• KIP-1312: Support unregistering controllers: In KRaft mode, brokers and controllers have a persisting registration in the metadata log. Brokers have to be explicitly unregistered when decommissioned but at the moment there isn't a similar process for controllers. Even if a controller is out of the current voter set, if its registration has an old metadata version, it could prevent upgrading the cluster metadata version. To resolve this issue, this KIP introduces a process to unregister controllers similar to the broker process.
   

• KIP-1313: Client instance ID in all request headers: Kafka clients are identified by their client.id configuration. This value is included in the header of all requests. This approach has a few issues. Kafka connections are expected to be long lived, so including an arbitrary string in each request can be wasteful. Also, in practice, client ids are often reused so fail their role at identifying a unique client. This KIP proposes adding a unique client instance id to each client, which would be automatically generated at each startup. This new client instance id will be included in each request but the original client.id, will only be sent in the initial request and be cached on the broker side.
   

• KIP-1324: Support client configuration observability: This KIP proposes a mechanism for clients to send part of their configurations to brokers for validations. This follows up KIP-714 which enables clients to send their metrics to brokers, to increase client observability from the broker side. This would help administrators identify client configuration issues and troubleshoot misbehaving clients.
   

• KIP-1326: Include broker static configurations in BrokerRegistrationRequest: IncrementalAlterConfigs and AlterConfigs requests can be sent to the controller directly. However controllers don't currently track broker configurations, so they can't perform the same validations that brokers do. This KIP proposes adding broker configurations in their registration requests to improve controller validations of configuration updates. It would also enable enforcing cluster-wide configurations in order to avoid unexpected behavior when brokers have different configurations.
   

• KIP-1331: Streams Group Topology Description Plugin: This KIP also aims at increasing client observability from the broker side but with a focus on Streams applications. It proposes a mechanism for Streams applications to send their full topology to brokers (currently only parts of the topology needed for assignment are sent), so it's easy for cluster operators and Streams users to inspect them in case of issues.
   

• KIP-1332: Dynamic memory allocation for the Kafka producer: This KIP aims at optimizing the memory usage of the producer. Today the producer always allocates the full batch size upfront regardless of how much data the batch will hold before it is sent. For producers sending data to many partitions, this can create memory pressure as the producer effectively needs (batch.size x number of partitions used) of memory available. The proposal is to allocate memory as needed when records are added to batches. This behavior would be enabled via a new configuration buffer.memory.allocation.strategy accepting static the current behavior or dynamic the new proposed behavior.
   

Community Releases

• strimzi-kafka-operator 1.0.0: Strimzi is a Kubernetes Operator for running Kafka. Strimzi 1.0 now only supports the v1 CRD APIs and removed all the older versions. It also brings a few new features including support for Kafka 4.1.2, HTTPS support to the bridge and in-place pod resizing for brokers, controllers and Connect nodes. 
   

• Debezium 3.5: Debezium is a Change Data Capture platform. This release contains dozens of new features and all connectors have received improvements. 
   

• kroxylicious 0.20.0: Kroxylicious is an open source pluggable framework for writing network proxies that understand the Apache Kafka protocol. This release introduces a new connection expiration filter and includes updates to the entity isolation and record validation filters. The Kubernetes operator also received improvements with support for OpenShift routes and server-side applies.
   

Blogs

• What is Apache Kafka and how does it work?
• Can Kafka Queues Make Consumers Faster?
• Stateful Kafka Streams Processors: Reducing Rebalancing - Lessons Learned