Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
      A secure, stable, and supported operating system.
    • Red Hat Openshift
      Openshift icon
      A cloud-native application platform.
    • Red Hat Ansible Automation Platform
      Ansible icon
      Simple IT automation anyone can use.
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
      • Red Hat Enterprise Linux for SAP
      • Microsoft SQL Server on Red Hat Enterprise Linux
      • CentOS Linux
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of Quarkus
      • Red Hat build of OpenJDK
      • View all
    • Kubernetes

      • Red Hat Openshift
      • Red Hat OpenShift Service on AWS (ROSA)
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Dedicated
      • Red Hat OpenShift Data Science
    • Integration & App Connectivity

      • Red Hat Service Interconnect
      • AMQ Broker
      • View all
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Automation Platform via AWS Marketplace
      • Red Hat Ansible Lightspeed with IBM watsonx Code Assistant
      • Ansible automation for applications and services
      • View all
    • Developer tools

      • Red Hat Developer Hub
      • Red Hat Trusted Software Supply Chain
      • Red Hat OpenShift Dev Spaces
      • View all
    • Openshift Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • App Dev Platform
      App Dev Platform icon showing coding brackets
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • C, C#, C++
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
    • Developer Productivity

      • Developer Tools
      • Visual Studio Code (VS Code)
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    Initeractive Lessons and Learning Paths

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform
    • Red Hat OpenShift AI
    • Java

    Developer Sandbox Activities

    • Get Started
    • Try Hands-On Activities

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Tutorials

    • Kubernetes
    • Application Development

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    API Catalog

    • API Catalog
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Events

    Developer Events

    • DevNation
      Join developers across the globe for live and virtual events led by Red Hat technology experts.
    • Explore All Events
    • Tech Talks

      • Getting Gitops
      • GitHub Makeover
      • OpenTelemetry on Kubernetes
      • View All Tech Talks
    • Deep Dives

      • Quinoa: A modern Quarkus UI with no hassles
      • Event-driven autoscaling through KEDA and Knative Integration
      • View All Deep Dives
    • Red Hat Summit

      Red Hat Summit
      Explore the Red Hat Summit 2023 virtual content hub and uncover insights that can fuel your future. Get on-demand access to the keynotes, sessions, and technical presentations from Red Hat Summit 2023 that are most relevant to your career.
    • View the content hub
  • Developer Sandbox

    Developer Sandbox (free)

    • Try hands-on activities in the Developer Sandbox
      Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore the Developer Sandbox IDE

    Featured Developer Sandbox activities

    • Deploy a Java application on Kubernetes in minutes
    • Learn Kubernetes using the Developer Sandbox
    • Deploy full-stack JavaScript apps to the Developer Sandbox

    Ready to start developing apps?

    • Explore the free Developer Sandbox
  • Blog

Blog

Kafka Monthly Digest: September 2023

October 2, 2023
Profile
Mickael Maison
Principal Software Engineer
Kafka Monthly Digest

    This 68th edition of the Kafka Monthly Digest covers what happened in the Apache Kafka community in September 2023.

    For last month’s digest, see Kafka Monthly Digest: August 2023.

    Releases

    There is one release in progress: 3.6.0.

    3.6.0

    The release process for 3.6.0 continued. Satish Duggana published the first release candidate on September 17. During testing, an issue with Kafka Connect and an issue with the ZooKeeper to KRaft migration system tests were found. Satish published RC1 on September 20, unfortunately snappy-java, one of the dependencies of Kafka, announced a high severity CVE on September 24. Satish published RC2 on September 29 and the vote is still on-going. You can find all the details in the release plan.

    Kafka Improvement Proposals

    Last month, the community submitted 9 KIPs (KIP-975 to KIP-983). I'll highlight a few of them:

    • KIP-975: Docker Image for Apache Kafka: This KIP proposes publishing an official Apache Kafka Docker image. As Kafka is often deployed and run in containerized environments this would help users get started with Kafka.
    • KIP-976: Cluster-wide dynamic log adjustment for Kafka Connect: Connect exposes an endpoint to adjust log levels at runtime. However, it only affects the Connect worker you send the request to, so if you want to adjust log levels of a Connect cluster, you need to send a request per worker. This KIP proposes updating this mechanism to allow updating log levels cluster-wide with a single request.
    • KIP-979: Allow independently stop KRaft controllers or brokers: With KRaft, the kafka-server-start.sh script is used to start both regular brokers and controllers. The kafka-server-stop.sh script has not been updated to handle that, so when running a broker and controller on the same host, the script currently stops everything. This KIP proposes updating the script to select what to stop.
    • KIP-981: Manage Connect topics with custom implementation of Admin: In Kafka 3.4, KIP-787 added the ability to use a custom Admin interface with MirrorMaker. This KIP extends this concept to Kafka Connect. Having a custom Admin interface is useful in environments using a centralized or federated system to manage topics, for example to control capacity or billing.

    Community Releases

    • strimzi-kafka-operator 0.37: Strimzi is a Kubernetes Operator for running Kafka. This version requires at least Kubernetes 1.21 to run. Connect and MirrorMaker now use stable identities by default and this new version adds support for the ppc64le platform. There are also important changes around Cruise Control, connector restart behavior and support for OpenTracing has been removed (only OpenTelemetry-based tracing is supported now).

    Blogs

    I selected some interesting blog articles that were published last month:

    • Scaling Kafka to Support PayPal’s Data Growth
    • Apache Kafka Message Compression
    • Online machine learning with the data streams from the database
    • The Stairway to Apache Kafka® Tiered Storage

    To learn more about Kafka, visit Red Hat Developer's Apache Kafka topic page.

    Featured Contributor Content

    • Join the Red Hat team at the Open Source in Finance Forum 2023

    • ISO C++ Jun 23 meeting trip report (core language)

    • How to install Ansible Automation Platform 2.4 on RHEL 9.1

    • I installed Podman and Podman Desktop on my Windows PC

    • Observability for Node.js applications in OpenShift

    • Getting started with Topology View in Cryostat 2.3

    Recent Articles

    • Kafka Bridge load balancing and failover using Tetrate Service Bridge

    • Getting started with MongoDB and Quarkus

    • How to deploy AngularJS apps to OpenShift

    • Getting started with tiered storage in Apache Kafka

    • Red Hat build of Keycloak provides FIPS-140-2 support

    Info alert: Please note the content in this blog post has not been thoroughly reviewed by the Red Hat Developer editorial team. Any opinions expressed in this post are the author’s own and do not necessarily reflect the policies or positions of Red Hat.

    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform
    • See all products
    • See all technologies

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog
    • Operators Marketplace

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Diversity, equity, and inclusion
    • Cool Stuff Store
    • Red Hat Summit

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility