Skip to main content
AI
  • Get started with AI

    • Red Hat AI

      Accelerate the development and deployment of enterprise AI solutions.

    • AI learning hub

      Explore learning materials and tools, organized by task.

    • AI interactive demos

      Click through scenarios with Red Hat AI, including training LLMs and more.

    • AI/ML learning paths

      Expand your Red Hat OpenShift AI knowledge using these learning resources.

    • AI quickstarts

      Focused AI use cases designed for fast deployment on Red Hat AI platforms.

    • No-cost AI training

      Foundational Red Hat AI training.

  • Featured resources

    • Red Hat OpenShift AI
    • Open source AI for developers
    • AI product application development
    • Open source-powered AI/ML for hybrid cloud
    • AI and Node.js cheat sheet

Red Hat AI Factory with NVIDIA

Red Hat AI Factory with NVIDIA is a co-engineered, enterprise-grade AI solution for building, deploying, and managing AI at scale across hybrid cloud environments.

Learn
  • Self-guided

    • Documentation

      Find answers, get step-by-step guidance, and learn how to use Red Hat products.

    • Learning paths

      Explore curated walkthroughs for common development tasks.

    • Guided learning

      Receive custom learning paths powered by our AI assistant.

    • See all learning
  • Hands-on

    • Developer Sandbox

      Spin up Red Hat's products and technologies without setup or configuration.

    • Interactive labs

      Learn by doing in these hands-on, browser-based experiences.

    • Interactive demos

      Click through product features in these guided tours.

  • Browse by topic

    • AI/ML
    • Automation
    • Java
    • Kubernetes
    • Linux
    • See all topics
  • Training & certifications

    • Courses and exams
    • Certifications
    • Skills assessments
    • Red Hat Academy
    • Learning subscription
    • Explore training
Build
  • Get started

    • Red Hat build of Podman Desktop

      A downloadable, local development hub to experiment with our products and builds.

    • Developer Sandbox

      Spin up Red Hat's products and technologies without setup or configuration.

  • Download products

    • Red Hat Enterprise Linux
    • Red Hat AI
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform
    • See all products
  • Featured

    • Red Hat build of OpenJDK
    • Red Hat JBoss Enterprise Application Platform
    • Red Hat OpenShift Dev Spaces
    • Red Hat Developer Toolset
  • References

    • E-books
    • Documentation
    • Cheat sheets
    • Architecture center
Community
  • Get involved

    • Events
    • Live AI events
    • Red Hat Summit
    • Red Hat Accelerators
    • Community discussions
  • Follow along

    • Articles & blogs
    • Developer newsletter
    • Videos
    • Github
  • Get help

    • Customer service
    • Customer support
    • Regional contacts
    • Find a partner

Join the Red Hat Developer program

Download Red Hat products and project builds, access support documentation, learning content, and more.

ConsoleDocsSupport Search
New For you

Recommended

We'll recommend resources you may like as you browse. Try these suggestions for now.

  • Product downloads
  • Cheat sheets
  • Red Hat Developer learning portal
  • Developer blog
  • E-books
  • Developer Sandbox
Log in

Get more with a Red Hat Developer account

  • Product downloads
  • Console access
  • Event registration
  • Training & trials
Explore more account benefitsLog in or register

A subscription may be required for some services.

Contact us

Welcome Apache Kafka to the Kubernetes Era!

<p>&nbsp;</p> <quillbot-extension-portal></quillbot-extension-portal>

October 25, 2018
Hugo Guerrero
Related topics:
ContainersEvent-drivenKubernetesMicroservices
Related products:
Streams for Apache Kafka

    We have pretty exciting news this week as Red Hat is announcing the General Availability of their Apache Kafka Kubernetes operator. Red Hat AMQ Streams delivers the mechanisms for managing Apache Kafka on top of OpenShift, our enterprise distribution for Kubernetes.

    Everything started last May 2018 when David Ingham (@dingha) unveiled the Developer Preview as new addition to the Red Hat AMQ offering. Red Hat AMQ Streams focuses on running Apache Kafka on OpenShift. In the microservices world, where several components need to rely on a high throughput communication mechanism, Apache Kafka has made a name for itself for being a leading real-time, distributed messaging platform for building data pipelines and streaming applications.

    A leader in the traditional infrastructure deployments, Apache Kafka was missing pieces to be a easily usable container-native citizen in the new Kubernetes era.  As a result, a team grouped in 2017 to create the upstream Strimzi project. This team worked to apply the new operator pattern to solve the gaps. With the development of new components deployed along the traditional Apache Kafka broker, these new Kubernetes operators are now able to manage cluster wide resources as well as entities as topics and authentication users.

    Even more, these Kubernetes operators are pretty simple to use and cover most of the more common configuration management of a cluster. After doing some Kubernetes Custom Resource Definition installations in OpenShift, any user is able to create an Apache Kafka cluster by just creating a new Kafka resource definition. Then, the cluster operator will take that definition and provision the required components to have a fully deployed cluster on top of your OpenShift infrastructure. The same applies for the creation of Topics and Users for that same Kafka cluster.

    Finally, the most interesting part, is the simple configuration to provision a cluster that's easily available from outside the OpenShift cluster via NodePorts, Load Balancers or Routes, the latter being  the easiest way to start working. Depending on your application requirements, you will be able to choose from using the simple secure over TLS approach or the traditional NodePort settings. As a result, the AMQ Streams operators embedded logic creates dynamic services, routes and certificates to access the Apache Kafka cluster from external clients. 

    Yes, it sounds like magic. That’s how good it is! Give Red Hat AMQ Streams a try by downloading Red Hat Container Development Kit for an OpenShift development environment and following our Getting Started Guide. Also, be aware of my HOW-TO guide for running Kafka on OpenShift, the enterprise Kubernetes!


    Here are some other articles on Apache Kafka and Red Hat AMQ Streams:

    • Announcing AMQ Streams: Apache Kafka on OpenShift
    • Smart-Meter Data Processing Using Apache Kafka on OpenShift
    • EventFlow: Event-driven microservices on OpenShift (Part 1)
    • Detecting credit card fraud with Red Hat Decision Manager 7
    • Introducing the Kafka-CDI Library
    Last updated: June 6, 2023

    Recent Posts

    • A day in the life of an NVIDIA H100 on Red Hat OpenShift 5

    • Migrate BuildConfigs to builds for Red Hat OpenShift with one plug-in install

    • Run Red Hat Enterprise Linux CoreOS on NVIDIA BlueField DPUs

    • Agent harness engineering: Separating model capabilities from enforcement

    • What's new for developers in Red Hat Enterprise Linux 10.3 and 9.9

    What’s up next?

     

    Red Hat Developers logo

    Platforms

    • Red Hat AI
    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform
    • See all products

    Build

    • Developer Sandbox
    • Developer tools
    • Interactive tutorials
    • API catalog

    Quicklinks

    • Learning resources
    • E-books
    • Cheat sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site status dashboard
    • Report a security problem

    Red Hat Developer

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit
    © 2026 Red Hat

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility
    Ask AI