Streams for Apache Kafka

A lightweight, high-performance, robust, event streaming platform.

hero

Streams for Apache Kafka: Real-time messaging and event streaming

Streams for Apache Kafka is a massively scalable, distributed, and high-performance data streaming platform based on the Apache Kafka® and Strimzi open source projects. It offers a distributed backbone that allows microservices and other applications to share data with high throughput and low latency. As more applications move to Kubernetes and Red Hat OpenShift, it is increasingly important to be able to run the communication infrastructure on the same platform. Red Hat OpenShift, as a highly scalable platform, is a natural fit for messaging technologies such as Apache Kafka. By leveraging Strimzi, streams for Apache Kafka makes running and managing Apache Kafka “OpenShift native” through the use of powerful operators that simplify the deployment, configuration, management, and use of Apache Kafka on Red Hat OpenShift. 

intro streams

Capabilities and features

Based on the Apache Kafka project, streams for Apache Kafka offers a distributed backbone that allows microservices and other applications to share data with both extremely high throughput and extremely low latency.

Real-time streaming

Real-time streaming

Offers a massively scalable and distributed data streaming platform that enables microservices and other applications to share data with high throughput and low latency.

Long-term data retention

Messaging

Provides message order guarantees along with message rewind/replay from data storage to reconstruct an application state. Allows message compaction to remove old records when using a key value log.

Data retention and replication

Data retention and replication

Enables long-term data retention that efficiently stores data for immediate access in a manner limited only by disk space. Provides replication of data to control fault tolerance.

Partitioning

Partitioning

Guarantees the processing of high volumes of messages, where distributing messages in partitions enables different consumers to process the volume of events.

Partition messages for scalability

Container images and operators

Streams for Apache Kafka provides container images and operators for running Kafka on OpenShift. Streams for Apache Kafka operators are purpose-built with operational knowledge to effectively manage Kafka on OpenShift.

 

Use cases

Enable digital experiences to deliver faster and better customer experiences

  • Replace batch data with real-time events.
  • Enable real-time applications to send/receive large volumes of data from different sources.
  • Allow organizations to horizontally scale when necessary by deploying more Kafka clusters.
  • Respond fast to real-world events and requests by collecting and analyzing time-bound data.
  • Free developers from coding data integration mechanisms and focus on stream processing.

Related content:

use case

Capture, communicate, and process events for modern, distributed application architectures

  • Identify and react immediately to critical events.
  • Share data instantaneously between teams within an organization and external strategic partners.
  • Build event-driven applications to support data streaming, events analysis, and decision making.
  • Simplify data integration by decoupling the data from your systems. 
  • Modernize existing systems and services.

Related content:

Create an event-driven architecture

Deliver scalable, reliable, and secure Kafka-centric microservice architectures

  • Publish events to Kafka brokers and decouple the data from the event-consuming services.
  • Meet event volumes by independently scaling up and down your microservices.
  • Avoid hard-coding integrations and connections between microservices applications.

Related content:

use case 3

Streams for Apache Kafka is open source

Streams for Apache Kafka combines many open source community projects  in building a highly scalable, distributed, and high-performance data streaming platform.

apache kafka

Apache Kafka is a distributed data streaming platform that is a popular event processing choice. It can handle publishing, subscribing to, storing, and processing event streams in real-time. Apache Kafka supports a range of use cases where high throughput, scalability, and low latency are vital.

strimzi

Strimzi is an open source project licensed under Apache License 2.0 that is part of the Cloud Native Computing Foundation (CNCF). Strimzi’s main focus is running Apache Kafka on Kubernetes while providing container images for Apache Kafka itself, Zookeeper, and other components that are part of the Strimzi ecosystem.

debezium"

Debezium is a distributed platform that converts information from your existing databases into event streams, enabling applications to detect, and immediately respond to row-level changes in the databases. Debezium is built on top of Apache Kafka and provides a set of Kafka Connect compatible connectors.

kroxylicious

Kroxylicious is an Apache Kafka protocol-aware proxy. It can be used to layer uniform behaviors onto a Kafka-based system in areas such as data governance, security, policy enforcement, and audit without needing to change either the applications or the Kafka cluster.

Apache Kafka, Kafka, and the Kafka logo are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.