Streams for Apache Kafka: Real-time messaging and event streaming
Streams for Apache Kafka is a massively scalable, distributed, and high-performance data streaming platform based on the Apache Kafka and Strimzi open source projects. It offers a distributed backbone that allows microservices and other applications to share data with high throughput and low latency. As more applications move to Kubernetes and Red Hat OpenShift, it is increasingly important to be able to run the communication infrastructure on the same platform. Red Hat OpenShift, as a highly scalable platform, is a natural fit for messaging technologies such as Apache Kafka. By leveraging Strimzi, streams for Apache Kafka makes running and managing Apache Kafka “OpenShift native” through the use of powerful operators that simplify the deployment, configuration, management, and use of Apache Kafka on Red Hat OpenShift.
Capabilities and features
Based on the Apache Kafka project, streams for Apache Kafka offers a distributed backbone that allows microservices and other applications to share data with both extremely high throughput and extremely low latency.
Real-time streaming
Offers a massively scalable and distributed data streaming platform that enables microservices and other applications to share data with high throughput and low latency.
Messaging
Provides message order guarantees along with message rewind/replay from data storage to reconstruct an application state. Allows message compaction to remove old records when using a key value log.
Data retention and replication
Enables long-term data retention that efficiently stores data for immediate access in a manner limited only by disk space. Provides replication of data to control fault tolerance.
Partitioning
Guarantees the processing of high volumes of messages, where distributing messages in partitions enables different consumers to process the volume of events.
Container images and operators
Streams for Apache Kafka provides container images and operators for running Kafka on OpenShift. Streams for Apache Kafka operators are purpose-built with operational knowledge to effectively manage Kafka on OpenShift.
Use cases
Enable digital experiences to deliver faster and better customer experiences
- Replace batch data with real-time events.
- Enable real-time applications to send/receive large volumes of data from different sources.
- Allow organizations to horizontally scale when necessary by deploying more Kafka clusters.
- Respond fast to real-world events and requests by collecting and analyzing time-bound data.
- Free developers from coding data integration mechanisms and focus on stream processing.
Related content:
Capture, communicate, and process events for modern, distributed application architectures
- Identify and react immediately to critical events.
- Share data instantaneously between teams within an organization and external strategic partners.
- Build event-driven applications to support data streaming, events analysis, and decision making.
- Simplify data integration by decoupling the data from your systems.
- Modernize existing systems and services.
Related content:
Deliver scalable, reliable, and secure Kafka-centric microservice architectures
- Publish events to Kafka brokers and decouple the data from the event-consuming services.
- Meet event volumes by independently scaling up and down your microservices.
- Avoid hard-coding integrations and connections between microservices applications.
Related content:
Streams for Apache Kafka is open source
Streams for Apache Kafka combines many open source community projects in building a highly scalable, distributed, and high-performance data streaming platform.
Apache Kafka is a distributed data streaming platform that is a popular event processing choice. It can handle publishing, subscribing to, storing, and processing event streams in real-time. Apache Kafka supports a range of use cases where high throughput, scalability, and low latency are vital.
Strimzi is an open source project licensed under Apache License 2.0 that is part of the Cloud Native Computing Foundation (CNCF). Strimzi’s main focus is running Apache Kafka on Kubernetes while providing container images for Apache Kafka itself, Zookeeper, and other components that are part of the Strimzi ecosystem.
Debezium is a distributed platform that converts information from your existing databases into event streams, enabling applications to detect, and immediately respond to row-level changes in the databases. Debezium is built on top of Apache Kafka and provides a set of Kafka Connect compatible connectors.
Kroxylicious is an Apache Kafka protocol-aware proxy. It can be used to layer uniform behaviors onto a Kafka-based system in areas such as data governance, security, policy enforcement, and audit without needing to change either the applications or the Kafka cluster.
