In the previous articles in this series, we first covered the basics of Red Hat AMQ Streams on OpenShift and then showed how to set up Kafka Connect, a Kafka Bridge, and Kafka Mirror Maker. Here are a few key points to keep in mind before we proceed:
- AMQ Streams is based on Apache Kafka.
- AMQ Streams for the Red Hat OpenShift Container Platform is based on the Strimzi project.
- AMQ Streams on containers has multiple components, such as the Cluster Operator, Entity Operator, Mirror Maker, Kafka connect, and Kafka Bridge.
Now that we have everything set up (or so we think), let’s look at monitoring and alerting for our new environment.
Continue reading “Understanding Red Hat AMQ Streams components for OpenShift and Kubernetes: Part 3”
Red Hat AMQ Streams is an enterprise-grade Apache Kafka (event streaming) solution, which enables systems to exchange data at high throughput and low latency. AMQ Streams is available as part of the Red Hat AMQ offering in two different flavors: one on the Red Hat Enterprise Linux platform and another on the OpenShift Container Platform. In this three-part article series, we will cover AMQ Streams on the OpenShift Container Platform.
To get the most out of these articles, it will help to be familiar with messaging concepts, Red Hat OpenShift, and Kubernetes.
Continue reading “Understanding Red Hat AMQ Streams components for OpenShift and Kubernetes: Part 1”
Red Hat continues to increase the features available for users looking to implement a 100% open source, event-driven architecture (EDA) through running Apache Kafka on Red Hat OpenShift and Red Hat Enterprise Linux. The Red Hat Integration Q4 release provides new features and capabilities, including ones aimed at simplifying usage and deployment of the AMQ streams distribution of Apache Kafka.
Continue reading “Red Hat simplifies transition to open source Kafka with new service registry and HTTP bridge”
As part of Red Hat’s AMQ offerings, Red Hat offers a Kafka-based event streaming solution both for traditional deployment and microservices-based deployment branded as Red Hat AMQ Streams. The Red Hat OpenShift AMQ Streams deployment option is based on Strimzi, an open source tool that makes Kafka deployment as a container on a Kubernetes platform easy because most of the deployment prerequisites are automated with the OpenShift Operator Framework.
In this article, we look at how to deploy Apache Kafka on Red Hat OpenShift 4, using reasonable sample microservice applications to showcase the endless possibility of innovation brought by OpenShift and Kafka.
Continue reading “Event-based microservices with Red Hat AMQ Streams”
We are thrilled to announce an updated release of the data streaming component of our messaging suite, Red Hat AMQ streams 1.2, which is part of Red Hat integration.
Red Hat AMQ streams, based on the Apache Kafka project, offers a distributed backbone that allows microservices and other applications to share data with extremely high throughput and extremely low latency. AMQ streams makes running and managing Apache Kafka a Kubernetes-native experience, by additionally delivering Red Hat OpenShift Operators, a simplified and automated way to deploy, manage, upgrade and configure a Kafka ecosystem installation on Kubernetes.
Continue reading “Announcing Red Hat AMQ streams 1.2 with Apache Kafka 2.2 support”
The Apache Kafka project includes a Streams Domain-Specific Language (DSL) built on top of the lower-level Stream Processor API. This DSL provides developers with simple abstractions for performing data processing operations. However, how one builds a stream processing pipeline in a containerized environment with Kafka isn’t clear. This second article in a two-part series uses the basics from the previous article to build an example application using Red Hat AMQ Streams.
Continue reading “Building Apache Kafka Streams applications using Red Hat AMQ Streams: Part 2”
The Apache Kafka project includes a Streams Domain-Specific Language (DSL) built on top of the lower-level Stream Processor API. This DSL provides developers with simple abstractions for performing data processing operations. However, how to build a stream processing pipeline in a containerized environment with Kafka isn’t clear. This two-part article series describes the steps required to build your own Apache Kafka Streams application using Red Hat AMQ Streams.
Continue reading “Building Apache Kafka Streams applications using Red Hat AMQ Streams: Part 1”
Scalability is often a key issue for many growing organizations. That’s why many organizations use Apache Kafka, a popular messaging and streaming platform. It is horizontally scalable, cloud-native, and versatile. It can serve as a traditional publish-and-subscribe messaging system, as a streaming platform, or as a distributed state store. Companies around the world use Apache Kafka to build real-time streaming applications, streaming data pipelines, and event-driven architectures.
Continue reading Intro to Apache Kafka and Kafka Streams for Event-Driven Microservices on DevNation Live
On October 25th Red Hat announced the general availability of their AMQ Streams Kubernetes Operator for Apache Kafka. Red Hat AMQ Streams focuses on running Apache Kafka on Openshift providing a massively-scalable, distributed, and high performance data streaming platform. AMQ Streams, based on the Apache Kafka and Strimzi projects, offers a distributed backbone that allows microservices and other applications to share data with extremely high throughput. This backbone enables:
- Publish and subscribe: Many to many dissemination in a fault tolerant, durable manner.
- Replayable events: Serves as a repository for microservices to build in-memory copies of source data, up to any point in time.
- Long-term data retention: Efficiently stores data for immediate access in a manner limited only by disk space.
- Partition messages for more horizontal scalability: Allows for organizing messages to maximum concurrent access.
One of the most requested items from developers and architects is how to get started with a simple deployment option for testing purposes. In this guide we will use Red Hat Container Development Kit, based on minishift, to start an Apache Kafka cluster on Kubernetes.
Continue reading “How to run Kafka on Openshift, the enterprise Kubernetes, with AMQ Streams”