The Apache Kafka project includes a Streams Domain-Specific Language (DSL) built on top of the lower-level Stream Processor API. This DSL provides developers with simple abstractions for performing data processing operations. However, how one builds a stream processing pipeline in a containerized environment with Kafka isn’t clear. This second article in a two-part series uses the basics from the previous article to build an example application using Red Hat AMQ Streams.
Continue reading “Building Apache Kafka Streams applications using Red Hat AMQ Streams: Part 2”
Scalability is often a key issue for many growing organizations. That’s why many organizations use Apache Kafka, a popular messaging and streaming platform. It is horizontally scalable, cloud-native, and versatile. It can serve as a traditional publish-and-subscribe messaging system, as a streaming platform, or as a distributed state store. Companies around the world use Apache Kafka to build real-time streaming applications, streaming data pipelines, and event-driven architectures.
Continue reading Intro to Apache Kafka and Kafka Streams for Event-Driven Microservices on DevNation Live
We have pretty exciting news this week as Red Hat is announcing the General Availability of their Apache Kafka Kubernetes operator. Red Hat AMQ Streams delivers the mechanisms for managing Apache Kafka on top of OpenShift, our enterprise distribution for Kubernetes.
Everything started last May 2018 when David Ingham (@dingha) unveiled the Developer Preview as new addition to the Red Hat AMQ offering. Red Hat AMQ Streams focuses on running Apache Kafka on OpenShift. In the microservices world, where several components need to rely on a high throughput communication mechanism, Apache Kafka has made a name for itself for being a leading real-time, distributed messaging platform for building data pipelines and streaming applications.
Continue reading “Welcome Apache Kafka to the Kubernetes Era!”
There is a major push in the United Kingdom to replace aging mechanical electricity meters with connected smart meters. New meters allow consumers to more closely monitor their energy usage and associated cost, and they enable the suppliers to automate the billing process because the meters automatically report fine-grained energy use.
This post describes an architecture for processing a stream of meter readings using Strimzi, which offers support for running Apache Kafka in a container environment (Red Hat OpenShift). The data has been made available through a UK research project that collected data from energy producers, distributors, and consumers from 2011 to 2014. The TC1a dataset used here contains data from 8,000 domestic customers on half-hour intervals in the following form:
Continue reading “Smart-Meter Data Processing Using Apache Kafka on OpenShift”
Using Apache Kafka in modern event-driven applications is pretty popular. For a better cloud-native experience with Apache Kafka, it’s highly recommended to check out Red Hat AMQ Streams, which offers an easy installation and management of an Apache Kafka cluster on Red Hat OpenShift.
This article shows how the Kafka-CDI library can handle difficult setup tasks and make creating Kafka-powered event-driven applications for MicroProfile and Jakarta EE very easy.
Continue reading “Introducing the Kafka-CDI Library”