Jakub Scholz

Jakub is a Principal Software Engineer in the Red Hat Messaging and IoT engineering team. He has a long-term experience in messaging and lately focuses mainly on Apache Kafka. He is one of the core maintainers of the Strimzi project, which delivers several operators and tools for running Apache Kafka on Kubernetes and OpenShift.

Recent Posts

Introduction to Strimzi: Apache Kafka on Kubernetes (KubeCon Europe 2020)

Introduction to Strimzi: Apache Kafka on Kubernetes (KubeCon Europe 2020)

Apache Kafka has emerged as the leading platform for building real-time data pipelines. Born as a messaging system, mainly for the publish/subscribe pattern, Kafka has established itself as a data-streaming platform for processing data in real-time. Today, Kafka is also heavily used for developing event-driven applications, enabling the services in your infrastructure to communicate with each other through events using Apache Kafka as the backbone. Meanwhile, cloud-native application development is gathering more traction thanks to Kubernetes.

Thanks to the abstraction layer provided by this platform, it’s easy to move your applications from running on bare metal to any cloud provider (AWS, Azure, GCP, IBM, and so on) enabling hybrid-cloud scenarios as well. But how do you move your Apache Kafka workloads to the cloud? It’s possible, but it’s not simple. You could learn all of the Apache Kafka tools for handling a cluster well enough to move your Kafka workloads to Kubernetes, or you could leverage the Kubernetes knowledge you already have using Strimzi.

Note: Strimzi will be represented at the virtual KubeCon Europe 2020 conference from 17-20 August 2020. See the end of the article for details.

Continue reading “Introduction to Strimzi: Apache Kafka on Kubernetes (KubeCon Europe 2020)”

Share
Consuming messages from closest replicas in Apache Kafka 2.4.0 and AMQ Streams

Consuming messages from closest replicas in Apache Kafka 2.4.0 and AMQ Streams

Thanks to changes in Apache Kafka 2.4.0, consumers are no longer required to connect to a leader replica to consume messages. In this article, I introduce you to Apache Kafka’s new ReplicaSelector interface and its customizable RackAwareReplicaSelector. I’ll briefly explain the benefits of the new rack-aware selector, then show you how to use it to more efficiently balance load across Amazon Web Services (AWS) availability zones.

For this example, we’ll use Red Hat AMQ Streams with Red Hat OpenShift Container Platform 4.3, running on Amazon AWS.

Continue reading “Consuming messages from closest replicas in Apache Kafka 2.4.0 and AMQ Streams”

Share
Using secrets in Kafka Connect configuration

Using secrets in Kafka Connect configuration

Kafka Connect is an integration framework that is part of the Apache Kafka project. On Kubernetes and Red Hat OpenShift, you can deploy Kafka Connect using the Strimzi and Red Hat AMQ Streams Operators. Kafka Connect lets users run sink and source connectors. Source connectors are used to load data from an external system into Kafka. Sink connectors work the other way around and let you load data from Kafka into another external system. In most cases, the connectors need to authenticate when connecting to the other systems, so you will need to provide credentials as part of the connector’s configuration. This article shows you how you can use Kubernetes secrets to store the credentials and then use them in the connector’s configuration.

Continue reading “Using secrets in Kafka Connect configuration”

Share
Accessing Apache Kafka in Strimzi: Part 5 – Ingress

Accessing Apache Kafka in Strimzi: Part 5 – Ingress

In the fifth and final part of this series, we will look at exposing Apache Kafka in Strimzi using Kubernetes Ingress. This article will explain how to use Ingress controllers on Kubernetes, how Ingress compares with Red Hat OpenShift routes, and how it can be used with Strimzi and Kafka. Off-cluster access using Kubernetes Ingress is available only from Strimzi 0.12.0. (Links to previous articles in the series can be found at the end.)

Continue reading “Accessing Apache Kafka in Strimzi: Part 5 – Ingress”

Share
Accessing Apache Kafka in Strimzi: Part 4 – Load balancers

Accessing Apache Kafka in Strimzi: Part 4 – Load balancers

In this fourth article of our series about accessing Apache Kafka clusters in Strimzi, we will look at exposing Kafka brokers using load balancers. (See links to previous articles at end.) This article will explain how to use load balancers in public cloud environments and how they can be used with Apache Kafka.

Continue reading “Accessing Apache Kafka in Strimzi: Part 4 – Load balancers”

Share
Accessing Apache Kafka in Strimzi: Part 3 – Red Hat OpenShift routes

Accessing Apache Kafka in Strimzi: Part 3 – Red Hat OpenShift routes

In the third part of this article series (see links to previous articles below), we will look at how Strimzi exposes Apache Kafka using Red Hat OpenShift routes. This article will explain how routes work and how they can be used with Apache Kafka. Routes are available only on OpenShift, but if you are a Kubernetes user, don’t be sad; a forthcoming article in this series will discuss using Kubernetes Ingress, which is similar to OpenShift routes.

Continue reading “Accessing Apache Kafka in Strimzi: Part 3 – Red Hat OpenShift routes”

Share
Accessing Apache Kafka in Strimzi: Part 2 – Node ports

Accessing Apache Kafka in Strimzi: Part 2 – Node ports

This article series explains how Apache Kafka and its clients work and how Strimzi makes it accessible for clients running outside of Kubernetes. In the first article, we provided an introduction to the topic, and here we will look at exposing an Apache Kafka cluster managed by Strimzi using node ports.

Specifically, in this article, we’ll look at how node ports work and how they can be used with Kafka. We also will cover the different configuration options available to users and the pros and cons of using node ports.

Continue reading “Accessing Apache Kafka in Strimzi: Part 2 – Node ports”

Share
Accessing Apache Kafka in Strimzi: Part 1 – Introduction

Accessing Apache Kafka in Strimzi: Part 1 – Introduction

Strimzi is an open source project that provides container images and operators for running Apache Kafka on Kubernetes and Red Hat OpenShift. Scalability is one of the flagship features of Apache Kafka. It is achieved by partitioning the data and distributing them across multiple brokers. Such data sharding has also a big impact on how Kafka clients connect to the brokers. This is especially visible when Kafka is running within a platform like Kubernetes but is accessed from outside of that platform.

This article series will explain how Kafka and its clients work and how Strimzi makes it accessible for clients running outside of Kubernetes.

Continue reading “Accessing Apache Kafka in Strimzi: Part 1 – Introduction”

Share