Kafka 101

Learning path | 7 resources | 1 hr and 30 mins | Published on March 24, 2022
Learn about the fundamentals of Apache Kafka. This tutorial covers basic concepts of Kafka and its components.

  1. Overview: Kafka 101
  2. What are topics? Page | 15 mins
  3. What are partitions? Page | 15 mins
  4. What are messages? Page | 15 mins
  5. What are producers? Page | 15 mins
  6. What are consumers? Page | 15 mins
  7. Kafka ecosystem Page | 15 mins

Overview: Kafka 101

Hello Kafka

What is Apache Kafka?

Apache Kafka is a distributed data streaming platform that enables applications to publish, subscribe to, store, and process streams of messages in real-time. Publish/subscribe (pub/sub) systems are characterized by senders pushing messages to a central point for classification. Subscribers receive messages of interest from the central point.

Apache Kafka offers a few major advantages over traditional pub/sub systems:

  • Storing messages with fault tolerance.
  • Processing streams (data streams) as they occur in real-time instead of in a batch.
  • Guarantees that messages are never overridden.
  • Support for very high throughput.

Publish/subscribe systems require a broker, which is the central point where messages are published. A typical Kafka cluster contains multiple brokers (Figure 1). Topics are hosted on the brokers, and each topic is split into one or more partitions. We will dive into each of these concepts in the next few sections.

Kafka provides brokers to take messages from producers Messages are produced to topics by producers.. Consumers consume messages from specific topics.
Figure 1: Kafka provides brokers to take messages from producers Messages are produced to topics by producers. Consumers consume messages from specific topics.