Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat OpenShift AI
      Red Hat OpenShift AI
    • Red Hat Enterprise Linux AI
      Linux icon inside of a brain
    • Image mode for Red Hat Enterprise Linux
      RHEL image mode
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • Red Hat Developer Hub
      Developer Hub
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Image mode for Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of OpenJDK
    • Kubernetes

      • Red Hat OpenShift
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Virtualization
      • Red Hat OpenShift Lightspeed
    • Integration & App Connectivity

      • Red Hat Build of Apache Camel
      • Red Hat Service Interconnect
      • Red Hat Connectivity Link
    • AI/ML

      • Red Hat OpenShift AI
      • Red Hat Enterprise Linux AI
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Lightspeed
    • Developer tools

      • Red Hat Trusted Software Supply Chain
      • Podman Desktop
      • Red Hat OpenShift Dev Spaces
    • Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • API Catalog
    • Product Documentation
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

Extending Kafka connectivity with Apache Camel Kafka connectors

May 19, 2020
Hugo Guerrero
Related topics:
Event-DrivenMicroservicesOpen source
Related products:
Red Hat build of Apache Camel

Share:

    Apache Kafka is one of the most used pieces of software in modern application development because of its distributed nature, high throughput, and horizontal scalability. Every day more and more organizations are adopting Kafka as the central event bus for their event-driven architecture. As a result, more and more data flows through the cluster, making the connectivity requirements rise in priority for any backlog. For this reason, the Apache Camel community released the first iteration of Kafka Connect connectors for the purpose of easing the burden on development teams.

    What is Apache Camel?

    The Camel community has built one of the busiest open source integration frameworks in the Apache Foundation. The Camel framework lets you quickly and easily integrate data consumer and producer systems. It implements the most used Enterprise Integration Patterns (EIPs), plus the interfaces and protocols that are used everywhere, as they emerge. Having everything under the same component base configuration lets you create the required building blocks to solve almost any integration requirement.

    What is the Camel Kafka Connector project?

    After maturing for almost a decade, the Camel community spun a couple of subprojects to foster innovation in areas like runtime support and container readiness. In particular, the Camel Kafka Connector subproject focuses on the use of Camel components as Kafka Connect connectors. With this in mind, they built a tiny layer between Camel and Kafka frameworks to easily use each Camel component as a Kafka connector that can be used effortlessly in the Kafka ecosystem.

    This project's first release allows the community to try and share feedback on the autogenerated connectors. Despite being the first release of this subproject, most of the underlying Camel components are battle-tested and used all around the world in production scenarios.

    Getting started with Camel Kafka connectors

    There are more than 340+ Camel Kafka connectors to get started, ranging from AWS S3 integration to Telegram or Slack. So it should be easy to find a use case to implement a small project.

    In order to simplify Kafka Connect cluster deployment, you can use the Kubernetes Operator from the Strimzi project to run on Minikube or Kind on your laptop. Then, search for and download the connector package zip version from the Maven central repository. The zip file should contain all of the library jars required to run the connector in your plugins path. Next, create a container image with your plugins on top of the Strimzi base image so you can use the image as the Kafka Connect base. And then, finally, configure the connector using the Kafka Connect REST API or the new Strimzi Kafka Connector custom resource definition (CRD).

    You can also try the connectors locally on your laptop without the need for Kubernetes. You just need a locally running Kafka instance and to follow these instructions.

    Want to see how it looks when running? Watch my video on integrating Slack to Kafka:

    Summary

    Apache Kafka is used as an event backbone in new organizations every day. Communities like Apache Camel are working on how to speed up development in key areas of the modern application, like integration. The Camel Kafka Connect project from the Apache Foundation has enabled their vast set of connectors to interact with Kafka Connect natively so that developers can start sending and receiving data from Kafka on their preferred systems.

    Featured image source: "Getting Kafka ready for the Camel Ride" is a derivative work of "Kafka" by "g p" and "Camel" by "Ziad Fhema," used under CC BY 2.0.

    Last updated: October 8, 2024

    Recent Posts

    • Expand Model-as-a-Service for secure enterprise AI

    • OpenShift LACP bonding performance expectations

    • Build container images in CI/CD with Tekton and Buildpacks

    • How to deploy OpenShift AI & Service Mesh 3 on one cluster

    • JVM tuning for Red Hat Data Grid on Red Hat OpenShift 4

    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit
    © 2025 Red Hat

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue