Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat OpenShift AI
      Red Hat OpenShift AI
    • Red Hat Enterprise Linux AI
      Linux icon inside of a brain
    • Image mode for Red Hat Enterprise Linux
      RHEL image mode
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • Red Hat Developer Hub
      Developer Hub
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Image mode for Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of OpenJDK
    • Kubernetes

      • Red Hat OpenShift
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Virtualization
      • Red Hat OpenShift Lightspeed
    • Integration & App Connectivity

      • Red Hat Build of Apache Camel
      • Red Hat Service Interconnect
      • Red Hat Connectivity Link
    • AI/ML

      • Red Hat OpenShift AI
      • Red Hat Enterprise Linux AI
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Lightspeed
    • Developer tools

      • Red Hat Trusted Software Supply Chain
      • Podman Desktop
      • Red Hat OpenShift Dev Spaces
    • Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • API Catalog
    • Product Documentation
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

Deploying the Mosquitto MQTT message broker on Red Hat OpenShift, Part 1

April 16, 2021
Kevin Boone
Related topics:
ContainersKubernetesEvent-Driven

Share:

    Mosquitto is a lightweight message broker that supports the Message Queuing Telemetry Transport  (MQTT) protocol. Mosquitto is widely used in Internet of Things (IoT) and telemetry applications, where a fully-featured message broker like Red Hat AMQ would be unnecessarily burdensome. Mosquitto also finds a role as a message bus for interprocess communication in distributed systems. Because it avoids complex features, Mosquitto is easy to tune and handles substantial application workloads with relatively modest memory and CPU resources.

    There are essentially two stages to making Mosquitto available on Red Hat OpenShift. First, you need to containerize the application in a way that is broadly compatible with OpenShift. Part of containerization involves installing the container image in a repository, from which OpenShift can download it. Second, you need to deploy the containerized image in a pod, providing whatever properties and configuration are necessary for the specific installation. The first half of this article shows how to build Mosquitto into an image suitable for use in a container. The second half will show you how to configure and deploy the Mosquitto image on OpenShift.

    Mosquitto basics

    Mosquitto's messaging model is publish-subscribe, like a "topic" in Java Message Service (JMS) terminology. In fact, the Mosquitto documentation uses the term "topic" for its message destinations. In the terminology of AMQ 7 (and ActiveMQ Artemis, on which AMQ 7 is based), Mosquitto destinations are non-durable anycast addresses by default. Mosquitto communicates only over MQTT: a lightweight, payload-neutral, non-transactional wire protocol.

    Note: A detailed description of Mosquitto is beyond the scope of this article. For more information, please see the Mosquitto website.

    It's relatively easy to deploy Mosquitto in an OpenShift pod: Mosquitto has (and needs) no clustering support, so it lends itself to simple deployment methods. While you might find uses for Mosquitto on OpenShift, this article focuses as much on software packaging in an OpenShift-compatible way as it does on Mosquitto itself. Many widely-used software packages were not designed for cloud or container operation; as a result, they present challenges to creating a maintainable, configurable installation.

    Using Mosquitto with OpenShift

    For this article, I make certain assumptions about how Mosquitto (or any other lightweight application) will be used on OpenShift:

    • You want to minimize pod startup time and container image size. There's no benefit to running multiple replicas of Mosquitto, so in the event of a crash, you need a new pod to spin up in milliseconds.
    • The application will be used both from within the OpenShift cluster, using a service, and from outside it, using a route. Compatibility with the OpenShift routing framework necessitates the use of Transport Layer Security (TLS) encryption in the wire protocol, as I will discuss later.
    • Client access will be authenticated rather than be left completely open, both for intercluster and external clients. In this example, I'll use simple user-password authentication, although Mosquitto also supports client certificates.
    • Although the installer will need to customize the deployment on OpenShift, an installation that uses only the defaults in the container image should be functional enough for testing. It should not be mandatory to provide a complex configuration just to get started. That means, for example, providing the container image with default TLS certificates.

    In the instructions, I'm also assuming you have a local workstation installation of Mosquitto--or at least, the Mosquitto test clients--for testing. On Fedora and Red Hat Enterprise Linux (RHEL) systems, typing dnf install mosquitto in the shell should do the trick.

    For the sake of brevity, I won't display all the relevant files in their entirety. You can obtain all of these files from my GitHub repository.

    Note that I'll use Podman to build and run images in my demonstration. Docker and Buildah should also work without modifying the example.

    Configuration files for containerization

    Mosquitto needs at least four files, or sets of files, to provide a rudimentary service on OpenShift. You might need to configure these files at installation time:

    • Credentials for at least one user
    • A set of TLS certificates
    • A configuration file for the broker
    • A startup script

    A standard Mosquitto installation provides default locations for configuration files. In this example, however, I've chosen to put all the files that might need to be modified by the deployer into a single directory in the container image. I've chosen the home directory for the user account that will own the broker. I'm using the directory /myuser. The directory's name is arbitrary, but administration is simpler if all the files that an installer will need to change are in the same directory.

    In the source repository, all the relevant configuration files and certificates are in the files directory, and will be copied to /myuser in the image when it is built.

    We'll also need a Dockerfile to build the container image from a base image, the Mosquitto binaries, and the configuration files. It will be possible (perhaps necessary is a better word) to test the generated image on the local workstation before installing it on OpenShift.

    Credentials file

    Mosquitto provides a simple user-password authentication mechanism based on a credentials file. The credentials file is in a proprietary format, containing hashed passwords. Mosquitto provides a utility named mosquitto_passwd for editing the file.

    The default image provides one user  named admin with the password admin, in a credentials file called passwd. I created this file as follows:

    $ touch files/passwd
    $ mosquitto_passwd -b files/passwd admin admin
    

    Note: In Part 2, I will explain how to provide different credentials when deploying the image on OpenShift.

    To create multiple users with different privileges, edit the main configuration file and add users to the credentials file.

    TLS certificates

    Mosquitto requires at least three certificates for a practical installation. Like all the other files that an installer may override, these will be installed into the image in the /myuser directory. These certificates are:

    • A root certification authority (CA) certificate against which all the others will be authenticated. In my example, this file is named ca.crt. You must share this certificate with clients. I'm generating this file using OpenSSL in the example, but in a production installation, it might be a trusted certificate from a commercial CA.
    • A server certificate authenticated by the CA. This will be called server.crt. This is the certificate that Mosquitto will supply to clients during the TLS handshake.
    • The primary key certificate corresponding to the server certificate. This will be called server.key.

    All these certificates must be in PEM format and are easily generated using the openssl utility.

    The following example shows the commands that I used to generate the certificates in the default image. You might want to use similar commands to bake different certificates into the image; however, I envisage all these files being overridden in OpenShift at installation time, from a secret or configmap (more about this in Part 2).

    $ openssl req -new -x509 -days 3650 -extensions v3_ca -keyout files/ca.key \
         -out files/ca.crt -subj "/O=acme/CN=com"
    $ openssl genrsa -out files/server.key 2048
    $ openssl req -new -out files/server.csr -key files/server.key -subj "/O=acme2/CN=com"
    $ openssl x509 -req -in files/server.csr -CA files/ca.crt -CAkey files/ca.key \
        -CAcreateserial -out files/server.crt -days 3650
    $ openssl rsa -in files/server.key -out files/server.key
    $ rm files/ca.key files/ca.srl files/server.csr
    $ chmod 644 files/server.key
    

    Main configuration file

    The main configuration file specifies the locations of the credentials files and certificate files and creates TCP listeners on ports 1883 (plaintext) and 8883 (TLS). These port numbers are conventional for MQTT, and the Mosquitto utilities assume they are in use unless otherwise specified. Here is the mosquitto.confconfiguration file for this example.

    # Port for plaintext communication
    port 1883
    
    # Location of the credentials file
    password_file /myuser/passwd
    
    # Port and certificates for TLS encrypted communication
    listener 8883
    certfile /myuser/server.crt
    cafile /myuser/ca.crt
    keyfile /myuser/server.key

    Note that all the referenced files are in /myuser. This directory will be copied and populated during the image build.

    In practice, this configuration file will probably need to be overridden. If it is, the new version will probably also need to specify a credentials file and certificate files.

    Startup script

    Starting up is trivially simple in this example. The script startup.sh just runs the mosquitto command, specifying the main configuration file:

    #!/bin/sh
    mosquitto -c /myuser/mosquitto.conf
    

    Building, testing, and publishing the image

    To create the smallest possible image, I'm using Alpine Linux as a base. Alpine is a minimal Linux specifically designed for containers. To further reduce its size and complexity, Alpine uses the MUSL C library rather than the Glibc that is almost ubiquitous in Linux. This decision mostly limits Alpine's use to applications that are available in the Alpine repository or can be built from source.

    Note: At the time of writing, the latest Alpine release is 3.12. The corresponding Alpine repository doesn't have a binary package for Mosquitto, so to simplify the image build, I'm specifying Alpine 3.11. To use a later version of Alpine, if the package is not present in the repository, you'll need to build Mosquitto from source. Building Mosquitto is not difficult, but to build a version for Alpine, you must build against the MUSL C library. A straightforward way to do this is to install a desktop version of Alpine Linux, perhaps in a virtual machine, and do the build there. Another approach is to build Mosquitto in-image, using a multistage build process to eliminate all the build tools from the final image. There's a lot more information on this multistage build process in my article Developing micro-microservices in C on Red Hat OpenShift. Here, I'm happy to use the binary package from the Alpine repository.

    Here is a simple Dockerfile that creates the image, using Alpine Linux as a base:

    FROM alpine:3.11
    RUN addgroup -g 1000 mygroup && \
         adduser -G mygroup -u 1000 -h /myuser -D myuser && \
         chown -R myuser:mygroup /myuser && \
         apk --no-cache add mosquitto
    WORKDIR /myuser
    COPY files/* /myuser/
    USER myuser
    EXPOSE 1883 8883
    CMD ["/myuser/start.sh"]
    

    There's nothing remarkable about this Dockerfile. It creates a single user, exposes the ports that Mosquitto uses, and specifies our startup script as the entry point.

    The total size of the final image is only about 7 MB.

    Building and testing the image locally

    Build the image using:

    $ podman build .
    

    Get the ID of the new image (using, for example, podman image list) and then run it, specifying the port mappings. In the following command, replace the italicized word image with the ID of your image:

    $ podman run -it -port 1883:1883 -p 8883:8883 image

    This command exposes the plaintext and TLS ports. We can test the image using the Mosquitto test clients mosquitto_pub (publish) and/or mosquitto_sub (subscribe). For example, to publish a message using the plaintext listener, enter:

    $ mosquitto_pub -t foo -m "Some text" -u admin -P admin
    

    The -t foo option specifies the Mosquitto topic, while -m specifies the data to send. The user (-u option) and password (-P option) must match the values given in the mosquitto_passwd command when creating the credentials file earlier. Note that the hostname and port default to localhost and 1883, which are appropriate in this case.

    To test the TLS listener, you'll need to specify the CA certificate. The certificate is in the source repository's files directory and in the image (I'll discuss how to obtain the certificate from the OpenShift pod in Part 2):

    $ mosquitto_sub -t foo --cafile files/ca.crt --insecure -u admin -P admin
    

    Again, the defaults for host and port will be appropriate. You need the --insecure switch to override certificate hostname checks; this is because the server certificate in the image has the hostname acme.com, not localhost.

    Note: Mosquitto is non-durable by default. So if you want to test that messages can be produced and consumed. you'll need to start the consumer before the producer.

    Publishing the image

    If the image works adequately on a workstation, you must publish it to a repository from which OpenShift can download it. The procedures for doing this are outside the scope of this article. I've published my image to quay.io using the following podman procedure:

    $ podman tag <image-id> mosquitto-ephemeral:0.1a
    $ podman login quay.io...
    $ podman push mosquitto-ephemeral:0.1a quay.io/kboone/mosquitto-ephemeral
    

    This is a public repository, so feel free to use my image for testing if you don't want to build your own. I'm using the tag ephemeral in the image name to indicate that this implementation does not support durable messaging.

    Of course, you could publish the image to another public image repository, or a private, institutional repository, or directly to the OpenShift internal repository if you're willing to expose it as a route. For detailed steps, see the article Using Podman to deploy an image directly to OpenShift 4 on my website.

    However you publish the image, you'll need to know the repository URI for the OpenShift deployment. In this example using the quay.io repository, that URI is:

    quay.io/kboone/mosquitto-ephemeral:latest
    

    Conclusion to Part 1

    This article described some of the design considerations that are relevant to deploying a lightweight application on OpenShift. I don't claim that my approach is the only one you could use, or even particularly optimal. It is, however, straightforward enough for a demonstration, and might highlight a variety of issues that packagers and deployers should consider. Look for Part 2, where we will deploy the Mosquitto image we built in this article on OpenShift.

    Last updated: April 15, 2021

    Recent Posts

    • More Essential AI tutorials for Node.js Developers

    • How to run a fraud detection AI model on RHEL CVMs

    • How we use software provenance at Red Hat

    • Alternatives to creating bootc images from scratch

    • How to update OpenStack Services on OpenShift

    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue