Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat OpenShift AI
      Red Hat OpenShift AI
    • Red Hat Enterprise Linux AI
      Linux icon inside of a brain
    • Image mode for Red Hat Enterprise Linux
      RHEL image mode
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • Red Hat Developer Hub
      Developer Hub
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Image mode for Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of OpenJDK
    • Kubernetes

      • Red Hat OpenShift
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Virtualization
      • Red Hat OpenShift Lightspeed
    • Integration & App Connectivity

      • Red Hat Build of Apache Camel
      • Red Hat Service Interconnect
      • Red Hat Connectivity Link
    • AI/ML

      • Red Hat OpenShift AI
      • Red Hat Enterprise Linux AI
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Lightspeed
    • Developer tools

      • Red Hat Trusted Software Supply Chain
      • Podman Desktop
      • Red Hat OpenShift Dev Spaces
    • Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • API Catalog
    • Product Documentation
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

Deploying the Mosquitto MQTT message broker on Red Hat OpenShift, Part 2

April 26, 2021
Kevin Boone
Related topics:
ContainersEvent-DrivenKubernetes
Related products:
Red Hat OpenShift

Share:

    The first half of this article introduced the Mosquitto  Message Queuing Telemetry Transport  (MQTT) message broker and showed how to build Mosquitto into an image suitable for use in a container. In this second half of the article, you will configure and deploy the Mosquitto image into an application that runs on Red Hat OpenShift. You can obtain the files for the example from my GitHub repository.

    Configuring OpenShift

    Assuming you've read Part 1, you should have the basic Mosquitto image in a repository, with a URI you can refer to when deploying on OpenShift. If you're familiar with the Docker and Podman family of tools, and with the software you want to deploy, configuration and deployment aren't particularly difficult. It took me less time to implement, build, and test the image than it did to describe how to do it for this article.

    I believe the most broadly compatible way to deploy the image to OpenShift is to use a deployment configuration in YAML format. Of course, there are other ways to deploy an image, but the steps in this article should work without changes on any OpenShift version.

    I will describe two deployments. The first uses only the defaults in the image. This should provide something that you can test. Then, I'll describe how to override configuration files to create a site-specific installation.

    Deploying a default image on OpenShift

    First, you need to create a YAML deployment configuration. Please note that the following snippet is not a complete deployment configuration: I have removed most of the boilerplate code, extracting just the content that is specific to this application. The complete YAML file is mosquitto-ephemeral.yaml in the source repository. This file specifies two services as well as the deployment configuration: one service for the plaintext port 1883, and one for the TLS port 8883. Exposed ports must also be listed in the deployment configuration, but that in itself does not create a service that other applications can connect to—you need services as well:

    kind: DeploymentConfig
    apiVersion: apps.openshift.io/v1
    metadata:
      name: mosquitto-ephemeral
    spec:
      replicas: 1
        spec:
          containers:
              name: mosquitto-ephemeral
              image: quay.io/kboone/mosquitto-ephemeral:latest
              imagePullPolicy: Always
              ports:
                - containerPort: 1883
                  protocol: TCP
                - containerPort: 8883
                  protocol: TCP
              resources:
                limits:
                  memory: 128Mi
    ---
    kind: Service
    metadata:
      name: mosquitto-ephemeral-tcp
    spec:
      ports:
          port: 1883
          targetPort: 1883
    ---
    kind: Service
    metadata:
      name: mosquitto-ephemeral-tls
    spec:
      ports:
          port: 8883
          targetPort: 8883
    

    The deployment configuration specifies the image to download (using its repository URI), the number of replicas (which has to be 1 to be useful in this case), and a memory limit. Mosquitto doesn't usually need much memory, so I've set a limit of 128 MB. In practice, you would need a lot of client load to use anything approaching that much memory. The full YAML deployment configuration specifies liveness and readiness probes and other configurations that are important in general, but not relevant to this article.

    To deploy the YAML file, and thus create the pod and the services, enter:

    $ oc apply -f mosquitto-ephemeral.yaml
    

    In a little while—and it really should be a little while, given the image to be pulled from the repository is only 7 MB—you should see the pod running:

    $ oc get pods
    NAME READY STATUS RESTARTS AGE
    mosquitto-ephemeral-1-5clnd 1/1 Running 1 23h
    

    You can check to be sure there are services for the plaintext and Transport Layer Security (TLS) ports as follows:

    $ oc get svc
    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    mosquitto-ephemeral-tcp ClusterIP 172.30.207.29 <none> 1883/TCP 24h
    mosquitto-ephemeral-tls ClusterIP 172.30.163.226 <none> 8883/TCP 24h
    

    At this point, other pods in the cluster should be able to connect to the MQTT broker using the service hostnames mosquitto-ephemeral-tcp and mosquitto-ephemeral-tls, with the appropriate ports.

    Define a route

    For MQTT clients to be able to connect to the broker pod from outside the OpenShift cluster, you must define a TLS-encrypted route with "passthrough" termination mode. The OpenShift router will not be able to route a plaintext MQTT connection, because the MQTT protocol does not carry a destination hostname as, for example, HTTP does in its "Host:" header. Instead, the router will identify the target pod by examining the server name identification (SNI) information in the TLS handshake. In passthrough mode, the router passes the TLS handshake to the target pod, so the certificate received by the client will be the pod's certificate, not the OpenShift router's certificate. It is the pod's certificate, which has been installed in the deployed image, that the client must trust.

    So, to create the route, enter:

    $ oc create route passthrough --service=mosquitto-ephemeral-tls \
            --port 8883 --hostname=mosquitto.apps.my_domain

    Note that the chosen hostname must be mapped by your DNS configuration to the OpenShift router, and the client must connect to the OpenShift router using that exact name. Remember that it is this hostname, appearing in the TLS handshake, that allows the router to find the correct pod.

    Get the CA certificate from the image

    To connect an MQTT client from outside the OpenShift cluster, you'll need the CA certificate from the image. How can you get the certificate from the image if you don't have the source code that created it? That question shouldn't come up in practice because the deployer will have replaced the default certificates in the image with site-specific ones. However, if you must, you can get the CA certificate directly from the pod as follows.

    First, get the number of the pod to which the client wants to connect:

    $ oc get pod
    

    Then, issue the following command, changing the italicized podnum to the number of the pod you just obtained:

    $ oc cp mosquitto-ephemeral-1-podnum:/myuser/ca.crt mosquitto_ca.crt
    

    Test the installation

    Having retrieved the certificate (or copied it from the source code repository), you can test the Mosquitto broker in the pod using its hostname and the port 443 (this is the default TLS port for the OpenShift router—not the pod's TLS port, which the client will never see):

    $ mosquitto_pub -t foo -m "text" --cafile mosquitto_ca.crt \
      --insecure -u admin -P admin --host mosquitto.apps.my_domain --port 443
    

    You still need --insecure in this command, because the hostname in the certificate (acme.com) doesn't match the hostname supplied by the client. In fact, you'll always need to disable hostname verification, unless you generate a server certificate whose hostname is a match for mosquitto.apps.my_domainor the equivalent in your own installation.

    Deploying a custom image on OpenShift

    The preceding steps—which, again, took longer to write about than to do—deployed the Mosquitto image with default settings; that is, settings that were baked into the image. In practice, some site-specific configuration will almost always be necessary. In this section, I'll show you how to provide a custom credentials file to specify a non-default user. You can use the same mechanism to override any file in the image, assuming you have access to the deployment configuration. I've chosen the credentials file because using it to demonstrate the principle requires substituting only a single file; to change the TLS certificates, we would have to substitute all three.

    Why not just edit the configuration files in the pod?

    Developers who are new to OpenShift sometimes ask why they can't just log into a pod and edit the configuration files. There are two reasons for this: one practical and one philosophical.

    The practical reason is that the pod's default user (myuser) doesn't have write access to the relevant files. They are owned by root, and intentionally so; this would be a bad way to manage an installation. It is possible (although strongly discouraged) to change the permissions on these configuration files. Then, an administrator could log into the pod, edit the files, and send the broker the SIGHUP signal to make it reload its configuration.

    The more philosophical reason is that the administrator's relationship to OpenShift pods should be as to cattle, not to pets. This metaphor refers to the fungibility of pods. The OpenShift infrastructure can terminate and restart a pod at any time. When this happens, the pod's filesystem is restored from the image, and any manual changes will be lost.

    Thus, the safe way to make configuration changes to a pod is to change the specification that builds the pod. Here, that means changing the deployment configuration. With this approach, all pods constructed from that deployment configuration will be identical.

    Specify a non-default user

    The most conceptually straightforward way to change the authentication configuration in the image is simply to override the existing /myuser/passwd file. This isn't necessarily the easiest way for the administrator, but it's probably the easiest to understand. To do this, we'll insert the new file into an OpenShift configmap, then modify the deployment configuration so that the new file gets inserted into the pod's filesystem when it starts up.

    First, create a new credentials file with a new user and password:

    $ touch passwd
    $ mosquitto_passwd -b passwd foo foo
    

    This credential file defines a single user, foo, with password foo.

    Now, create a configmap called passwd from the file passwd:

    $ oc create configmap passwd --from-file=passwd=./passwd
    

    The configmap and file don't have to have the same name. In fact, you can store multiple files in the same configmap if necessary.

    Modify the deployment configuration

    Now, you need to modify the deployment configuration so that it inserts the passwd file from the configmap, replacing the default file in the image. As before, I'm showing only the relevant part of the YAML file; the complete file is mosquitto-ephemeral-passwd.yaml in the source repository:

    spec:
         containers:
            volumeMounts:
              - name: passwd-mount
                mountPath: /myuser/passwd
                subPath: passwd
    
        volumes:
            configMap:
              name: passwd-mount
              items:
                - key: passwd
                  path: passed
    

    The volumeMounts section defines paths to files that are to be mounted (that is, made available), and identifies the sources of those files. There should be a corresponding volumes section that defines the actual sources. In this case, the file's source is a configmap. The name passwd-mount associates the file to be mounted with the source of that file.

    You can use the same technique to provide custom versions of any other files in the image, including the TLS certificates.

    Alternatives to file substitution

    One advantage of this file-mounting technique is that, if no mappings are present in the deployment configuration, the image will fall back to its defaults. As a result, it's possible to get something working for test purposes without complex configuration.

    In practice, this file substitution method can be awkward and error-prone for the administrator, particularly when it isn't entirely clear what files need to be replaced and what format they have. The image's maintainer needs to provide very clear documentation of the purpose, format, and location of each file that might need to be provided by the deployer.

    An alternative approach is to put specific configuration properties (rather than whole files) into a configmap, and have the pod's startup script unpack the configmap and build a set of configuration files at runtime.

    In fact, it's possible to omit the configmap completely, and specify environmental properties directly in the deployment configuration. These properties are injected into ordinary Linux environment variables that can be read by the startup script. This script, similar to the technique previously described, can build configuration files from the properties at runtime.

    An entirely more elegant approach is to use application templates. A template provides a way to generate a deployment configuration based on values that can be entered into a web form in the OpenShift console. I won't describe this approach in detail because most of Red Hat's OpenShift products are moving away from the use of templates to Operators. Operators provide a very powerful way to configure complex software installations, but it seems unlikely that an installation of the Mosquitto message broker will ever need, or benefit from, such sophistication.

    Conclusion to Part 2

    This article introduced you to the Mosquitto MQTT message broker, which is widely used in Internet of Things (IoT) and telemetry applications. I used Mosquitto to demonstrate how to containerize and deploy packages that were not designed for containerization. As you've seen, OpenShift works with a wide range of tools for loading images, authenticating users, and other tasks.

    Last updated: February 5, 2024

    Recent Posts

    • Storage considerations for OpenShift Virtualization

    • Upgrade from OpenShift Service Mesh 2.6 to 3.0 with Kiali

    • EE Builder with Ansible Automation Platform on OpenShift

    • How to debug confidential containers securely

    • Announcing self-service access to Red Hat Enterprise Linux for Business Developers

    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit
    © 2025 Red Hat

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue