Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat OpenShift AI
      Red Hat OpenShift AI
    • Red Hat Enterprise Linux AI
      Linux icon inside of a brain
    • Image mode for Red Hat Enterprise Linux
      RHEL image mode
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • Red Hat Developer Hub
      Developer Hub
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Image mode for Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of OpenJDK
    • Kubernetes

      • Red Hat OpenShift
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Virtualization
      • Red Hat OpenShift Lightspeed
    • Integration & App Connectivity

      • Red Hat Build of Apache Camel
      • Red Hat Service Interconnect
      • Red Hat Connectivity Link
    • AI/ML

      • Red Hat OpenShift AI
      • Red Hat Enterprise Linux AI
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Lightspeed
    • Developer tools

      • Red Hat Trusted Software Supply Chain
      • Podman Desktop
      • Red Hat OpenShift Dev Spaces
    • Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • API Catalog
    • Product Documentation
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

Introducing Tempo Monolithic mode

August 14, 2024
Andreas Gerstmayr
Related topics:
Observability
Related products:
Red Hat OpenShift

Share:

    Starting with Red Hat OpenShift distributed tracing 3.2 (tempo-operator v0.10.0), the Tempo operator supports an additional mode of deploying Tempo: the Tempo Monolithic mode, as a tech preview feature.

    In this article, we introduce Tempo Monolithic mode, describe the pros and cons, and present example deployment manifests.

    What is Tempo Monolithic mode?

    In Tempo Monolithic mode all core components such as compactor, distributor, ingester, querier, and query-frontend of Tempo are contained in a single binary, in a single container. This vastly simplifies the deployment, as only a single pod is created, and avoids potential issues arising from distributed deployments such as connectivity issues between pods or nodes, scheduling issues, etc. Additionally, this mode supports storing traces in-memory, and in a Persistent Volume. However, this mode comes at the expense of scalability: this mode does not scale horizontally. To scale your Tempo deployment horizontally, continue using the Tempo Microservices deployment via the TempoStack Custom Resource (CR).

    The Monolithic mode is ideal for small deployments, demo and test setups, and is the recommended migration path of the Red Hat OpenShift distributed tracing platform (Jaeger) all-in-one deployment.

    Migrating between TempoStack and TempoMonolithic

    For Tempo deployments using object storage, you can switch between TempoStack and TempoMonolithic by using the same storage secret (don't forget to stop the previous deployment). Migration from/to a TempoMonolithic instance using in-memory or Persistent Volume storage is not supported.

    Example Deployment using in-memory storage

    The following manifest creates a TempoMonolithic deployment with trace ingestion over OTLP/gRPC and OTLP/HTTP, storing traces in a 2 GiB tmpfs volume (in-memory storage) and exposing Jaeger UI via a Route:

    apiVersion: tempo.grafana.com/v1alpha1
    kind: TempoMonolithic
    metadata:
      name: sample
    spec:
      storage:
        traces:
          backend: memory
          size: 2Gi
      jaegerui:
        enabled: true
        route:
          enabled: true

    Once the pod is ready, you can send traces to tempo-sample:4317 (OTLP/gRPC) and tempo-sample:4318 (OTLP/HTTP) inside the cluster. The Tempo API is available at tempo-sample:3200 inside the cluster.

    Jaeger UI is available at the location of the tempo-sample-jaegerui Route (Navigate to the OpenShift Console → Networking → Routes).

    To persist the traces in a Persistent Volume instead, use the following options in the storage block:

    spec:
      [...]
      storage:
        traces:
          backend: pv
          size: 10Gi

    Configuring retention

    Tempo supports retention based on time. You can configure the maximum block age using the following spec:

    spec:
      [...]
      extraConfig:
        tempo:
          compactor:
            compaction:
              block_retention: 24h

    Note that this does not translate to the time of any trace or span. Internally, the ingester component of Tempo batches traces up to max_block_bytes (default: 500MB) or max_block_duration (default: 30m) into a block and flushes it to the configured storage. Another component, the compactor, combines (merges) multiple blocks in a time window of compaction_window (default: 1h) to a new block, and marks the input blocks for deletion. The block_retention (default: 14d) setting specifies at which block age (last modification time of a block) blocks are deleted.

    Therefore, the block_retention is only indirectly related to the last received trace—due to (possibly multiple) compaction cycles, traces may be stored longer than defined by the block_retention setting. For more information about the available settings, refer to these docs.

    Authentication and authorization

    Tempo can be configured in single-tenancy (default) or multi-tenancy mode.

    For single-tenancy, the Jaeger UI route is protected by an oauth-proxy sidecar. By default, any user or service account with permissions to list pods in the namespace of the Tempo instance, can access Jaeger UI. To change this, you can set the Subject Access Review (sar) option of the spec.jaegerui.authentication.sar field:

    spec:
      [...]
      jaegerui:
        authentication:
          sar: '{"namespace": "<my_namespace>", "resource": "pods", "verb": "get"}'

    For details on the SAR setting format, please refer to these docs.

    For multi-tenancy, the access rules can be configured via ClusterRole objects. The following example CR deploys a TempoMonolithic instance with multi-tenancy and two tenants, dev and prod:

    apiVersion: tempo.grafana.com/v1alpha1
    kind: TempoMonolithic
    metadata:
      name: multitenant
    spec:
      jaegerui:
        enabled: true
        route:
          enabled: true
      multitenancy:
        enabled: true
        mode: openshift
        authentication:
        - tenantName: dev
          tenantId: 1610b0c3-c509-4592-a256-a1871353dbfa
        - tenantName: prod
          tenantId: 1610b0c3-c509-4592-a256-a1871353dbfb

    The tenantName should be a human-readable identifier of the tenant, and the tenantId must be a unique identifier. Tempo uses the tenantId to distinguish between tenants in the trace storage. If no storage is specified in the TempoMonolithic CR, the operator defaults to in-memory storage.

    To grant users and ServiceAccounts permission to read and write traces, please follow the instructions in the documentation: Configuring Multitenancy. Configuring role-based access control (RBAC) in TempoMonolithic deployments is identical to configuring RBAC in TempoStack deployments.

    Related Posts

    • How to deploy the new Grafana Tempo operator on OpenShift

    • How to monitor 3scale API Management using Prometheus and Grafana

    • Generate automated Grafana metrics dashboards for MicroProfile apps

    • Introduction to microservices observability with Eclipse MicroProfile

    Recent Posts

    • Unleashing multimodal magic with RamaLama

    • Integrate Red Hat AI Inference Server & LangChain in agentic workflows

    • Streamline multi-cloud operations with Ansible and ServiceNow

    • Automate dynamic application security testing with RapiDAST

    • Assessing AI for OpenShift operations: Advanced configurations

    What’s up next?

    Read Operating OpenShift, a practical guide to running and operating OpenShift clusters more efficiently using a site reliability engineering (SRE) approach. Learn best practices and tools that can help reduce the effort of deploying a Kubernetes platform.

    Get the e-book
    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue