Skip to main content
Redhat Developers  Logo
  • Products

    Platforms

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat AI
      Red Hat AI
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • View All Red Hat Products

    Featured

    • Red Hat build of OpenJDK
    • Red Hat Developer Hub
    • Red Hat JBoss Enterprise Application Platform
    • Red Hat OpenShift Dev Spaces
    • Red Hat OpenShift Local
    • Red Hat Developer Sandbox

      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Secure Development & Architectures

      • Security
      • Secure coding
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • Product Documentation
    • API Catalog
    • Legacy Documentation
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

Integrate incident detection with OpenShift Lightspeed via MCP

October 9, 2025
Alberto Falossi Tomas Remes
Related topics:
Artificial intelligenceObservability
Related products:
Red Hat OpenShift Container PlatformRed Hat OpenShift Lightspeed

Share:

    Incident detection in Red Hat OpenShift is now integrated with OpenShift Lightspeed, the AI-powered virtual assistant for OpenShift. This brings incident analysis directly into the conversational interface, which changes how you explore and resolve cluster issues.

    As part of the Cluster Observability Operator (COO), incident detection simplifies observability by grouping related alerts into incidents. This helps reduce alert fatigue and allows you to focus on the root cause of a problem.

    With incident data available in Lightspeed, you can move beyond static views and interact with your cluster in natural language. You can ask whether problems are related, drill into symptoms, or request the event chains behind issues, as shown in Figure 1.

    Root-cause analysis with OpenShift Lightspeed and incident detection
    Figure 1: Querying OpenShift Lightspeed about the root cause of an ingress controller being down.

    Integration with OpenShift Lightspeed is achieved via the Model Context Protocol (MCP), as the incident detection engine is exposed through an MCP server. MCP is an open source standard for connecting large language model (LLM) applications with external systems.

    Installing the Cluster Health MCP server

    Follow these steps to install the Cluster Health MCP server and configure OpenShift Lightspeed to use it.

    Note

    The feature is developer preview only. Consult the developer preview support statement to learn more.

    Prerequisites:

    • A cluster running OpenShift 4.19
    • An OpenAI (or other LLM provider) API key
    • The OpenShift CLI (oc)
    1. Install Cluster Observability Operator 1.2.2+ from OperatorHub using the Red Hat OpenShift Container Platform web console.
      1. Make sure to select Enable Operator recommended cluster monitoring on this Namespace during the Cluster Observability Operator installation (Figure 2), or incident detection won't work properly.

        Select “Enable Operator recommended cluster monitoring on this Namespace”
        Figure 2: Select the option to enable Operator recommended cluster monitoring.
    2. Enable the incident detection feature by following the instructions in the previous blog post.
    3. Install the developer preview of the Cluster Health MCP server:

      oc apply -f https://raw.githubusercontent.com/openshift/cluster-health-analyzer/refs/heads/mcp-dev-preview/manifests/mcp/01_service_account.yaml
      oc apply -f https://raw.githubusercontent.com/openshift/cluster-health-analyzer/refs/heads/mcp-dev-preview/manifests/mcp/02_deployment.yaml
      oc apply -f https://raw.githubusercontent.com/openshift/cluster-health-analyzer/refs/heads/mcp-dev-preview/manifests/mcp/03_mcp_service.yaml
    4. Install OpenShift Lightspeed 1.0.5+ from OperatorHub.
    5. Store your OpenAI (or other LLM provider) API key in a secret:

      oc create secret generic -n openshift-lightspeed credentials --from-literal=apitoken=<YOUR_API_KEY>

      An API token is required to interact with the LLM, and it must be available as a secret in the cluster.

    6. Configure OpenShift Lightspeed with the LLM provider and add the Cluster Health MCP server. You can follow the official documentation or use the command and following YAML, preconfigured for the OpenAI provider:

      oc apply -f - <<EOF
      apiVersion: ols.openshift.io/v1alpha1
      kind: OLSConfig
      metadata:
        name: cluster
      spec:
        featureGates:
        - MCPServer
        llm:
          providers:
          - name: myOpenai
            type: openai
            credentialsSecretRef:
              name: credentials
            url: https://api.openai.com/v1
            models:
            - name: gpt-4.1-mini
        mcpServers:
        - name: cluster-health
          streamableHTTP:
            enableSSE: false
            sseReadTimeout: 10
            timeout: 5
            url: 'http://cluster-health-mcp-server.openshift-cluster-observability-operator.svc.cluster.local:8085/mcp'
        ols:
          defaultModel: gpt-4.1-mini
          defaultProvider: myOpenai
      EOF

      Note we are using the OpenAI provider here, but you are free to choose others.

    7. The configuration takes a few minutes to apply. If you open the chat before it is complete, you will see the warning message Waiting for OpenShift Lightspeed service.

    Incident analysis in the Lightspeed chat

    Once the Cluster Health MCP server is installed, incident data will be included in your chat context whenever it's relevant to the conversation. The Cluster Health MCP tools are typically triggered by phrases semantically related to "cluster health," "alerts," "issues," and, of course, "incidents."

    For instance, you could ask:

    Are there incidents firing?

    Or variations of these questions. If any incidents are currently firing, the response will include a report (Figure 3).

    Incident report
    Figure 3: OpenShift Lightspeed summarizes 3 critical incidents currently firing.

    Note that when the MCP tool is invoked, a get_incidents label appears below the response. If you don't see this label, it means the LLM decided not to call the tool for some reason. Remember that LLMs are not deterministic. They decide when to call MCP tool functions based on several factors, such as the model version, available MCP servers, the current context, and the conversation history. To ensure incident data is included, try to add the word "incident" in your question.

    LLMs may also use incident data indirectly, even in prompts that don't explicitly ask for it. For example, the following prompt could use the incident data to generate a report:

    I'm seeing KubeAPITerminatedRequests in the alerts. Analyze the incident and tell me what's happening

    Another use case is to start with a symptom and ask the LLM to find the root cause:

    Why is [component/service/...] down?

    When an incident is mentioned in a conversation, you can request its details directly from the chat (for example, "explain incident X", as illustrated in Figure 4) or by consulting the Observe → Incidents page. A direct link to the incident page is sometimes embedded in the conversation, but it depends on the language model being used.

    Getting details about an incident
    Figure 4: Requesting details about a given incident.

    Security considerations

    The Cluster Health MCP server is a read-only component and can only access data the currently logged-in user can see in the OpenShift web console. If the user lacks sufficient permissions to query the in-cluster Prometheus and Alertmanager, OpenShift Lightspeed will not be able to access incident data.

    It's important to note that the Cluster Health MCP is designed for exclusive use with OpenShift Lightspeed. It cannot be used with generic MCP clients because it requires a custom authentication header. This might change in the future, as authentication standards for MCP servers are still evolving and aren't yet fully defined.

    What's next

    We are actively working to enhance the quality of AI-driven responses and plan to incorporate a wider range of signals to provide even richer incident analysis. As the standards for LLMs and MCPs continue to evolve, so will this tool.

    As part of this developer preview release, you have the opportunity to get hands-on experience right away. We encourage you to explore its capabilities and see how it fits into your workflow. Your feedback is crucial to this process, and we invite you to share your ideas, questions, and recommendations through the Red Hat OpenShift feedback form.

    Related Posts

    • Incident detection for OpenShift tech preview is here

    • How incident detection simplifies OpenShift observability

    • Supercharge your Red Hat OpenShift local environment with Red Hat OpenShift Lightspeed

    • Kubernetes MCP server: AI-powered cluster management

    • How to deploy MCP servers on OpenShift using ToolHive

    • How to build a simple agentic AI server with MCP

    Recent Posts

    • Integrate incident detection with OpenShift Lightspeed via MCP

    • One model is not enough, too many models is hard: Technical deep dive

    • What's new in Ansible Automation Platform 2.6

    • Quantum computing 101 for developers

    • LLM Compressor 0.8.0: Extended support for Qwen3 and more

    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Platforms

    • Red Hat AI
    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform
    • See all products

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit
    © 2025 Red Hat

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue