Observability signal correlation for Red Hat OpenShift has now been released as a technology preview and is available as part of the latest Cluster Observability Operator 0.4.0 release. The backend of observability signal correlation is powered by Korrel8r—open source observability project founded within Red Hat—aimed at providing a set of rules for easily correlating metrics, logs, alerts, netflows, and additional observability signals and resources across different data stores.
The output of Korrel8r is displayed in the form of a node graph in a dedicated troubleshooting panel that Red Hat OpenShift users can trigger within the OpenShift web console. The panel is interactive, which allows users—by clicking each node—to be automatically redirected to the relevant web console UI that displays that specific information (e.g., metric, log, pod, etc.).
Back in July, an enhanced developer preview of observability signal correlation was made available to OpenShift users with the Cluster Observability Operator 0.3.0 release. This developer preview specifically included a troubleshooting panel ONLY accessible by navigating to Observe > Alerting UI of the OpenShift web console. Nevertheless, with the latest technology preview release, a series of enhancements have now improved usability and eased data exploration tasks.
As of today, key new functionalities include:
An improved troubleshooting panel that provides OpenShift users with the possibility to select and focus on a specific starting signal.
More visibility into Korrel8r’s queries with the option of selecting the depth.
Enhanced usability as the observability signal correlation’s troubleshooting panel can be triggered from the application launcher menu in the masthead of the OpenShift web console—without needing to access the Observe > Alerting UI.
Disclaimer:
This third functionality (enhanced usability) is tied to the release of OpenShift 4.17. Note that OpenShift 4.16 users can still make use of the troubleshooting panel by triggering it in the Observe>Alerting UI, as described in the enhanced developer preview dedicated blog.
Technology preview: Everything you need to know
With this introduction in mind, let’s deep dive into the details of this technology preview release.
Getting started
To make use of observability signal correlation’s latest functionalities, OpenShift users need to:
- Install the Cluster Observability Operator 0.4.0 version.
- Deploy an OpenShift 4.17 cluster to be able to launch the troubleshooting panel from an application launcher button at the top of the OpenShift web console’s masthead.
- Install OpenShift Network Observability to visualize correlated netflows.
- Install OpenShift Logging to visualize correlated logs.
Note:
The troubleshooting panel displays the observability signals/resources that are installed for your OpenShift cluster. K8s resources, alerts, and metrics are always available as part of the Red Hat OpenShift Container Platform. Other signals are optional.
Focus functionality
With the technology preview release of observability signal correlation, OpenShift users can now make use of a Focus functionality directly in the Troubleshooting side panel. What does it entail?
The default display of observability signal correlation’s output is a neighborhood graph of the resources/signals currently displayed in a dedicated troubleshooting panel (see Figure 1). Specifically, the neighborhood graph starts at the current resource—in this case an alert—and it includes related objects up to 3 steps away from the starting signal.
Nonetheless, the brand new Focus functionality allows users to easily switch the starting signal and gain new perspectives depending on the troubleshooting questions that need to be answered (see Figure 2).
By clicking Pod (displayed in the graph) and then the Focus button positioned on the top left of the panel, OpenShift users are now provided with correlated signals and resources to that specific pod, as shown in Figure 3.
Show query and additional experimental functionalities
A series of experimental functionalities that allow OpenShift users to customize their own correlation experience have been added as part of this technology preview release. Those features are optional and aim at providing a greater flexibility to users. More specifically:
- "Show Query" displays the experimental features.
- "Hide Query" hides the experimental features.
- "Korrel8r query" identifies the starting point for the node graph and gets updated by the Focus functionality which is described above. Note that this query language is experimental and, as such, it is likely to evolve in the future.
- "Neighborhood depth" provides users with the possibility to increase or decrease the depth of correlation to be able to explore a smaller or larger neighborhood.
"Goal class" provides users with the opportunity to opt for a goal directed search instead of a neighborhood search, which is the standard default. A goal directed search shows all available paths from the starting point to the goal signal/resource. Valid goals currently include any K8s resource, alert, metric, and network event, as well as application, infrastructure, and audit logs. Figure 4 depicts the Show Query feature.
Troubleshooting panel: Universal launcher
An additional functionality will be available with the OpenShift 4.17 release: OpenShift users will be able to trigger the observability signal correlation’s troubleshooting panel directly from the masthead of the OpenShift web console, as shown in Figures 5 and 6. This new feature improves usability and accessibility to the correlation tool and the thus the panel, without users having to necessarily access the Observe > Alerting UI to find the dedicated troubleshooting panel hyperlink.
What's next?
We continue to work on enhancements that provide greater flexibility and usability for OpenShift users. We are working on extending support for additional observability signals/resource types. Up next? Traces. We are also planning on consolidating the experimental features described earlier in the article. That is why your feedback is important. You can share your questions and recommendations with us using the Red Hat OpenShift feedback form.
