Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat OpenShift AI
      Red Hat OpenShift AI
    • Red Hat Enterprise Linux AI
      Linux icon inside of a brain
    • Image mode for Red Hat Enterprise Linux
      RHEL image mode
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • Red Hat Developer Hub
      Developer Hub
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Image mode for Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of OpenJDK
    • Kubernetes

      • Red Hat OpenShift
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Virtualization
      • Red Hat OpenShift Lightspeed
    • Integration & App Connectivity

      • Red Hat Build of Apache Camel
      • Red Hat Service Interconnect
      • Red Hat Connectivity Link
    • AI/ML

      • Red Hat OpenShift AI
      • Red Hat Enterprise Linux AI
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Lightspeed
    • Developer tools

      • Red Hat Trusted Software Supply Chain
      • Podman Desktop
      • Red Hat OpenShift Dev Spaces
    • Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • API Catalog
    • Product Documentation
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

Automating JDK Flight Recorder in containers

November 9, 2021
Andrew Azores
Related topics:
ContainersJavaKubernetes
Related products:
Red Hat OpenShift

Share:

    This article is part of a series of hands-on guides to using Cryostat 2.0, or JDK Flight Recorder for containers. This article introduces Cryostat's new API for automated rules. We'll walk through two use cases highlighting the API's compact but powerful rule definitions. You'll see how to use rule definitions to specify a match expression for one or more target Java applications, and how to configure the type of flight recording you want to start on these targets.

    Once you've created a rule, Cryostat immediately matches it against all existing discovered targets and starts your flight recording. Cryostat will also apply the rule to newly discovered targets that match its definition. You can create multiple rules to match different subsets of targets or to layer different recording options for your needs.

    The automated rules API is brand new in Cryostat 2.0, and we haven't yet developed the user interface (UI) for it. For now, we'll use curl to interact with the Cryostat HTTP API directly.

    Read all of the articles in this series:

    • Part 1: Get started with Cryostat 2.0
    • Part 2: Configuring Java applications to use Cryostat
    • Part 3: Java monitoring for custom targets with Cryostat
    • Part 4: Automating JDK Flight Recorder in containers
    • Part 5: Creating Custom JFR event templates with Cryostat 2.0

    Note: The Red Hat build of Cryostat 2.0 is now widely available in technology preview. Cryostat 2.0 introduces many new features and improvements, such as automated rules, a better API response JSON format, custom targets, concurrent target JMX connections, WebSocket push notifications, and more. The Red Hat build includes the Cryostat Operator to simplify and automate Cryostat deployment on OpenShift.

    Use case 1: Continuous monitoring in a containerized JVM

    Previously, if we wanted to enable always-on continuous monitoring using JDK Flight Recorder (JFR) in a containerized Java virtual machine (JVM), we would set JVM flags on the target application, then restart the application to start monitoring. With Cryostat's automated rules, we can enable JDK Flight Recorder at runtime to continuously monitor an already-running target application, with no restart, no redeploy, and no downtime.

    We can also adjust the continuous monitoring event template at runtime, with no downtime. First, we create a rule with the template we want to use. Then, we create a new template and a new rule with the new or updated event template. Finally, we delete the original rule with the clean=true parameter. Here's the source:

    
    $ export CRYOSTAT=https://cryostat.example.com # replace this URL with your actual Cryostat instance URL
    
    $ curl -F name="firstRule" -F matchExpression=”target.alias==‘com.example.MainClass’” -F eventSpecifier=”template=Profiling,type=TARGET” $CRYOSTAT/api/v2/rules # Create a Rule named firstRule using the Profiling template
    
    $ curl $CRYOSTAT/api/v1/targets/$serviceUri/templates/Profiling/type/TARGET -o profiling.jfc # Download the Profiling template to your computer
    
    $ $EDITOR profiling.jfc # Edit the Profiling template to update the event definitions inside, and to rename it from Profiling to Demo
    
    $ mv profiling.jfc demo.jfc # Rename the file to demo.jfc so it's more identifiable as the Demo profile
    
    $ curl -F template=@demo.jfc $CRYOSTAT/api/v1/templates # Upload the new Demo profile to Cryostat
    
    $ curl -F name="secondRule" -F matchExpression=”target.alias==‘com.example.MainClass’” -F eventSpecifier=”template=Demo,type=CUSTOM” $CRYOSTAT/api/v2/rules # Create a second Rule using the Demo profile
    
    $ curl -X DELETE $CRYOSTAT/api/v2/rules/firstRule?clean=true # Delete the first Rule that used the Profiling template and clean up any active recordings it created
    
    

    Use case 2: Custom monitoring with Kubernetes labels or annotations

    We can define a rule that applies to any target application that has platform-specific attributes, such as Kubernetes labels or annotations. Here's an example in JSON notation:

    
    {
      "name": "k8sMonitoring",
      "description": "Enable the Demo template on any target with the jfrMonitoring=true annotation",
      "matchExpression": "target.annotations.platform[‘jfrMonitoring’]==’enabled’",
      "eventSpecifier": "template=Demo,type=CUSTOM",
      "archivalPeriodSeconds": 300,
      "preservedArchives": 12
    }
    
    

    Once we've created this rule definition, Cryostat will check all of the existing target applications and watch for new targets that appear with the jfrMonitoring=enabled annotation. Any matching targets found will have a recording started automatically using the custom Demo template from our first use case. It will take an archived snapshot every five minutes and maintain an hour’s worth of these archives in storage.

    With this rule definition in place, Kubernetes or Red Hat OpenShift users can use familiar tools like kubectl/oc or the OpenShift console to mark target applications for monitoring, without needing to interact directly with the Cryostat API or UI. This opens the door to further automating your workflow.

    As an example, you might use or implement an Operator that monitors traffic flow or pod restarts and enables monitoring on pods after some criterion threshold is met, then disables it again if the target application's behavior returns to normal. As a Kubernetes administrator, you could receive a notification when this occurs and check the Cryostat archives to retrieve JDK Flight Recorder data from the target application recorded during the problematic period, or you could view these archived recordings in Cryostat’s Grafana dashboard.

    Note: An important caveat is that Cryostat does not watch for changes in the Kubernetes annotations or labels; it only watches to see if target applications appear or disappear. To apply the annotation to a target application, we must apply the annotation or label to the application pod (which will cause Kubernetes to roll out a new replica), and not to the deployment.

    How to use match expressions in Cryostat

    The matchExpression in a rule definition is a Java-like snippet of code that Cryostat interprets and uses to determine if a rule should be applied to any given target.  matchExpressions should thus evaluate to a boolean value. The simplest matchExpressions would be the booleans true or false; if we use true, the rule will apply to every target. The expression has a target object in global scope, with the following form in JSON notation:

    
    {
      “alias”: “myAppAlias”,
      “connectUrl”: “service:jmx:rmi:///jndi/rmi://cryostat:9091/jmxrmi”,
      “labels”: {
        “com.example/service”: “customer-login”,
      },
      “annotations”: {
        “platform”: {
          “io.kubernetes/annotation”: “annotated”
        },
        “cryostat”: {
          “PORT”: 9091,
          “HOST”: “cryostat”,
          “NAMESPACE”: “myproject”
        }
      }
    }
    
    

    The alias, connectUrl, labels, annotations.platform, and annotations.cryostat properties are all guaranteed to be present on the target object. alias and connectUrl will be non-empty strings. The labels and platform annotations may be empty—in OpenShift or Kubernetes, these are populated from the labels and annotations applied to the target’s pod, if any. The Cryostat annotations map will vary per platform, but on OpenShift or Kubernetes you can expect the HOST, PORT, NAMESPACE, and POD_NAME keys to be present and non-empty.

    Here are some examples of matchExpressions:

    target.alias == ’com.example.MainClass’
    
    target.alias == ’myAlias’
    
    target.labels[‘com.example/service’] == ’customer-login’
    
    target.labels[‘com.example/service’] != ’customer-login’
    
    target.annotations.cryostat.PORT > 3000
    
    target.annotations.cryostat.PORT > 3000 && target.annotations.platform[‘io.kubernetes/annotation’] == ‘enabled’
    
    !!target.annotations.platform[‘io.kubernetes/annotation’]
    
    /^customer-login[0-9]*$/.test(target.alias)
    

    Conclusion

    In this article, you've learned how to use match expressions to create automated rules for Cryostat monitoring. Once you've set up your automated rules, Cryostat will continuously monitor applications that meet the criteria defined in those rules, with no need to restart or redeploy those applications. Visit Cryostat.io for further details.

    Last updated: September 20, 2023

    Related Posts

    • Introduction to Cryostat: JDK Flight Recorder for containers

    • Announcing Cryostat 2.0: JDK Flight Recorder for containers

    • Configuring Java applications to use Cryostat

    • Java monitoring for custom targets with Cryostat

    Recent Posts

    • AI meets containers: My first step into Podman AI Lab

    • Live migrating VMs with OpenShift Virtualization

    • Storage considerations for OpenShift Virtualization

    • Upgrade from OpenShift Service Mesh 2.6 to 3.0 with Kiali

    • EE Builder with Ansible Automation Platform on OpenShift

    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit
    © 2025 Red Hat

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue