Java monitoring for custom targets with Cryostat

By default, Cryostat automatically recognizes discoverable, valid targets with remote Java Management Extensions (JMX) and JMX-RMI-enabled connectivity. Discoverability is based on various mechanisms, including Kubernetes API-based service endpoints and the Java Discovery Protocol (JDP).

With Cryostat 2.0, developers can now create connections by providing custom URLs for known potential Java application targets that are valid but not yet discoverable. You can save custom target definitions so they are available even after the Cryostat instance restarts. This feature is available on the Cryostat web front-end through the Create Target functionality introduced in this article.

Cryostat 2.0 also introduces the new clientlib directory. By adding target application JARs to this directory, you can specify custom connection protocols for any custom targets you create. We'll show you how to add a custom target that uses one such connection protocol.

Note: The Red Hat build of Cryostat 2.0 is now widely available in technology preview. Cryostat 2.0 introduces many new features and improvements, such as automated rules, a better API response JSON format, custom targets, concurrent target JMX connections, WebSocket push notifications, and more. The Red Hat build includes the Cryostat Operator to simplify and automate Cryostat deployment on OpenShift.

Prerequisites and setup

For this demonstration, we will connect to a custom target with a client-defined connection protocol. As a prerequisite, you must have Cryostat installed and running on a Red Hat OpenShift cluster via the Cryostat Operator. See Cryostat.io for installation details.

We'll use a simple WildFly application for our target. First, we need to add the application to the same namespace where our Cryostat instance is running:

oc new-app quay.io/andrewazores/wildfly-demo:v0.0.2 && oc expose svc wildfly-demo --port=8082

Now that the application is running on the cluster, we'll see it alongside the cryostat-operator and the Cryostat instance when we enter oc status.

Note: See the previous article in this series for instructions for configuring Java applications to use Cryostat. See the end of this article for more ways to learn about Cryostat.

Adding the client-defined connection protocol

The application listens on port 9990 using the JMX remote+http protocol, but not JMX-RMI, which is Cryostat's default connection protocol. For the Cryostat instance to recognize and communicate with our target application (that is, accept a custom connection protocol), we must supply it with additional JARs.

Download the WildFly client JAR

For a WildFly application, the JARs can be obtained by downloading the full WildFly release. The version chosen must correspond to the one used to create the application, which in this case is version 23.0.1.Final. So, we need to download the Jakarta EE Full & Web Distribution tarball for this version. Once downloaded, we'll extract the tarball as follows:

tar -xvf ~/Downloads/wildfly-23.0.1.Final.tar.gz

The required jboss-client.jar is located in the extracted wildfly-23.0.1.Final/bin/client/ directory.

Add the client JAR to clientlib

Now that we have the required JAR, we only need to add it to the clientlib directory in the persistent volume mounted to our Cryostat instance. First, we need to get the name of the pod that houses the instance:

oc get pods -l app=cryostat-sample

Now that we know the pod name, we can transfer the JARs using the oc cp command:

oc cp ~/Downloads/wildfly-23.0.1.Final/bin/client/jboss-client.jar ${PROJECT_NAME}/${CRYOSTAT_POD_NAME}:/opt/cryostat.d/clientlib.d

Note: The PROJECT_NAME is the name of the project (i.e., namespace) containing your cluster and CRYOSTAT_POD_NAME is your result from running the previous command.

Restart your Cryostat instance

Because we've added the JARs to the Cryostat classpath, we need to restart the Cryostat instance for the file to be loaded:

oc rollout restart deployment/cryostat-sample -n ${PROJECT_NAME}

Adding the custom target for Cryostat monitoring

Once the pod is back up and running, we can add the WildFly application as a custom target for Cryostat to monitor. First, we need to get the IP address of the pod housing the application.

Get the pod name and IP address

Start by getting the pod name:

oc get pods -l deployment=wildfly-demo

Now, use the pod name to get the IP address:

oc get pod ${WILDFLY_POD_NAME} -o jsonpath='{.status.podIP}' ; echo

Note: WILDFLY_POD_NAME is your result from running the previous command.

Add WildFly as a custom target

Now that the IP address is known, we can go ahead and add the WildFly application as a custom target.

To use Cryostat's web front-end, check oc status and follow the application URL for the cryostat-sample service. When prompted for an authorization token, provide your OpenShift account token, which you can retrieve using oc whoami -t. Figure 1 shows the Cryostat login page.

Figure 1. The Cryostat login page.

Press the + icon beside Target JVM to create the custom target. For your connection URL, enter service:jmx:remote+http://${IP_ADDRESS}:9990, where IP_ADDRESS is your address for the WildFly application pod. For the Alias, we’ll go with “WildFly," as shown in Figure 2.

Figure 2. Creating a custom target definition.

The screen in Figure 3 confirms the target was created.

Figure 3. Confirmation the target was created.

Authenticating with the target JVM

Once the WildFly target is created, we can select it and navigate to Recordings, as shown in Figure 4.

Figure 4. Select the newly-created target.

We will be prompted to authenticate with the target JVM (i.e., the WildFly application). The application is set up to use JMX authentication with the username “admin” and password “password." Figure 5 shows the authentication screen.

Figure 5. Enter your credentials to connect to the new target.

Once we’re all set up, we can create a new flight recording on the WildFly target application, as shown in Figure 6.

Figure 6. Starting a new recording on the custom target.

Figure 7 shows that we were successful.

Figure 7. The recording running on the target.

Conclusion

In this article, you've learned how to define a custom target for Cryostat monitoring using a client-defined protocol. Once you have set up your custom target in Cryostat, you may interact with it in a variety of ways. Visit Cryostat.io and see the following articles for details:

• Introduction to Cryostat: JDK Flight Recorder for containers
• New features in Cryostat 2.0, plus tips for getting started
• Tutorial: Configuring Java applications to use Cryostat