Speed up Java application startup time with AppCDS

Application startup time for OpenJDK apps can be important for cloud use cases. Start/stop cycles are much more frequent for an application deployed in Kubernetes since configuration changes usually result in a new deployment—a.k.a restart—of the application. Oftentimes, changing something as simple as an environment variable results in redeployments. The time it takes until the application is ready to handle requests matters. In this article, we’ll show how application class data sharing (AppCDS) can help reduce startup time of your Java application.

In our example, we achieve a startup time as low as ~440 milliseconds or ~47% faster than the corresponding startup time without AppCDS. The actual numbers will vary depending on the precise deployment and resource configuration in use as well as on the nature of the application itself. Nevertheless, AppCDS is an OpenJDK feature well worth looking into.

Application class data sharing (AppCDS)

Application class data sharing is an OpenJDK feature introduced in JDK 10. AppCDS is an extension to the JVM's default class data sharing (CDS) technique. In a nutshell, CDS allows you to produce an archive file (as a first step), which can later be used at JVM runtime (in a second step) for class loading of your application. For the first step, we distinguish between statically produced CDS archives and dynamically produced CDS archives. More about this later.

At application runtime, -XX:SharedArchiveFile=<CDS-archive-file>, can be used to point to the AppCDS file. If a CDS archive is available at runtime then the default class loading from bytecode and the associated verification steps are circumvented and the memory-mapped CDS archive is used instead. Therefore, AppCDS can have a significant impact on application startup time depending on the number of loaded classes. However, the difficult part for setting up class data sharing for applications is that the JDK being used at dump time needs to be the exact same as the one being used to deploy (run) the application. Fortunately, the UBI OpenJDK containers can help there.

Static AppCDS dumps

As mentioned previously, generating a shared class data archive can be done statically or dynamically. Both refer to the first step, the generating-the-CDS-archive-step in the using-application-class-data-sharing process. So what’s a static CDS dump? In a nutshell, static CDS dumps is an earlier feature (JDK 10) and can be invoked with -Xshare:dump JVM option. It requires dumping a class list file as a first step (-Xshare:off -XX:DumpLoadedClassList=<classlist.file>), and using that class list file in order to generate the AppCDS archive for the given classes - of the class list file - in a second step (-Xshare:dump -XX:SharedClassListFile=<classlist.file>)

Dynamic AppCDS dumps

In contrast, a dynamic dump will be produced using the -XX:ArchiveClassesAtExit JVM option. It’s a newer JVM feature introduced in JDK 13 so as to improve usability of AppCDS. While dynamic dumps don’t require the class list dumping step anymore, the resulting application class data archive file produced with a dynamic dump depends on the presence of the default CDS archive file of the JDK (typically 'classes.jsa' or 'classes_nocoops.jsa' files part of your JDK installation). Failing to use the default CDS archive file of your JDK will render the application CDS dump unusable, too.

Dynamic CDS archives include a checksum of the default CDS archive from the JDK when the dynamic dump files get produced. In contrast, a static CDS dump still works at runtime when the JDK itself doesn't include the default CDS archive in its installation, but is otherwise the same as was used at CDS dump time.

Quarkus and application class data sharing

Quarkus gained a nice feature which automates generating the shared classes archive for your Quarkus application. For the purpose of this blog post, we combine this Quarkus feature with the UBI 9 OpenJDK 17 image and a little bit of configuration (templates) in OpenShift in order to make this otherwise fairly tricky deployment option a real breeze. Let’s get right into an example of doing so. We deploy the Quarkus getting-started json sample application (see Figure 1) to Red Hat OpenShift with AppCDS turned on.

In our example we use the Source-to-image (S2I) build strategy with some custom configuration so as to 1) build the application using Quarkus’ uber-jar packaging type, 2) define some JVM parameters which need to match during build time and runtime, and 3) set the relevant options for turning on AppCDS at runtime.

Let’s look at the configuration in detail:

MAVEN_S2I_ARTIFACT_DIRS=target
MAVEN_OPTS=-XX:+UseCompressedOops -XX:+UseCompressedClassPointers
MAVEN_ARGS=-Dquarkus.package.create-appcds=true -Dquarkus.package.type=uber-jar package
S2I_SOURCE_DEPLOYMENTS_FILTER=app-cds.jsa rest-json-quickstart*runner.jar
JAVA_OPTS_APPEND=-XX:+UseCompressedClassPointers -XX:+UseCompressedOops -Xshare:on -XX:SharedArchiveFile=/deployments/app-cds.jsa -Dquarkus.http.host=0.0.0.0
AB_JOLOKIA_OFF=true
JAVA_APP_JAR=/deployments/rest-json-quickstart-1.0.0-SNAPSHOT-runner.jar

The important configuration settings are MAVEN_OPTS, MAVEN_ARGS, S2I_SOURCE_DEPLOYMENTS_FILTER, JAVA_OPTS_APPEND, and JAVA_APP_JAR. See the OpenJDK image documentation for the full details.

• MAVEN_OPTS has been specified, so as to turn compressed OOPs and compressed class pointers on at build time, since those settings need to match the runtime config (see JAVA_OPTS_APPEND) for AppCDS to work.
• S2I_SOURCE_DEPLOYMENTS_FILTER tells the OpenJDK builder image which artifacts to copy to the assembled image.
• MAVEN_ARGS specifies some additional parameters to use for the maven build.
• We use -Dquarkus.package.create-appcds=true so as to tell Quarkus to generate app-cds.jsa for us and to use uber-jar packaging.
• JAVA_APP_JAR is being used to tell the image which jar should be used to run the application.
• JAVA_OPTS_APPEND is being used to turn on AppCDS and fail the deployment if AppCDS cannot be used (-Xshare:on -XX:SharedArchiveFile=/deployments/app-cds.jsa) as well as to turn on compressed oops and compressed class pointers.

Note that which application CDS dump is being produced by Quarkus depends on the Java bytecode version of your application. For example, if -Dmaven.compiler.source=11 and -Dmaven.compiler.target=11 is being used, then a static CDS dump is being performed. If -Dmaven.compiler.source=17 and -Dmaven.compiler.target=17 is being used, a dynamic CDS dump is being performed. The reason for this is that for JDK 11, the dynamic CDS dumps feature did not yet exist. Please see the previous section as to what the difference between the two is. In our example, we have that fixed to JDK 11 source/target level since we wanted to create a static CDS dump file. In our experiments that yielded the best results in terms of startup time for this particular application. This might vary for your application.

Finally, we have created some OpenShift templating so as to be able to continuously deploy the sample application from source to the UBI 9 OpenJDK 17 runtime image with a click of a button and AppCDS turned on. Automation can trigger updated builds on a push to the source code repository as well. We leave that exercise to the reader, though.

Application class data sharing can make a real difference when it comes to application startup time. When comparing the effects of class data sharing, there are basically three cases to consider when running on JDK 17. 1: the base case, with -Xshare:off, which disables the default JDK-provided CDS archive as well as AppCDS; 2) the common case with default JDK-provided CDS archive enabled and AppCDS turned off; and 3) the demo case of this article: default JDK-provided CDS archive enabled and AppCDS turned on as well.

In order to give some context to the following numbers let's consider our application setup in OpenShift. The rest-json-quickstart application is being deployed with a limit of 2 CPU cores and 256MB memory (guaranteed K8s QoS class). This is a really small JVM setup. Deliberately so, since the benefit of AppCDS shows up in resource-constrained environments most prominently. Specifically, class loading at application startup can attribute to a significant amount of CPU time.

Also note that this example uses the uber-jar packaging type of Quarkus. In our experiments, using a static AppCDS dump yielded the best results in terms of speed-up at application start. Using a dynamic AppCDS dump was slightly slower than using a static AppCDS dump with JDK 17. Since the default Quarkus packaging type is fast-jar, which leverages a custom Quarkus classloader, it didn’t lend itself as nicely for AppCDS and a static AppCDS dump due to JDK-8265602. Our workaround was to use uber-jar instead so as to avoid the custom Quarkus classloader.

Base case (-Xshare:off)

This case is using environment variable JAVA_OPTS_APPEND with value -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -Xshare:off -Dquarkus.http.host=0.0.0.0

Starting the Java application using /opt/jboss/container/java/run/run-java.sh ...
INFO exec -a "java" java -XX:MaxRAMPercentage=80.0 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -Xshare:off -Dquarkus.http.host=0.0.0.0 -cp "." -jar /deployments/rest-json-quickstart-1.0.0-SNAPSHOT-runner.jar
INFO running in /deployments
__  ____  __  _____   ___  __ ____  ______
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2023-10-30 14:48:33,888 INFO  [io.quarkus] (main) rest-json-quickstart 1.0.0-SNAPSHOT on JVM (powered by Quarkus 3.4.3) started in 0.939s. Listening on: http://0.0.0.0:8080
2023-10-30 14:48:33,890 INFO  [io.quarkus] (main) Profile prod activated.
2023-10-30 14:48:33,890 INFO  [io.quarkus] (main) Installed features: [cdi, resteasy-reactive, resteasy-reactive-jackson, smallrye-context-propagation, vertx]
2023-10-30 14:48:34,288 INFO  [org.acm.res.jso.LoggingFilter] (executor-thread-1) Request GET /fruits from IP 10.129.2.1:39416

Default JDK-CDS archive only (no application CDS)

This case is using environment variable JAVA_OPTS_APPEND with value -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -Xshare:on -Dquarkus.http.host=0.0.0.0

Starting the Java application using /opt/jboss/container/java/run/run-java.sh ...
INFO exec -a "java" java -XX:MaxRAMPercentage=80.0 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -Xshare:on -Dquarkus.http.host=0.0.0.0 -cp "." -jar /deployments/rest-json-quickstart-1.0.0-SNAPSHOT-runner.jar
INFO running in /deployments
__  ____  __  _____   ___  __ ____  ______
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2023-10-30 14:39:44,069 INFO  [io.quarkus] (main) rest-json-quickstart 1.0.0-SNAPSHOT on JVM (powered by Quarkus 3.4.3) started in 0.869s. Listening on: http://0.0.0.0:8080
2023-10-30 14:39:44,071 INFO  [io.quarkus] (main) Profile prod activated.
2023-10-30 14:39:44,071 INFO  [io.quarkus] (main) Installed features: [cdi, resteasy-reactive, resteasy-reactive-jackson, smallrye-context-propagation, vertx]
2023-10-30 14:39:44,937 INFO  [org.acm.res.jso.LoggingFilter] (executor-thread-1) Request GET /fruits from IP 10.129.2.1:40286

AppCDS and default JDK CDS archive

This case is using environment variable JAVA_OPTS_APPEND with value -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -Xshare:on -XX:SharedArchiveFile=/deployments/app-cds.jsa -Dquarkus.http.host=0.0.0.0

Starting the Java application using /opt/jboss/container/java/run/run-java.sh ...
INFO exec -a "java" java -XX:MaxRAMPercentage=80.0 -XX:+UseParallelGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -Xshare:on -XX:SharedArchiveFile=/deployments/app-cds.jsa -Dquarkus.http.host=0.0.0.0 -cp "." -jar /deployments/rest-json-quickstart-1.0.0-SNAPSHOT-runner.jar
INFO running in /deployments
__  ____  __  _____   ___  __ ____  ______
 --/ __ \/ / / / _ | / _ \/ //_/ / / / __/
 -/ /_/ / /_/ / __ |/ , _/ ,< / /_/ /\ \   
--\___\_\____/_/ |_/_/|_/_/|_|\____/___/   
2023-10-30 14:36:47,595 INFO  [io.quarkus] (main) rest-json-quickstart 1.0.0-SNAPSHOT on JVM (powered by Quarkus 3.4.3) started in 0.497s. Listening on: http://0.0.0.0:8080
2023-10-30 14:36:47,596 INFO  [io.quarkus] (main) Profile prod activated.
2023-10-30 14:36:47,596 INFO  [io.quarkus] (main) Installed features: [cdi, resteasy-reactive, resteasy-reactive-jackson, smallrye-context-propagation, vertx]
2023-10-30 14:36:48,472 INFO  [org.acm.res.jso.LoggingFilter] (executor-thread-1) Request GET /fruits from IP 10.129.2.1:39794

Startup time summary

For the three cases under investigation above, we have the following startup times:

This means in the simple sample Quarkus application case, adding AppCDS to the application brought the application bootup time as determined by Quarkus itself from about 939 milliseconds down to about 497 milliseconds, which amounts to a speed-up of about 1.88 or -  in other words - AppCDS has reduced the startup time by about 47 percent. Nice!

Debugging class data sharing issues

When debugging Class Data Sharing (CDS) issues of OpenJDK applications the -Xlog:cds=info switch as well as the -XX:+PrintSharedArchiveAndExit options are useful. The former shows basic mapping information of which CDS archive is being mapped, gives some clues as to which AppCDS archive is being used (static or dynamic) and -XX:+PrintSharedArchiveAndExit in combination with -Xshare:on will provide you with some details as to why the mapping of a CDS archive fails. The option -Xshare:on is useful in order to make the JVM boot-up fail if CDS sharing isn’t possible. Note that the OpenJDK default, -Xshare:auto, is to only print a warning and continue without CDS and/or AppCDS should the mapping of the class data sharing archive fail.

Conclusion

In this article, we have seen that adding application class data sharing can boost your OpenJDK application’s startup time. What’s more, using UBI 9 OpenJDK 17 images, setting this otherwise pretty tricky process up in OpenShift can be well automated. If you want to recreate the setup yourself, you can do so by the following steps:

git clone https://github.com/jerboaa/quarkus-quickstarts.git
cd quarkus-quickstarts
git checkout -b quickstart-3.4-s2i-cds origin/quickstart-3.4-s2i-cds
cd rest-json-quickstart
oc new-project quarkus-test
oc process -p NAMESPACE=quarkus-test \
   -f openshift/quarkus_runtime_appcds_template.yaml | oc create -f -

If you want to deploy using a dynamic AppCDS archive and fast-jar packaging type instead, you can do that by using the quickstart-3.4-s2i-cds-fastjar branch as follows:

git clone https://github.com/jerboaa/quarkus-quickstarts.git
cd quarkus-quickstarts
git checkout -b quickstart-3.4-s2i-cds-fastjar origin/quickstart-3.4-s2i-cds-fastjar
cd rest-json-quickstart
oc new-project quarkus-test-fastjar
oc process -p NAMESPACE=quarkus-test-fastjar \
   -f openshift/quarkus_runtime_appcds_template.yaml | oc create -f -

Note that due to an OpenShift builder bug, it’s recommended to use OpenShift Container Platform 4.14.2 or above to test and/or deploy such a setup. It’s also worth mentioning that once resources get imported with the template both builds, bc/quarkus-rest-json-appcds-s2i and bc/quarkus-rest-json-appcds-runtime need to be finished before the deployment deployment/quarkus-rest-json-appcds can be successful. This is the deployment using application class data sharing as described in this article.

oc process -p NAMESPACE=quarkus-test-fastjar \
    -p JAVA_OPTS_APPEND="-XX:+UseCompressedClassPointers -XX:+UseCompressedOops -Xshare:off -Dquarkus.http.host=0.0.0.0" \
    -f openshift/quarkus_runtime_appcds_template.yaml | oc create -f -

You can view the logs of the deployment once ready with oc logs deployment/quarkus-rest-json-appcds | head. The deployed application can be reached with the exposed service URL as printed by the following command: echo http://$(oc get route/quarkus-rest-json-appcds -o jsonpath='{.spec.host}’)/fruits.html

Onward to your next application data sharing OpenJDK deployment! Enjoy!