[{"content":"In Part 1 of this series of 3 blog posts we introduced the specific performance challenges OpenJDK faces lowering application \u2018startup\u2019, \u2018warmup\u2019 and \u2018initial footprint\u2019 costs and provided an overview of what Leyden is doing to address those challenges.\nPart 2 described how to use the new capabilities offered by Leyden and presented test results which show that very significant progress has already been made and is set to continue.\nPart 3 provides a more detailed account of how Leyden\u2019s proposed solution operates and presents a first look at tooling that allows you to assess the benefits that result and tune your application to make the most of what Leyden offers.\nWhat is inside the Ahead of Time Cache? Ideally, an AOT cache would simply include everything needed to allow a production run to skip straight through to its warmed up state. However, in practice training runs don\u2019t always cover all the things that can happen at runtime and hence that the assets contained in any generated AOT cache will be more or less complete.\nIn order to have some idea of how effective a training run has been it\u2019s helpful to be able to look at a cache and see what is in it. Full details of the tooling that allows you to do that are presented in part 3 of this blog series. However, in order to prepare for that, we need to provide an overview of the JVM assets that end up in the cache and how the JVM uses them. We will follow up with some examples to show how effectively this improves startup and warmup.\nSo, let\u2019s take a deeper look at what exactly is inside the AOT Cache. There are several different ways of classifying the contents:\nThe most straightforward way to classify AOT cache assets is to distinguish between Static and Dynamic data.\nStatic assets are data that are available in or directly derived from bytecode, data that exist, even if only implicitly, at build time.\nDynamic assets are data that get generated, or are collected, at runtime as a side-effect of execution. Some of them record information that can be used to trigger compilation and drive feedback-driven optimizations, including speculative optimization, beyond what an ahead-of-time compiler would be able to do. They can also include the compiled code that is generated as a result of that compilation.\nFinally, they include training data, created as a training run progresses to track what the JVM has done and why. Training data identify what JVM assets need to be stored into the cache when it is created. They are also installed in the cache, indexing the other assets and helping identify how to use them in production.\nWe can also distinguish two types of data depending on their purpose:\nOn one hand, there is the JVM data \u2014 metadata, heap data and code. This is a network of C++ objects that are used during normal JVM running to define and regulate Java execution. These objects must always exist, even when running without a cache, in order for the JVM to be able to run an app. This object network needs to be dumped to the archive on disk in a format that allows it to be quickly and correctly restored to the relevant memory areas of the production JVM in a valid (C++ Object) format and layout that matches the JVM\u2019s expectations.\nOn the other hand we also have Leyden\u2019s own Cache Management Data, i.e. training data, which exists specifically to support creation and consumption of AOT cache. Training data are also saved and restored as C++ object data but the format and layout of these objects is determined solely by the Leyden cache management code. Its sole purpose is to track and regulate what assets get written to the AOT cache after training completes and what assets can or should be restored in production.\nLet\u2019s see in detail what each data type means.\nJVM Metadata Metadata stored in the AOT cache is a superset of what was stored in a CDS archive. The subset which overlaps with CDS is the static metadata. The latter represents the structure and hierarchy of classes in JDK and application code. Primarily, it helps avoid the cost of parsing bytecode, as it is in the same format as the JVM\u2019s own internal metadata model: classes, methods, fields, inheritance between classes,… which can be mapped directly into memory. Having this information stored in the cache speeds up the time the Hotspot takes to decode the different class files, and to build the dependency graph.\nWhile starting the application, the Java Heap memory gets filled with objects and instances that are going to be used during runtime. Some of those heap data objects can be cached too because they are quite predictable, like Strings hardcoded in the source code, java.lang.Class instances, some content of class static fields, objects needed to run lambdas, the class graph module,… Those are all assets that are created in memory in the same way on every run.\nThe heap data cached at the moment is restricted to very specific cases as it has to behave exactly the same on each and every run, but the type of data cached is expanding on each JDK version.\nJVM Profile and Linkage Data The cache also includes dynamic JVM metadata i.e. MethodCounter, MethodData and ConstantPoolCache objects. These objects are created and attached to the static metadata methods and classes and their content is updated as a side-effect of executing method code.\nMethodCounter objects track how often the method they are attached to has been called. They are primarily used to trigger compilation via the baseline (C1) or optimizing (C2) compiler. The interpreter increments a method\u2019s call count up to a threshold before scheduling a C1 compile, possibly including code that gathers further profile information. Instrumentation code in (Tier 2 andImage description 3) C1 compiled methods also updates the call count and when a higher threshold is reached either upgrades to (Tier 4) C2 compiled code or reverts to (Tier 1) C1 compiled code which includes no instrumentation\nMethodProfile objects record detailed information about their associated method\u2019s hot and cold paths, argument types and other details of how it executes, most notably any history of speculative deoptimization. Apart from the deoptimization case, which applies for both C1 and C2 code, MethodProfile objects only receive updates via instrumentation code in (Tier 2 and 3) C1 compiled methods.\nConstantPoolCache objects are attached to a clasImage descriptions and track the linkage of call and field access sites in any of the class\u2019s methods. Prelinking avoids work at the first call or first field access and this is especially valuable when the call is an invokedynamic i.e. the bytecode that implements a lambda invocation.\nLinking a lambda involves running Java \u2018bootstrap\u2019 code that identifies a private class that owns the bytecode for the lambda body, asking it to construct and return a MethodHandle that can be used to execute the target. If a lambda can be run during training then the target class and method can be pre-loaded and the MethodHandle stored in the heap and linked from the ConstantPoolCache, avoiding the need to run the \u2018bootstrap\u2019 in production. If the lambda is executed repeatedly in production the called bytecode may even be inlined into the compiled code for the caller. Effectively, executing as lambda in training removes all setup overheads in production, making lambdas as cheap to use as a direct method call.\nJVM Code and Code Management Data AdapterHandlers are a set of utilities used by the Hotspot to marshall method parameters when performing certain types of call. AdapterHandlers can be cached, avoiding the need to generate them on demand. They are identified by their AdapterFingerprint and indexed via a table of AdapterHandleEntry objects.\nAlongside these handlers various StubBlobs needed by the runtime are also cached. These blobs contain JITted code that implements one or more \u2018stub\u2019 routines. Stub routines include architecture- and OS-specific code used by the JVM to perform operations that are hard to write in a platform agnostic way. Examples include: flushing code regions after update by the JIT or call linker, unwinding the stack when an exception occurs, replacing a compiled stackframe with one or more interpreter frames when execution of a deopt trap forces a bail-out etc. There are also many stubs that provide hand crafted, high-performance implementations of math, crypto or memory copy methods that are used in place of Java implementations on some architectures, especially where hand-crafted code can use specialized hardware instructions to outperform the JIT compiler. Much of the stub and adapter code has to be generated before the JDK can fully startup. Storing it in the cache and reloading it in production provides a small but noticeable performance improvement.\nLeyden premain also includes CompiledMethods, i.e. pre-compiled Java methods, in the cache. This includes both C1 and C2 (Tiers 1 - 4) and in some cases different tier compiled versions of the same method. Having compiled code immediately available, especially Tier 4 code, is an enormous boost to performance. Lower tier code may be useful when the method only reached that tier during training or as a fallback if we need to deoptimize and reprofile. Pre-compiled Java methods are an enhancement we expect to add soon to the mainline JDK.\nLeyden Training Data Training data is part of the Leyden specific code. It tracks which methods have actually been loaded, executed, and used during the training run and how they have been used. Normally all loaded classes have associated class training data, but these may be omitted if, say, the class is loaded by a custom (user-defined) loader, is modified by an agent or fails to resolve because of linkage errors.There is a usage threshold which means that only methods that have been executed above that threshold will have associated method training data. Likewise, compiled method training data only exists for methods actually compiled during training. This helps both in keeping a smaller footprint in the cache and removing less useful data so processing the cache is faster.\nHow Do I Know Leyden Is Helping? Depending on how well you train your deployment you may see different improvements in time to reach application start (startup time) and time to reach peak performance (warmup time). Log output is one useful way to measure these two metrics but the details will depend on what monitoring capabilities are available in your test or production environment. However, simply measuring these two times (or even recording warmup profiles) doesn\u2019t help with the problem of explaining why, for some given training regime, you get a specific improvement or perhaps, in some cases, no measurable improvement.\nFor any given AOT cache (or set of alternative caches) it is very helpful to have some idea of what assets were included or excluded in the training set, which ones were written into the cache and what benefit they provide during a production run. In particular, it is useful to have both aggregate statistics and information on individual assets and their relationships. The Leyden project has provided a tool precisely to address these needs. Let’s see a practical example of how to diagnose an AOT Cache.\nFor the purpose of this article, we are going to use the following example application: https:\/\/github.com\/Delawen\/bad-good-cache\nThis is a web application that has a simple API and a basic html interface to use it.\nThe first thing we need to do is to compile this application on the root folder:\n$ mvn clean package\nTraining the application Once we have the jar created, we use it to start a training run:\n$ java -XX:AOTCacheOutput=target\/app.aot -Xlog:aot+map=trace,aot+map+oops=trace:file=target\/aot.map:none:filesize=0 -Xlog:class+load=info,aot+resolve*=trace,aot+codecache+exit=debug,aot*=warning:file=target\/training.log:level,tags -jar target\/quarkus-app\/quarkus-run.jar\nThe arguments we are going to use are the following:\n-XX:AOTCacheOutput=target\/app.aot Which will create an AOT file called app.aot -Xlog:aot+map=trace,aot+map+oops=trace:file=target\/aot.map:none:filesize=0 Which will create a map file that indexes and describes the previously created AOT file. -Xlog:class+load=info,aot+resolve*=trace,aot+codecache+exit=debug,aot*=warning:file=target\/training.log:level,tags Which will generate training log files with relevant information To help us train the application, we are going to use the `oha` tool, that helps us run a series of requests that will showcase a user using the application:\n$ oha --urls-from-file src\/main\/resources\/urls.txt -n 100\nNow that we have trained the application, let’s stop it with ctrl+c. It will take some time to stop while it builds the cache. It will do both the training and assembly steps at once.\nWe should have created three types of files:\ntarget\/app.aot : The AOT cache itself target\/aot.map : The map file target\/training.log : The logs for the training run Now that we have the AOT cache, we can start a production run, in which we will also save log files:\n$ exit\nThe arguments we are going to use are the following:\n-XX:AOTCache=target\/app.aot Make use of the AOT file called app.aot -Xlog:class+load=info,aot+resolve*=trace,aot+codecache+exit=debug,aot*=warning:file=target\/production.log:level,tags Which will generate a production log file with relevant information And we can use the application normally. Let’s play a bit on http:\/\/localhost:8080\/\nOn this run, we created the production.log file.\nAnalyzing the Cache After using it, we can stop it and analyze how the AOT Cache behaved with our AOT Cache diagnostics tool: https:\/\/github.com\/Delawen\/leyden-analyzer\nThe first step is loading all the information into the tool, to run a proper analysis:\n> load aotCache --background target\/aot.map\n> load trainingLog --background target\/training.log*\n> load productionLog --background target\/production.log*\nNow we are ready to start our analysis. A good place to start is the info command that shows a summarized version of what is inside the cache:\nAre we training the right thing? The first thing that should catch our attention is that there’s more than 10% of classes that were used on the production run but were not cached. That’s not usual, so let’s dig into whatImage description those classes are. There are hundreds of them, so if we filter by our package name, that would make our exploration easier:\nWhat does this mean? Let’s take a closer look:\nThis class was not loaded during training but it was loaded during production. Something went wrong with our training.\nWe can explore the class org.cutecats.rest.json.CatPhotoGenerator by looking at the source code. There, we discover that it should be used by org.cutecats.rest.json.CatResource.\nSo, this class was loaded both on training and production runs, and the metadata is included in the AOT Cache. But for some reason, none of its methods were profiled during the training run. This means that our training run did not make extensive use of this class. Maybe we should take a look at our training run.\nAnd indeed, there is an obvious mistake: the urls.txt file that oha used to create the requests only contains the static html pages. None of our Java classes are executed, although Quarkus loaded them at the beginning as services.\nLet’s run again the training, changing the url to the Java endpoints instead of the html pages: http:\/\/localhost:8080\/cats and http:\/\/localhost:8080\/list\nDon’t forget to remove the log and aot files from target\/ after each try to have clean runs (the clean on the maven command should do that).\nIf we analyze the results again with our tool, we should see a different result:\nWe have increased the percentage of the classes used (96%) in production that were cached compared to our last attempt (89%). That’s an improvement.\nDid we load all relevant classes during Training? Let’s check again for classes loaded in production that were not cached:\nSomething is still not working as intended. Maybe we should approach this from the other side: are we executing some testing code that replaces the real production code that should be executed during training?\nLet’s check if there’s something being stored in the cache that we don’t really need:\nWe can see a suspicious class called DummyPhotoGenerator. That’s supposed to be used only for testing purposes, not for real training and production. Using DummyPhotoGenerator instead of the CatPhotoGenerator class is making the code and classes used by CatPhotoGenerator not being used. If we explore our source code, we will discover that there is a \u201ctest\u201d argument on the \/cats endpoint that distinguishes between testing and production.\nTo fix our training, we have to call the endpoint \/cats with a test=false argument. Because the training run is not a test run.\nThe training run has to be as close to production as possible. If we use test classes, not only will they be stored in the cache and be loaded on production run, but they may also hide real production code from being trained.\nLet’s try again, now using http:\/\/localhost:8080\/cats?test=false in the urls.txt file.\nWe have increased a bit more the percentage of classes loaded, which is always a good sign. Image description Do we have any other classes loaded during production that were not cached during training? Are there any testing classes loaded during training or production runs?\nWe made sure that:\nAll our classes used in production are included in the cache\nNone of our testing classes are included in the cache\nAlthough we still don’t have the aspiring ideal 100% classes cached, we are really close (98%) and we can be happy with the list of classes cached. We can now focus on how good the profiling of the methods is.\nAre our methods properly trained? Maybe you already noticed another important information we have been ignoring until now: all our classes are labelled as “[Untrained]”. Let’s take a closer look at that.\nProfiling is done on each method independently, so let’s take a look at one of our methods that we know should be well trained. The describe command is pretty self explanatory on this case:\nLet’s follow the recommendation and do more requests during the training run.\n$ oha --urls-from-file src\/main\/resources\/urls.txt -n 10k\nAnd with 10 000 requests done, we can see that we got our method completely profiled and compiled to the higher level:\nThere\u2019s still more improvements that can be done to the training that will greatly depend on your application, but now we have all the basics covered.\n","permalink":"https:\/\/delawen.com\/2026\/03\/17\/How-is-Leyden-improving-Java-Performance-3\/","summary":"

In Part 1<\/a> of this series of 3 blog posts we introduced the specific performance challenges OpenJDK faces lowering application \u2018startup\u2019, \u2018warmup\u2019 and \u2018initial footprint\u2019 costs and provided an overview of what Leyden is doing to address those challenges.<\/p>\n

Part 2<\/a> described how to use the new capabilities offered by Leyden and presented test results which show that very significant progress has already been made and is set to continue.<\/p>\n

Part 3 provides a more detailed account of how Leyden\u2019s proposed solution operates and presents a first look at tooling that allows you to assess the benefits that result and tune your application to make the most of what Leyden offers.<\/p>","title":"How is Leyden improving Java Performance? Part 3 of 3"},{"content":"In Part 1 of this series of 3 blog posts we introduced the specific performance challenges OpenJDK faces lowering application \u2018startup\u2019, \u2018warmup\u2019 and \u2018initial footprint\u2019 costs and provided an overview of what Leyden is doing to address those challenges.\nPart 2 describes how to use the new AOT capabilities offered by Leyden and presents test results which show that very significant progress has already been made and is set to continue.\nPart 3 provides a more detailed account of how Leyden\u2019s proposed solution operates, and offers a first look at tooling that allows you to assess the benefits that result and tune your application to make the most of what Leyden offers.\nHow to use an AOT Cache To use an AOT cache (on JDK 25+), you need to add some JVM arguments to your app launch command. There are two ways of doing it, in 2 or 3 steps.\nJoint Training and Assembly steps \u2014 writing of the AOT cache is performed in a forked Java runtime at training run exit:\nTraining+Assembly Run: java -XX:AOTCacheOutput=${aot-cache-file} -jar app.jar Production Run: java -XX:AOTCache=${aot-cache-file} -jar app.jar Step 1 of the two step model runs your application until it exits (whether by means of some exit mechanism built into the application or simply by typing Ctrl-C on the console). At that point a separate Assembly JVM is forked to consume the training data collected during the training run and generate an AOT cache using the name supplied via the AOTCache command line option. The training JVM waits for the Assembly JVM to finish writing this file before it completes its own exit.\nStep 2 runs the production application using the AOT cache specified by the AOTCache command line option.\nSeparate Training and Assembly steps \u2014 allows the assembly run to be executed independently without delaying the training run exit:\nTraining Run: java - XX:AOTMode=record -XX:AOTConfiguration=${aot-cache-conf-file} -jar app.jar Assembly Run: java -XX:AOTMode=create -XX:AOTConfiguration=${aot-cache-conf-file} -XX:AOTCacheOutput=${aot-cache-file} -jar app.jar Production Run: java -XX:AOTCache=${aot-cache-file} -jar app.jar The three step model allows you to manage training and assembly as independent steps.\nStep 1 runs your application until it exits, at which the training data collected during the training run is dumped to an AOT configuration file specified using the AOTConfiguration command line option.\nIn step 2 this training data is passed to a new JVM using the same command line option and is used to generate an AOT cache to the file specified using the AOTCacheOutput command line option.\nStep 3 runs the production application using the AOT cache specified by the AOTCache command line option.\nThe 3 step workflow is sometimes preferable because it allows the training JVM to exit more quickly. Dumping of training data is usually quick even if it is not instantaneous. Generation of the AOT cache takes substantially longer because there is a lot more work involved in sorting and laying out that data in a format that meets the JVM\u2019s needs.\nAlso, with the Leyden premain release, the Assembly JVM will perform a \u2018cleanroom\u2019 compilation of all the methods to be included in the cache, possibly compiling them at more than one compilation level. This adds more time to the cache generation step.\nHow to properly execute the Training Run? The best way to train your application and generate the AOT cache is a canary deployment, where you run your application in the real production environment with training enabled, allowing it to collect training data as it runs. However, that\u2019s not always feasible, especially on containerized production environments that don\u2019t have disk-write privileges.\nDepending on how your deployment is set up this may be the type of circumstance where you choose the 3 step training model, allowing your training JVM to exit quickly and relegating the assembly to a separate, follow-on deployment. Note that the assembly JVM does not run your application code so will not need access to resources like networks or databases.\nRecording requests made to your application and replaying them on a test server (either in real time or delayed), is also a very good way to generate the AOT cache, as it reproduces exactly the same kind of behaviour you can expect on production. Alternatively, you can generate synthetic request data that simulates the behaviour you expect to encounter in real production, although that may reduce the relevance or accuracy of the resulting AOT cache assets.\nIf you have a strong testing framework, and you are using Quarkus, you can always generate the AOT cache using integration tests. Note that you will need to run the methods repeatedly (probably several thousands of calls) to generate the proper compilation optimizations.\nThe best results arise when the raining run resembles a production run as closely as possible. However, whatever training method you employ, on any cache produced will only be usable in production if you run with the same JVM and the same command line JVM options.\nYou can add extra jars at the end of the production run classpath but the initial segment must be the same as the classpath provided during training.\nAt the moment of writing this article, you also need to deploy on the same CPU family and operating system. In upcoming versions which will include compiled code in the cache, the production hardware must implement the exact same CPU features as the hardware used for the training run. If the CPU features are not identical then compiled and stub code assets will be ignored (other cache assets will still be usable).\nRemember to follow these basic constraints when generating the cache: same hardware, same Java version, same Operating System, and same JVM arguments.\nShould I start using AOT Cache in Java already? The short answer is yes.\nWhether your application gets significantly faster now, or if you are interested in testing it to help Leyden development move towards your interests, you should start using the AOT cache already.\nNote that you need at least JDK 25 to be able to use it. Performance gains are incremental with each new JDK release. The actual improvements that you can achieve using Leyden depend strongly on your application and how you use it.\nLet\u2019s see some examples. We are going to run them over JDK 26.\nHeavy Mathematical Example First we are going to use a benchmark application that runs heavy mathematical operations via a REST API. We are going to train this application twice to compare how different training affects performance on production.\nThis application makes use of an aot-jar from Quarkus which is optimized for Leyden and available since version 3.32.0.\nWe are going to use a training run that randomly calls the following urls:\n\/nqueens\/16 : to calculate the nqueens problem with a 16 board size \/fibonacci\/100 : to calculate fibonacci series with input 100 \/nqueens : to calculate the nqueens problem with either 16 or 8 as the board size \/fibonacci : to calculate fibonacci with a random number between 1 and 100 The idea is to have a load that is partly random (as an API with real users would be) but has a preference over specific branches or loop unrolling sizes.\nWe will do a training with 1000 requests and a second training with 60 000 requests. That should represent how different training affects final performance. We are going to run the application on a Linux machine assigning 2 cores to our application.\nDue to the kind of things Java 26 is storing in the AOT cache, we can theorize that there won\u2019t be much difference between the different trainings when comparing startup time (startup of the application and opening the port), as most of the code run during initialization is run only once, so adding more requests to the training won\u2019t improve that initialization time. This is something that may change in future developments as more assets are included in the AOT cache.\nBy using the Java AOT Cache Diagnostics Tool, we can compare the contents of the cache generated on each training.\n1000 requests training 60 000 requests training As expected, both trainings seem to have cached the same amount of metadata (slightly above 99% of classes used), because the code loaded into memory on both cases should be more or less the same (some timeout or runtime exception thrown may explain differences). This means that the startup time using any of the generated caches should be similar.\nThe training with 60 000 requests has many more methods that are profiled and compiled at a higher level because it had a longer time to profile and optimize. That should lead to better outcomes on the warmup time.\nAnyway, we should notice an improvement on startup time compared to the regular Java deployment, because we have a lot of metadata, profile and linkage data and some heap data already cached during AOT. And as we can see on the following graph, time to first response (which includes initialization) is already a third.\nThe other interesting measurement is how much and for how long response times are disrupted during the early stages of application execution. There is always some small variation in response times even when an app is fully warmed up \u2013 often referred to as jitter.\nHowever, during warmup the housekeeping work that the JVM has to do can significantly increase jitter. Individual responses may be delayed because they require the thread to execute one-off events like load or initialize a class, link a call site or field access site, or update profile data. Background JIT compilation will also steal CPU cycles, potentially pre-empting request handling in Java threads. Finally, early requests will mostly execute relatively slowly in the interpreter, while later requests will gradually respond more quickly as the JIT compiler delivers compiled code.\nIn theory, this is where longer training sessions should have a bigger impact. Better training results in more cached classes and heap objects needed in production, more pre-linking of calls and accesses, more method profile data to allow earlier and better informed compilation. So, we should see not only an improvement compared to the regular java version, but also a difference between the two trained caches.\nThe graph above shows individual response times for requests for each of the three deployments, Traditional Java (no AOT), AOT trained with 1000 requests, and AOT trained with 60 000 requests.\nIn all cases, the request rate is constant and within the peak capacity of the server. In all 3 cases the jitter slowly decays as the request count increases, eventually converging to a low, random variation.\nHowever, it is also very clear that\nThere is a lot more JVM housework being performed in the non-AOT case than when using AOT, reaching peak performance later. The well trained cache suffers less jitter, i.e. removes a lot more housework, than the weakly trained app. Note that JDK26 does store training data on the cache, but does not store compiled code. This means, future versions of the JDK will show a much larger difference between weak and strong training regimes.\nSimple REST API Now, we are going to do the same with a simple REST API application using Quarkus that connects to a database to extract data. This time we employ the Leyden premain JVM which caches code compiled code as well as all the other AOT cache assets mentioned earlier.\nWe use a single training run with 10 000 requests, executing a test that calls the endpoint \/fruits repeatedly. In contrast with the previous example, in this case we are not going to observe as much advantage from speculative compilation because the code is simpler and the entrypoint is always called with the same parameters. But we should still see an improvement all the same.\nLet\u2019s take a look at the time to first response to see if it is improved by Leyden AOT:\nStartup time in this example is slower than in the previous example because we have to initialize connections to the database and load the database model.\nNow, let\u2019s take a look at the response times and see if the warmup time is also improved thanks to the AOT cache.\nBoth runs suffer jitter during the first requests, at which point most of the housekeeping work is completed. The dramatic drop off in jitter for the AOT run around request 45 indicates that at this point almost all loading, initialization and linking costs have been met and the necessary compiled code has been delivered. By contrast the non-AOT run is still suffering jitter even after 100 requests i.e. it has still not warmed up to reach peak performance.\nThese are only a couple of examples that showcase how Leyden is already improving your startup and warmup time.\nHow far Leyden can help your application can only be discovered by trying it.\n","permalink":"https:\/\/delawen.com\/2026\/03\/17\/How-is-Leyden-improving-Java-Performance-2\/","summary":"

In Part 1<\/a> of this series of 3 blog posts we introduced the specific performance challenges OpenJDK faces lowering application \u2018startup\u2019, \u2018warmup\u2019 and \u2018initial footprint\u2019 costs and provided an overview of what Leyden is doing to address those challenges.<\/p>\n

Part 2 describes how to use the new AOT capabilities offered by Leyden and presents test results which show that very significant progress has already been made and is set to continue.<\/p>","title":"How is Leyden improving Java Performance? Part 2 of 3"},{"content":"In this series of 3 blog posts we will explain how OpenJDK project Leyden is helping to improve a specific area of performance where Java has notably lagged behind other languages i.e. application \u2018startup\u2019, \u2018warmup\u2019, and \u2018initial footprint\u2019.\nPart 1 explains what those terms mean and why Java faces challenges in matching the behaviour of other languages. It then provides an overview of what Leyden has done to improve startup and warmup in existing JDK releases and what is planned for upcoming releases.\nPart 2 describes how to use the new capabilities offered by Leyden and presents test results which show that very significant progress has already been made and is set to continue.\nPart 3 provides a more detailed account of how Leyden\u2019s proposed solution operates and presents a first look at tooling that allows you to assess the benefits that result and tune your application to make the most of what Leyden offers.\nA Brief History of Java Performance Java has been one of the most popular programming object-oriented languages for decades. Its success relies heavily on the fact that it offers a portable, managed runtime that makes it easy and safe to resolve many common programming challenges. In particular, Java was the first portable language to make it straightforward for programmers to deliver multi-threaded applications which allocate and manage storage at runtime without risk of invalid memory accesses.\nThe fact that Java remains popular still surprises some programmers, given that it belongs to the family of dynamic languages that, most notably, includes Lisp, Smalltalk and Self. Dynamic languages allow their code base to be incrementally defined as the program executes. That code base is often implemented using a language-specific virtual machine. Dynamic languages were traditionally executed by interpreting either the source code or an intermediate bytecode derived from the source. This often caused lower performance than native-compiled, non-dynamic languages.\nHowever, modern Java runtimes rely on powerful \u2018just-in-time\u2019 (JIT) compilers to translate bytecode to native machine code at runtime. JIT compilation, a technique originally tried in Smalltalk nearly 40 years ago, has improved Java performance by orders of magnitude from the early days of an interpreter-only runtime. The use of runtime execution profiling supports feedback directed optimization and speculative optimization. This has allowed Java JIT compilers to achieve peak performance that far exceeds what can be achieved with programs that are compiled ahead-of-time (AOT).\nWhy Java takes time to reach peak performance The downside of dynamic class loading and JIT compilation is that a Java runtime takes some time to achieve this impressive peak performance.\nWhen a new Java application is launched, it is normally a \u2018cold start\u2019. Details of all the classes and methods the application needs to use are only available in a compact bytecode representation, stored on disk either in application supplied class files or embedded in the Java platform\u2019s jmod files.The Java Virtual Machine (JVM) has to parse and unwrap this bytecode, constructing its own \u2018metadata\u2019 model of the class and method base, one that the interpreter and compiled code can efficiently operate over. It also has to set the base state of each loaded class, running Java \u2018static init\u2019 code to populate the class\u2019s static fields, before it can execute any of the class\u2019s methods.\nIn addition, the JVM has to perform dynamic linkage. When compilation or execution of a Java method first encounters a call (invoke bytecode) or a data access (get\/putfield bytecode) the JVM has to link that call or data access site. That involves replacing references to the target class and method\/field, which occur as symbol names in the bytecode, with a direct memory reference. This identifies first the target metadata class, and then the target metadata method or field. If the target class has not yet been encountered during execution, this linking step may trigger further bytecode loading, parsing, and class initialization.\nThe JVM normally starts off executing Java methods in the interpreter. Of course, it could always execute native code, compiling the Java method bytecode either immediately at load or lazily at first call. However, compilation takes time to complete so it is normally better done in the background while proceeding to interpret. Indeed, JIT compilation frequently pays off more when done selectively. Methods that only get called once or twice can take more cycles to compile than to simply interpret the bytecode.\nFurthermore, without runtime execution profile data as input, the compiler is unable to make informed, feedback-directed optimizations that significantly improve performance of the compiled code. Most importantly, it cannot simplify the compiled code by speculating that previous execution patterns will continue, replacing code that lies on untaken \u2019cold\u2019 branches with traps. Speculative compilation, an optimization first used in the Self compiler over 30 years ago, reduces both the size and the complexity of bytecode that feeds into a specific compilation. That, in turn, enables deep inlining of method calls and offers the possibility to identify many more derived optimizations. The rare case where a trap on a cold branch gets executed is handled by deoptimizing i.e. jumping back into the interpreter and recompiling the method with an updated branch profile.\nHousekeeping considered harmful During early stages of application execution, the JVM housekeeping overheads listed above are at their highest. Class loading and initialization, class linking, and recording of method execution profile data occur frequently as side effects of execution, for both application and JDK runtime methods, impeding direct forward progress of the application. Method compilation proceeds in dedicated, background compiler threads, but this still steals CPU cycles, once again, impeding application progress.\nThe impedance of JVM housekeeping work gradually decreases, as more and more of the required JDK code and application code is gradually linked into the runtime. At the same time delivery of compiled code improves application execution speed incrementally.\nAfter some time, a steady state is reached where most or all classes are loaded and linked, most or all methods have been profiled, and all \u2018hot\u2019 methods have been compiled with highly efficient code. Very occasionally variation in input data or a phase change in program behaviour drives the application down a cold path, triggering deoptimization and incurring extra JVM overheads. However, by and large, applications mostly warm up and continue to run with steady peak performance.\nLeyden Project \u2018premain\u2019 Experiment Project Leyden has been experimenting with reducing the impedance of JVM house keeping tasks in the \u2018premain\u2019 branch of the project repository. The observation that drives the Leyden premain experiment is that, most of the time, the housekeeping operations that occur during an application run involve doing exactly or almost exactly the same work with the same result, certainly in the early stages where the impedance is high. On every run a lot of the same byecode gets loaded and linked, the same classes get initialized, the same methods turn out to be hot, and end up getting compiled with the same or very similar profile information.\nThis is especially true for the JDK runtime code that runs before entering the application main method, likewise for JDK library code that the application calls out to. The JVM will always load base classes like java.lang.Object, java.lang.Class, or java.util.String. The same String instances, hard coded as literals in JDK methods, are added to the heap on every single run. Container classes like List and HashTable are commonly reused for the same purposes.\nJDK classes are fixed for any given release so their class, method and field metadata will always be the same and they will always cross-reference each other (i.e. be linked) in exactly the same way. In fact the Leyden premain branch gets its name from its original focus, which was optimizing this JDK execution that happens before entering application main.\nThe idea of profiting from this identity of JDK metadata across runs is not new. Since JDK13 Class Data Sharing (CDS) has been able to optimize away class loading and bytecode parsing for JDK classes by storing the JVM\u2019s metadata model of the JDK classes in a CDS archive, allowing it to be reloaded \u2019oven-ready\u2019 on subsequent runs.\nThat version of CDS provided an effective, albeit limited, limited warm-start capability for the JDK, halving the time taken for the JDK to start up i.e. complete JDK initialization and enter the application main routine. CDS also helped application warmup by lowering initial costs involved in callouts to JDK library code.\nWith application classes there is no strong guarantee that the same classes will be present in the same format between one run and the next. Or that classes loaded and used on one run will always be loaded and used in the same way on subsequent runs. However, so long as the same jars appear in the classpath and the class bytecode is loaded without runtime-specific agent transformations, then it is possible and, for many classes, quite probable that saved metadata will be reusable.\nMore recent versions of CDS have supported save and restore metadata for application classes via a dynamic CDS archive, allowing the JVM to bypass loading and bytecode parsing costs for those classes on subsequent runs, improving both application startup and warmup.\nLeyden\u2019s premain branch builds on this success but it is addressing a bigger prize than just archived metadata. The broader internal JVM state \u2014 not just metadata but static field data, linkage data, method profiles, compiled code \u2014 which is slowly constructed during warmup, may vary depending on precisely what happens on each run. However, most of what is created on one run, if it could be saved in an archive \u2013 as CDS currently does with metadata, ought to be reusable on a subsequent run, short circuiting the housekeeping overheads normally incurred to create it.\nEven if some saved state might turn out not to be useful, because, say, a class was not referenced or a method not called in the subsequent run, the ability to reuse some of the state should still pay off. The cost of reloading the required state can be made much lower than the cost of recreating, meaning the application can reach peak performance earlier, with less impedance from the JVM. The more reusable state that can be saved the greater the reduction in impedance.\nTraining and Production Runs So, the basic idea behind is to run your application twice:\nTraining Run: in which we cache the metadata, the profiling statistics, some heap data, compiled code,… Production Run: loads the previously (ahead of time) cached information, so the run starts hot. This is the \u201creal\u201d run in which we make use of our app. Of course, this only makes sense if the training run accurately represents the production run.\nTo achieve this, we need to respect the following constraints:\nSame Hardware: Or the compiled code may not be able to run, and the optimizations made may even be against performance in our production run. Same Java version and source code: If we change the source code, anything cached related to the source code gets deprecated and becomes useless. Same Operative System Family: There are pieces of the JVM that behave differently on Linux, Windows or MacOS. We can\u2019t just reuse our cached information if we change it. Same JVM options (mostly): We could maybe change some JVM options (like use a different garbage collector). But then, profiling statistics that we cache, and information about how the application behaves, may no longer be valid to our new configuration. Better not to play with these settings. [optional] No Custom Classloaders: The cache will ignore (for now) the classes loaded with a custom classloader. This means that part of the application will not be hot when run for the second time. Some of the AOT improvements developed in the Leyden branch have already been made available in the latest JDK LTS version at the moment of writing this article (25). But the plan for subsequent releases is that more features will be migrated, more things will be cached, and the performance gains will get better and better. The performance gains strongly depend on your app usage, the JDK version you are using, and how good is the training you are doing.\nIn the next post we will explain how to use the new AOT capabilities that are available in both JDK25. We will also present test results which show that very significant progress has already been made and is set to continue on capabilities already present in JDK 26 and on.\n","permalink":"https:\/\/delawen.com\/2026\/03\/17\/How-is-Leyden-improving-Java-Performance\/","summary":"

In this series of 3 blog posts we will explain how OpenJDK project Leyden is helping to improve a specific area of performance where Java has notably lagged behind other languages i.e. application \u2018startup\u2019, \u2018warmup\u2019, and \u2018initial footprint\u2019.<\/p>\n

Part 1 explains what those terms mean and why Java faces challenges in matching the behaviour of other languages. It then provides an overview of what Leyden has done to improve startup and warmup in existing JDK releases and what is planned for upcoming releases.<\/p>","title":"How is Leyden improving Java Performance? Part 1 of 3"},{"content":"There are many initiatives inside the OpenJDK that are revolutionizing the way our Java applications behave. Project Leyden accelerates peak performance by improving ahead of time caching. But, do you know how to use it with your application?\nOn this session we will discuss what Leyden is, what is the state of art of the ahead of time cache, and showcase a few examples. We will prove (with potential demo effect!) that Leyden decreases warm-up time, and present the tools you can use to optimize your application!\n","permalink":"https:\/\/delawen.com\/events\/2026-01-22\/","summary":"

There are many initiatives inside the OpenJDK that are revolutionizing the way our Java applications behave. Project Leyden accelerates peak performance by improving ahead of time caching. But, do you know how to use it with your application?<\/p>\n

On this session we will discuss what Leyden is, what is the state of art of the ahead of time cache, and showcase a few examples. We will prove (with potential demo effect!) that Leyden decreases warm-up time, and present the tools you can use to optimize your application!<\/p>","title":"jChampionsConference: Ahead of time, the final frontier"},{"content":" ","permalink":"https:\/\/delawen.com\/events\/2025-12-15\/","summary":"

\n