(Mostly) Zero Trust: Sandboxing Java applications with GraalVM and Espresso

With the death of SecurityManager, isolating untrusted Java code execution (for example, in case of plugins or scripting) becomes even more difficult. But, luckily, the maturity of GraalVM, both from features and stability perspectives, offers a compelling solution to sandboxing Java code.

In today's post, we are going to explore GraalVM polyglot capabilities, more specifically GraalVM Espresso - an implementation of the Java Virtual Machine Specification built upon GraalVM as a Truffle interpreter. To make things more interesting, there are two runtime models that we are going to talk about:

• out of process: isolated host/guest JVM with prebuilt Espresso runtime
• out of process: isolated, potentially even different, host/guest JVM runtimes

At first, the amount of the details that GraalVM polyglot programming throws at you feels overwhelming, but the documentation really helps to navigate over them. The latest version of the GraalVM available as of this writing is 24.2.1 and this is what we are going to use. To keep things simple but meaningful, we are going to sandbox an application that:

• exposes one interface ApplicationService

        public interface ApplicationService {
            ApplicationInfo getApplicationInfo();
            ApplicationInfo setApplicationInfo(ApplicationInfo info);
        }
        

• exposes one custom data type (Java record)

        public record ApplicationInfo(String env, String version) {
        }
        

• provides the default ApplicationService implementation

      public class AppRunner {
          public static void main(String[] args) {
              System.out.println(getApplication().getApplicationInfo());
          }
  
          public static ApplicationService getApplication() {
              return new ApplicationService() {
                  private ApplicationInfo info = new ApplicationInfo("unconstrained", "1.0.0");
  
                  @Override
                  public ApplicationInfo getApplicationInfo() {
                      return info;
                  }
  
                  @Override
                  public ApplicationInfo setApplicationInfo(ApplicationInfo info) {
                      this.info = info;
                      return info;
                  }
              };
          }
     }
        

What the host will be doing with this sandboxed application? Quite simple: it will get access to default ApplicationService implementation, query the ApplicationInfo and change it to its own provided value. Such an exercise would demonstrate the essential interoperability dance between host and guest in the GraalVM world. What is interesting, the guest (application we are going to sandbox) has no idea it is running inside the GraalVM Espresso managed runtime.

Let us start building with provided GraalVM Espresso runtime (which is based on JDK-21). Here are the dependencies that we need to get started.

        <dependency>
            <groupId>org.graalvm.polyglot</groupId>
            <artifactId>polyglot</artifactId> 
            <version>24.2.1</version>
        </dependency>
        <dependency>
            <groupId>org.graalvm.espresso</groupId>
            <artifactId>espresso-language</artifactId> 
            <version>24.2.1</version>
        </dependency>
        <dependency>
            <groupId>org.graalvm.espresso</groupId>
            <artifactId>polyglot</artifactId>
            <version>24.2.1</version>
        </dependency>
        <dependency>
            <groupId>org.graalvm.espresso</groupId>
            <artifactId>espresso-runtime-resources-jdk21</artifactId>
            <version>24.2.1</version>
        </dependency>
        <dependency>
            <groupId>org.graalvm.polyglot</groupId>
            <artifactId>java-community</artifactId> 
            <version>24.2.1</version>
            <type>pom</type>
        </dependency>

To have access to sandboxed application types, we would also need to include the dependency on its API (just for sake of simplicity, we include the application module itself):

        <dependency>
            <groupId>com.example.graalvmt</groupId>
            <artifactId>app</artifactId> 
            <version>0.0.1-SNAPSHOT</version>
        </dependency>

The bootstrapping is pretty straightforward. The first thing the host application has to do is to create a new polyglot engine:

        final Engine engine = Engine
            .newBuilder()
            .build();

The next step establishes the polyglot context and is a bit more engaging. Please don't panic if some things are not clear from the start, we are going to look at all of them shortly.

        final Context context = Context
            .newBuilder("java")
            .option("java.Properties.java.class.path", application)
            .option("java.Polyglot", "true")
            .option("java.EnableGenericTypeHints", "true")
            .allowExperimentalOptions(true)
            .allowNativeAccess(true)
            .allowCreateThread(false)
            .allowHostAccess(HostAccess
                .newBuilder(HostAccess.ALL)
                .targetTypeMapping(
                        Value.class,
                        ApplicationInfo.class,
                        v -> true,
                        v -> new ApplicationInfo(
                            v.invokeMember("env").asString(), 
                            v.invokeMember("version").asString())
                    )
                .build()
            )
            .allowIO(IOAccess.NONE)
            .allowPolyglotAccess(PolyglotAccess
                .newBuilder()
                .allowBindingsAccess("java")
                .build())
            .engine(engine)
            .build();

Let us go over this snippet line by line. Since this is a polyglot context, we limit its languages to java only. We also provide the classpath to our sandboxed application JAR file using java.Properties.java.class.path property. We explicitly prohibiting thread creation with allowCreateThread(false) and I/O access with allowIO(IOAccess.NONE) but have to allow native access (allowNativeAccess(true)), this is a limitation of the GraalVM Espresso engine at the moment (please check [Espresso] Support running without native access. for more details).

The host access configuration needs some wider explanation. We do allow access to host from guest (sandboxed application) with HostAccess.ALL policy and also provide the type mapping for ApplicationInfo record class so we could reconstruct it from generic polyglot Value class. Last but not least, we allow polyglot access java bindings with allowBindingsAccess("java"). And by and large, this is it!

With the context bootstrapped, we are ready to interact with the sandboxed application (or in terms of GraalVM, guest) through polyglot bindings. To illustrate the host/guest isolation, let us make an inquiry about their JVM runtime versions.

        final Value runtime = context.getBindings("java").getMember("java.lang.Runtime");
        System.out.println("Host JVM version: " + Runtime.version());
        System.out.println("Guest JVM version: " + runtime.invokeMember("version"));
  

Depending on your system and settings, you may see something along these lines:

Host JVM version: 24.0.1+9-30
Guest JVM version: 21.0.2+13-LTS-jvmci-23.1-b33

Clearly, the host and guest JVMs are far apart. According to our plan, let us retrieve the instance of the ApplicationService and ask for ApplicationInfo:

        final ApplicationService service = context.getBindings("java")
            .getMember("com.example.AppRunner")
            .invokeMember("getApplication")
            .as(ApplicationService.class);
        System.out.println("ApplicationInfo? " + service.getApplicationInfo());

We should see in the console:

ApplicationInfo? ApplicationInfo[env=unconstrained, version=1.0.0]

Now let us change the ApplicationInfo by construction a new instance on the host and sending it back to the guest, using ApplicationService instance at hand:

        final ApplicationInfo newInfo = context.getBindings("java")
            .getMember("com.example.ApplicationInfo")
            .newInstance("sandboxed", "1.0.0")
            .as(ApplicationInfo.class);

        final ApplicationInfo info = service.setApplicationInfo(newInfo);
        System.out.println("ApplicationInfo? " + info);

This time, we should see in the console a different picture:

ApplicationInfo? ApplicationInfo[env=sandboxed, version=1.0.0]

With all details (hopefully) explained, we could see how things work in action. The easiest way to run the host application is by using Apache Maven and its Exec Maven Plugin (please change JAVA_HOME accordingly):

$ export JAVA_HOME=/usr/lib/jvm/java-24-openjdk-amd64/
$ mvn package exec:java -Dexec.mainClass="com.example.graalvm.sandbox.SandboxRunner" -Dexec.args="app/target/app-0.0.1-SNAPSHOT.jar" -f sandbox-bundled-jvm/

Here is how the complete output in the console looks like:

Host JVM version: 24.0.1+9-30
Guest JVM version: 21.0.2+13-LTS-jvmci-23.1-b33
ApplicationInfo? ApplicationInfo[env=unconstrained, version=1.0.0]
ApplicationInfo? ApplicationInfo[env=sandboxed, version=1.0.0]

Let us recap what we have done: we basically isolated the application we wanted to sandbox into separate JVM instance but preserved full access to its API, limiting some very key JVM capabilities (no threads, no I/O). This is very powerful but has one limitation: we are bounded by the GraalVM Espresso runtime. What if we could have used any other JVM? Good news, it is totally an option and this is what we are going to do next.

As of this writing, GraalVM Espresso allows running either Java 8, Java 11, Java 17, or Java 21 guest JVMs. However, the host could be running any other JVM, and why not latest JDK-24? There aren't actually too many changes that we need to do to the Context:

    final Context context = Context
            .newBuilder("java")
            .option("java.JavaHome", jdkHome)
            .option("java.Classpath", application)
            .option("java.Properties.java.security.manager", "allow")
            .option("java.PolyglotInterfaceMappings", getInterfaceMappings())
            .option("java.Polyglot", "true")
            .option("java.EnableGenericTypeHints", "true")
            .allowExperimentalOptions(true)
            .allowNativeAccess(true)
            .allowCreateThread(false)
            .allowHostAccess(HostAccess
                .newBuilder(HostAccess.ALL)
                .targetTypeMapping(
                        Value.class,
                        ApplicationInfo.class,
                        v -> true,
                        v -> new ApplicationInfo(
                            v.invokeMember("env").asString(), 
                            v.invokeMember("version").asString())
                    )
                .build()
            )
            .allowIO(IOAccess.NONE)
            .allowPolyglotAccess(PolyglotAccess
                .newBuilder()
                .allowBindingsAccess("java")
                .build())
            .engine(engine)
            .build();

Notice that we do provide own JDK installation (using java.JavaHome) and classpath (using java.Classpath). More to that, we have an opportunity to pass any additional JVM properties (for example, "java.Properties.java.security.manager", "allow" is equivalent to passing -Djava.security.manager=allow, however we don't really use it). The new option here is java.PolyglotInterfaceMappings:

    private static String getInterfaceMappings(){
        return "com.example.ApplicationService;";
    }

It allows to automatically constructing guest proxies for host objects that implement declared interfaces in the list (essentially, we could provide our own ApplicationService and push it to the guest). And one important note, we don't need espresso-runtime-resources-jdk21 dependency anymore. Everything else stays unchanged.

If we run the host application this time:

$ export JAVA_HOME=/usr/lib/jvm/java-24-openjdk-amd64/
$ mvn package exec:java -Dexec.mainClass="com.example.graalvm.sandbox.SandboxRunner" -Dexec.args="/usr/lib/jvm/java-17-openjdk-amd64/ app/target/app-0.0.1-SNAPSHOT.jar" -f sandbox-custom-jvm/

The output will be slightly different:

Host JVM version: 24.0.1+9-30
Guest JVM version: 17.0.15+6-Ubuntu-0ubuntu124.04
ApplicationInfo? ApplicationInfo[env=unconstrained, version=1.0.0]
ApplicationInfo? ApplicationInfo[env=sandboxed, version=1.0.0]

Sky is the limit! GraalVM with its powerful polyglot features and GraalVM Espresso engine offers an unique capability to isolate (sandbox) untrusted Java code execution, using a JDK of your choice (you can even run JDK-21 with full-fledged SecurityManager activated if dimed necessary). We have just looked into some foundational concepts (there are still some limitations) however there are much more to uncover here (and please, note that GraalVM evolves very fast, the limitations of today become features of tomorrow). Hope you find it useful.

The complete source code of the project is available on Github.

I 🇺🇦 stand 🇺🇦 with 🇺🇦 Ukraine.

Apache CXF at speed of ... native!

GraalVM has been around for quite a while, steadily making big waves in OpenJDK community (looking at you, JEP 483: Ahead-of-Time Class Loading & Linking). It is wonderful piece of JVM engineering that gave birth to new generation of the frameworks like Quarkus, Helidon and Micronaut, just to name a few.

But what about the old players, like Apache CXF? A large number of applications and services were built on top of it, could those benefit from GraalVM, and particularly native image compilation? The answer to this question used to vary a lot, but thanks to steady progress, GraalVM strives to make it as frictionless as possible.

In today's post, we are going to build a sample Jakarta RESTful web service using Apache CXF and Jakarta XML Binding, and compile it to native image with GraalVM Community 21.0.2+13.1.

Let us start off with the data model, which consists of a single POJO, class Customer.

import jakarta.xml.bind.annotation.XmlRootElement;

@XmlRootElement(name = "Customer")
public class Customer {
    private long id;
    private String name;

    public long getId() {
        return id;
    }

    public void setId(long id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }
}

The class CustomerResource, a minimal Jakarta RESTful web service implementation, exposes a few endpoints to manage Customers, for simplicity - the state is stored in memory.

import java.util.Collection;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.atomic.AtomicInteger;

import jakarta.ws.rs.DELETE;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.POST;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.PathParam;
import jakarta.ws.rs.Produces;
import jakarta.ws.rs.core.MediaType;
import jakarta.ws.rs.core.Response;
import jakarta.ws.rs.core.Response.Status;

@Path("/")
@Produces(MediaType.APPLICATION_XML)
public class CustomerResource {
    private final AtomicInteger id = new AtomicInteger();
    private final Map<Long, Customer> customers = new HashMap<>();

    @GET
    @Path("/customers")
    public Collection<Customer> getCustomers() {
        return customers.values();
    }

    @GET
    @Path("/customers/{id}")
    public Response getCustomer(@PathParam("id") long id) {
        final Customer customer = customers.get(id);
        if (customer != null) {
            return Response.ok(customer).build();
        } else {
            return Response.status(Status.NOT_FOUND).build();
        }
    }

    @POST
    @Path("/customers")
    public Response addCustomer(Customer customer) {
        customer.setId(id.incrementAndGet());
        customers.put(customer.getId(), customer);
        return Response.ok(customer).build();
    }

    @DELETE
    @Path("/customers/{id}")
    public Response deleteCustomer(@PathParam("id") long id) {
        if (customers.remove(id) != null) {
            return Response.noContent().build();
        } else {
            return Response.status(Status.NOT_FOUND).build();
        }
    }
}

The last piece we need is to have running web container to host the CustomerResource service. We are going to use Eclipse Jetty but any other HTTP transport supported by Apache CXF will do the job.

import org.apache.cxf.jaxrs.JAXRSServerFactoryBean;
import org.apache.cxf.jaxrs.lifecycle.SingletonResourceProvider;

public class Server {
    public static org.apache.cxf.endpoint.Server create() {
        final JAXRSServerFactoryBean sf = new JAXRSServerFactoryBean();
        sf.setResourceClasses(CustomerResource.class);
        sf.setResourceProvider(CustomerResource.class, new SingletonResourceProvider(new CustomerResource()));
        sf.setAddress("http://localhost:9000/");
        return sf.create();
    }
    
    public static void main(String[] args) throws Exception {
        var server = create();
        server.start();
    }
}

Literally, this is all we need from the implementation perspective. The Apache Maven dependencies list is limited to handful of those:

<dependencies>
	<dependency>
		<groupId>org.apache.cxf</groupId>
		<artifactId>cxf-rt-transports-http</artifactId>
		<version>4.0.5</version>
	</dependency>
	<dependency>
		<groupId>org.apache.cxf</groupId>
		<artifactId>cxf-rt-transports-http-jetty</artifactId>
		<version>4.0.5</version>
	</dependency>
	<dependency>
		<groupId>org.apache.cxf</groupId>
		<artifactId>cxf-rt-frontend-jaxrs</artifactId>
		<version>4.0.5</version>
	</dependency>
	<dependency>
		<groupId>ch.qos.logback</groupId>
		<artifactId>logback-classic</artifactId>
		<version>1.5.7</version>
	</dependency>
</dependencies>

Cool, so what is next? The GraalVM project provides Native Build Tools to faciliate building native images, including dedicated Apache Maven plugin. However, if we just add the plugin into the build, the resulting native image won't be functionable, even if the build succeeds:

$./target/cxf-jax-rs-graalvm-server

Exception in thread "main" java.lang.ExceptionInInitializerError
        at [email protected]/java.lang.Class.ensureInitialized(DynamicHub.java:601)
        at com.example.jaxrs.graalvm.Server.create(Server.java:27)
        at com.example.jaxrs.graalvm.Server.main(Server.java:35)
        at [email protected]/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)
Caused by: java.util.MissingResourceException: Can't find bundle for base name org.apache.cxf.jaxrs.Messages, locale en
        at [email protected]/java.util.ResourceBundle.throwMissingResourceException(ResourceBundle.java:2059)
        at [email protected]/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1697)
        at [email protected]/java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1600)
        at [email protected]/java.util.ResourceBundle.getBundle(ResourceBundle.java:1283)
        at org.apache.cxf.common.i18n.BundleUtils.getBundle(BundleUtils.java:94)
        at org.apache.cxf.jaxrs.AbstractJAXRSFactoryBean.<clinit>(AbstractJAXRSFactoryBean.java:69)
        ... 4 more

Why is that? GraalVM operates under closed world assumption: all classes and all bytecodes that are reachable at run time must be known at build time. Since a majority of the frameworks, Apache CXF included, does not comply with such assumptions, GraalVM needs some help: tracing agent. The way we are going to let GraalVM capture all necessary metadata is pretty straightforward:

• add test cases which exercise the service logic (more is better)
• run test suite using tracing agent instrumentation
• build the native image using the metadata collected by the tracing agent

If that sounds like a plan to you, let us add the test case first:

import java.io.InputStream;
import java.io.IOException;

import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;

import jakarta.ws.rs.client.ClientBuilder;
import jakarta.ws.rs.client.Entity;
import jakarta.ws.rs.core.Response;

import static org.hamcrest.CoreMatchers.equalTo;
import static org.hamcrest.MatcherAssert.assertThat;

public class ServerTest {
    private org.apache.cxf.endpoint.Server server;
    
    @BeforeEach
    public void setUp() {
        server = Server.create();
        server.start();
    }
    
    @Test
    public void addNewCustomer() throws IOException {
        var client = ClientBuilder.newClient().target("http://localhost:9000/customers");
        try (InputStream in = getClass().getResourceAsStream("/add_customer.xml")) {
            try (Response response = client.request().post(Entity.xml(in))) {
                assertThat(response.getStatus(), equalTo(200));
            }
        }
    }

    @Test
    public void listCustomers() {
        var client = ClientBuilder.newClient().target("http://localhost:9000/customers");
        try (Response response = client.request().get()) {
            assertThat(response.getStatus(), equalTo(200));
        }
    }

    @AfterEach
    public void tearDown() {
        server.stop();
        server.destroy();
    }
}

Awesome, with tests in place, we could move on and integrate Native Build Tools into our Apache Maven build. It is established practice to have a dedicated profile for native image since the process could take quite a lot of time (and resources):

<profiles>
	<profile>
		<id>native-image</id>
		<activation>
			<property>
				<name>native</name>
			</property>
		</activation>
		<build>
			<plugins>
				<plugin>
					<groupId>org.graalvm.buildtools</groupId>
					<artifactId>native-maven-plugin</artifactId>
					<extensions>true</extensions>
					<version>0.10.2</version>
					<executions>
						<execution>
							<goals>
								<goal>compile-no-fork</goal>
							</goals>
							<phase>package</phase>
						</execution>
					</executions>
					<configuration>
						<agent>
							<enabled>true</enabled>
							<defaultMode>direct</defaultMode>
							<modes>
								<direct>config-output-dir=${project.build.directory}/native/agent-output</direct>
							</modes>
						</agent>
						<mainClass>com.example.jaxrs.graalvm.Server</mainClass>
						<imageName>cxf-jax-rs-graalvm-server</imageName>
						<buildArgs>
							<buildArg>--enable-url-protocols=http</buildArg>
							<buildArg>--no-fallback</buildArg>
							<buildArg>-Ob</buildArg>
						</buildArgs>
						<metadataRepository>
							<enabled>false</enabled>
						</metadataRepository>
						<resourcesConfigDirectory>${project.build.directory}/native</resourcesConfigDirectory>
					</configuration>
				</plugin>
			</plugins>
		</build>
	</profile>
</profiles>

It may look a bit complicated but fear not. The first thing to notice is that we configure tracing agent in the <agent> ... </agent> section. The captured metadata is going to be dumped into ${project.build.directory}/native/agent-output folder. Later on, the native image builder will refer to it as part of the <resourcesConfigDirectory> ... </resourcesConfigDirectory> configuration option. The profile is activated by the presence of native property.

Time to see each step in action! First thing first, run tests and capture the metadata:

$ mvn clean -Dnative test

...

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  6.977 s
[INFO] Finished at: 2024-08-30T15:50:55-04:00
[INFO] ------------------------------------------------------------------------

If we list the content of the target/native folder, we should see something like that:

$ tree target/native/

target/native/
└── agent-output
    ├── agent-extracted-predefined-classes
    ├── jni-config.json
    ├── predefined-classes-config.json
    ├── proxy-config.json
    ├── reflect-config.json
    ├── resource-config.json
    └── serialization-config.json    

If curious, you could inspect the content of each file, since it is just JSON, but we are going to proceed to the next step right away:

$ mvn -Dnative -DskipTests package

GraalVM Native Image: Generating 'cxf-jax-rs-graalvm-server' (executable)...
========================================================================================================================
Warning: Could not resolve org.junit.platform.launcher.TestIdentifier$SerializedForm for serialization configuration.
Warning: Could not resolve org.junit.platform.launcher.TestIdentifier$SerializedForm for serialization configuration.
[1/8] Initializing...                                                                                    (5.7s @ 0.10GB)
 Java version: 21.0.2+13, vendor version: GraalVM CE 21.0.2+13.1
 Graal compiler: optimization level: b, target machine: x86-64-v3
 C compiler: gcc (linux, x86_64, 9.4.0)
 Garbage collector: Serial GC (max heap size: 80% of RAM)
 2 user-specific feature(s):
 - com.oracle.svm.thirdparty.gson.GsonFeature
 - org.eclipse.angus.activation.nativeimage.AngusActivationFeature
------------------------------------------------------------------------------------------------------------------------
Build resources:
 - 9.99GB of memory (64.5% of 15.49GB system memory, determined at start)
 - 16 thread(s) (100.0% of 16 available processor(s), determined at start)
[2/8] Performing analysis...  [*****]                                                                   (31.2s @ 1.59GB)
   12,363 reachable types   (85.2% of   14,503 total)
   21,723 reachable fields  (63.2% of   34,385 total)
   61,933 reachable methods (57.6% of  107,578 total)
    3,856 types,   210 fields, and 2,436 methods registered for reflection
       62 types,    69 fields, and    55 methods registered for JNI access
        4 native libraries: dl, pthread, rt, z
[3/8] Building universe...                                                                               (4.8s @ 1.85GB)
[4/8] Parsing methods...      [**]                                                                       (3.0s @ 1.16GB)
[5/8] Inlining methods...     [***]                                                                      (2.2s @ 1.35GB)
[6/8] Compiling methods...    [*****]                                                                   (25.4s @ 1.89GB)
[7/8] Layouting methods...    [***]                                                                      (6.2s @ 1.49GB)
[8/8] Creating image...       [***]                                                                      (7.5s @ 2.03GB)
  32.41MB (49.57%) for code area:    38,638 compilation units
  30.78MB (47.08%) for image heap:  325,764 objects and 152 resources
   2.19MB ( 3.35%) for other data
  65.38MB in total

...

========================================================================================================================
Finished generating 'cxf-jax-rs-graalvm-server' in 1m 26s.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:30 min
[INFO] Finished at: 2024-08-30T15:58:42-04:00
[INFO] ------------------------------------------------------------------------

And we should end up with a fully functional executable, let us make sure this is the case:

$ ./target/cxf-jax-rs-graalvm-server

Aug 30, 2024 4:03:22 PM org.apache.cxf.endpoint.ServerImpl initDestination
INFO: Setting the server's publish address to be http://localhost:9000/
16:03:22.987 [main] INFO  org.eclipse.jetty.server.Server -- jetty-11.0.22; built: 2024-06-27T16:27:26.756Z; git: e711d4c7040cb1e61aa68cb248fa7280b734a3bb; jvm 21.0.2+13-jvmci-23.1-b30
16:03:22.993 [main] INFO  o.e.jetty.server.AbstractConnector -- Started ServerConnector@5725648b{HTTP/1.1, (http/1.1)}{localhost:9000}
16:03:22.994 [main] INFO  org.eclipse.jetty.server.Server -- Started Server@1f9511a6{STARTING}[11.0.22,sto=0] @24ms
16:03:22.994 [main] INFO  o.e.j.server.handler.ContextHandler -- Started o.a.c.t.h.JettyContextHandler@375bd1b5{/,null,AVAILABLE}

If we open up another terminal window and run curl from the command line, we should be hitting the instance of our service and getting successful responses back:

$curl http://localhost:9000/customers -H "Content-Type: application/xml" -d @src/test/resources/add_customer.xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Customer>
	<id>3</id>
    <name>Jack</name>
</Customer>

All the credits go to GraalVM team! Before we wrap up, you might be asking yourself if this the only way? And the short answer is "no": ideally, you should be able to add Native Build Tools and be good to go. The GraalVM Reachability Metadata Repository is the place that enables users of GraalVM native image to share and reuse metadata for libraries and frameworks in the Java ecosystem. Sadly, Apache CXF is not there just yet ... as many others.

The complete project sources are available on Github.

I 🇺🇦 stand 🇺🇦 with 🇺🇦 Ukraine.

All Your Tests Belong to You: Maintaining Mixed JUnit 4/JUnit 5 and Testng/JUnit 5 Test Suites

If you are seasoned Java developer who practices test-driven development (hopefully, everyone does it), it is very likely JUnit 4 has been your one-stop-shop testing toolbox. Personally, I truly loved it and still love: simple, minimal, non-intrusive and intuitive. Along with terrific libraries like Assertj and Hamcrest it makes writing test cases a pleasure.

But time passes by, Java has evolved a lot as a language, however JUnit 4 was not really up for a ride. Around 2015 the development of JUnit 5 has started with ambitious goal to become a next generation of the programmer-friendly testing framework for Java and the JVM. And, to be fair, I think this goal has been reached: many new projects adopt JUnit 5 from the get-go whereas the old ones are already in the process of migration (or at least are thinking about it).

For existing projects, the migration to JUnit 5 will not happen overnight and would probably take some time. In today's post we are going to talk about the ways to maintain mixed JUnit 4 / JUnit 5 and TestNG / JUnit 5 test suites with a help of Apache Maven and Apache Maven Surefire plugin.

To have an example a bit more realistic, we are going to test a UploadDestination class, which basically just provides a single method which says if a particular destination scheme is supported or not:

import java.net.URI;

public class UploadDestination {
    public boolean supports(String location) {
        final String scheme = URI.create(location).getScheme();
        return scheme.equals("http") || scheme.equals("s3") || scheme.equals("sftp");
    }
}

The implementer was kind enough to create a suite of JUnit 4 unit tests to verify that all expected destination schemes are indeed supported.

import static org.junit.Assert.assertTrue;

import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;

@RunWith(Parameterized.class)
public class JUnit4TestCase {
    private UploadDestination destination;
    private final String location;

    public JUnit4TestCase(String location) {
        this.location = location;
    }

    @Before
    public void setUp() {
        destination = new UploadDestination();
    }

    @Parameters(name= "{index}: location {0} is supported")
    public static Object[] locations() {
        return new Object[] { "s3://test", "http://host:9000", "sftp://host/tmp" };
    }

    @Test
    public void testLocationIsSupported() {
        assertTrue(destination.supports(location));
    }
}

In the project build, at very least, you need to add JUnit 4 dependency along with Apache Maven Surefire plugin and, optionally Apache Maven Surefire Reporter plugin, to your pom.xml, the snippet below illustrates that.

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.13.1</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M5</version>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-report-plugin</artifactId>
                <version>3.0.0-M5</version>
            </plugin>
        </plugins>
    </build>

No magic here, triggering Apache Maven build would normally run all unit test suites every time.

...
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.example.JUnit4TestCase
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.011 s - in com.example.JUnit4TestCase
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
...

Awesome, let us imagine at some point another teammate happens to work on the project and noticed there are no unit tests verifying the unsupported destination schemes so she adds some using JUnit 5.

package com.example;

import static org.junit.jupiter.api.Assertions.assertFalse;

import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;

class JUnit5TestCase {
    private UploadDestination destination;

    @BeforeEach
    void setUp() {
        destination = new UploadDestination();
    }

    @ParameterizedTest(name = "{index}: location {0} is supported")
    @ValueSource(strings = { "s3a://test", "https://host:9000", "ftp://host/tmp" } )
    public void testLocationIsNotSupported(String location) {
        assertFalse(destination.supports(location));
    }
}

Consequently, another dependency appears in the project's pom.xml to bring JUnit 5 in (since its API is not compatible with JUnit 4).

    <dependencies>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter</artifactId>
            <version>5.7.0</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

Looks quite legitimate, isn't it? But there is a catch ... the test run results would surprise this time.

...
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.example.JUnit5TestCase
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.076 s - in com.example.JUnit5TestCase
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
...

The JUnit 4 test suites are gone and such a behavior is actually well documentented by Apache Maven Surefire team in the Provider Selection section of the official documentation. So how we could get them back? There are a few possible options but the simplest one by far is to use JUnit Vintage engine in order to run JUnit 4 test suites using JUnit 5 platform.

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M5</version>
                <dependencies>
                    <dependency>
                        <groupId>org.junit.vintage</groupId>
                        <artifactId>junit-vintage-engine</artifactId>
                        <version>5.7.0</version>
                    </dependency>
                </dependencies>
            </plugin>

With that, both JUnit 4 and JUnit 5 test suites are going to be executed side by side.

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.example.JUnit5TestCase
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.079 s - in com.example.JUnit5TestCase
[INFO] Running com.example.JUnit4TestCase
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.009 s - in com.example.JUnit4TestCase
[INFO] 
[INFO] Results:
[INFO]
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------

The lesson to learn here: please watch carefully that all your test suites are being executed (the CI/CD usually keeps track of such trends and warns you right away). Especially, be extra careful when migrating to latest Spring Boot or Apache Maven Surefire plugin versions.

Another quite common use case you may run into is mixing the TestNG and JUnit 5 test suites in the scope of one project. The symptoms are pretty much the same, you are going to wonder why only JUnit 5 test suites are being run. The treatment in this case is a bit different and one of the options which seems to work pretty well is to enumerate the test engine providers explicitly.

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0-M5</version>
                <dependencies>
                    <dependency>                                      
                        <groupId>org.apache.maven.surefire</groupId>  
                        <artifactId>surefire-junit-platform</artifactId>      
                        <version>3.0.0-M5</version>                   
                    </dependency>
                    <dependency>                                      
                        <groupId>org.apache.maven.surefire</groupId>  
                        <artifactId>surefire-testng</artifactId>      
                        <version>3.0.0-M5</version>                   
                    </dependency>                          
                </dependencies>
            </plugin>

The somewhat undesired effect in this case is the fact that the test suites are run separately (there are other ways though to try out), for example:

[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running com.example.JUnit5TestCase
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.074 s - in com.example.JUnit5TestCase
[INFO] 
[INFO] Results:
[INFO]
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.315 s - in TestSuite
[INFO] 
[INFO] Results:
[INFO]
INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] 
[INFO] ------------------------------------------------------------------------

To be fair, I think JUnit 5 is a huge step forward towards having modern and concise test suites for your Java (and in general, JVM) projects. These days there are seamless integrations available with mostly any other test framework or library (Mockito, TestContainers, ... ) and the migration path is not that difficult in most cases. Plus, as you have seen, co-existence of JUnit 5 with older test engines is totally feasible.

As always, the complete project samples are available on Github: JUnit 4/JUnit 5, TestNG / JUnit 5.