FixTestNativeImageRun Gradle task to resolve test failures on Native Image run by kimeta · Pull Request #787 · oracle/graalvm-reachability-metadata

kimeta · 2025-11-19T17:04:55Z

What does this PR do?

Fixes: #786

This task runs the Java agent on failed tests for a new library version, and creates a version-specific metadata directory.
It will be executed by automation script #769 to update the metadata for each failing library version.

vjovanov · 2025-11-22T05:45:56Z

tests/tck-build-logic/src/main/groovy/org.graalvm.internal.tck-harness.gradle

+    task.setGroup(METADATA_GROUP)
+}
+// gradle fixTestNIRun --testLibraryCoordinates=<maven-coordinates> --newLibraryVersion=<library version which needs fix>
+tasks.register("fixTestNIRun", FixTestNativeImageRun.class) { task ->


Give it a more descriptive name.

Maybe fixTestNativeImageRun or fixTestNativeImageRunFailure?

vjovanov · 2025-11-22T05:47:46Z

tests/tck-build-logic/src/main/java/org/graalvm/internal/tck/GenerateMetadataTask.java

+import java.util.List;
+import java.util.stream.Collectors;
+
+public abstract class GenerateMetadataTask extends DefaultTask {


What does it do?

Explained in the added javadoc.

vjovanov · 2025-11-22T05:48:04Z

tests/tck-build-logic/src/main/java/org/graalvm/internal/tck/FixTestNativeImageRun.java

+import java.nio.file.Path;
+import java.util.regex.Pattern;
+
+public abstract class FixTestNativeImageRun extends DefaultTask {


Explain the usage.

vjovanov · 2025-11-22T05:51:51Z

tests/tck-build-logic/src/main/java/org/graalvm/internal/tck/GenerateMetadataTask.java

+
+    @Option(option = "coordinates", description = "Coordinates in the form of group:artifact:version")
+    public void setCoordinates(String coordinates) {
+        this.coordinates = coordinates;


We have CoordinateUtils now and they should be handled correctly.

vjovanov · 2025-11-22T05:52:35Z

tests/tck-build-logic/src/main/java/org/graalvm/internal/tck/GenerateMetadataTask.java

+    }
+
+    @Option(option = "allowedPackages", description = "Comma separated allowed packages (or - for none)")
+    public void setAllowedPackages(String allowedPackages) {


Isn't this set in index.json? We override here?

This name is ambiguous, this refers to packages that are ‘allowed’ in the agent filter. In ContributeTask, the variable is named packages, but perhaps we should use a more descriptive name. What about includedPackages or agentAllowedPackages, agentFilterPackages?

vjovanov · 2025-11-22T05:53:46Z

tests/tck-build-logic/src/main/java/org/graalvm/internal/tck/utils/MetadataGenerationUtils.java

+     * Executes the given executable with arguments in an optional working directory via Gradle ExecOperations.
+     * Captures output and throws a RuntimeException if the exit code is non-zero.
+     */
+    public static void invokeCommand(ExecOperations execOps, String executable, List<String> args, String errorMessage, Path workingDirectory) {


These are general utils. Is there a lib that can do that?

This is essentially a wrapper for org.gradle.process.ExecOperations#exec, handling all similar invocation preparations. Maybe we should create a GeneralUtils class for these types of methods, including printing methods for task progress: #787 (comment)?

vjovanov · 2025-11-22T05:58:45Z

tests/tck-build-logic/src/main/java/org/graalvm/internal/tck/FixTestNativeImageRun.java

+        // Ensure the tests build.gradle has an agent block; if not, create user-code-filter.json and add the agent block.
+        Path buildFilePath = testsDirectory.resolve("build.gradle");
+        if (!Files.isRegularFile(buildFilePath)) {
+            throw new RuntimeException("Cannot find tests build file at: " + buildFilePath);


GradleException, we should also add this to AGENTS.md.

vjovanov · 2025-11-22T05:59:15Z

tests/tck-build-logic/src/main/java/org/graalvm/internal/tck/FixTestNativeImageRun.java

+            throw new RuntimeException("Cannot find tests build file at: " + buildFilePath);
+        }
+        String buildGradle = Files.readString(buildFilePath, java.nio.charset.StandardCharsets.UTF_8);
+        boolean hasAgentBlock = Pattern.compile("(?s)\\bagent\\s*\\{").matcher(buildGradle).find();


Can this even happen?

Yes, some test directories for the libraries in the repository don’t contain any files related to agent filters.

vjovanov · 2025-11-22T06:00:12Z

tests/tck-build-logic/src/main/java/org/graalvm/internal/tck/utils/MetadataGenerationUtils.java

+        String buildGradle = Files.readString(buildFilePath, StandardCharsets.UTF_8);
+        boolean hasAgentBlock = Pattern.compile("(?s)\\bagent\\s*\\{").matcher(buildGradle).find();
+        if (hasAgentBlock) {
+            InteractiveTaskUtils.printUserInfo("Agent block already present in: " + BUILD_FILE + " - skipping");


Maybe general utils?

vjovanov · 2025-11-22T06:03:04Z

docs/CONTRIBUTING.md

+If you already have the test project structure in this repository and need to generate or regenerate metadata, use the `generateMetadata` task:
+
+```shell
+./gradlew generateMetadata --coordinates=com.example:my-library:1.0.0


Add these to DEVELOPING.md and to testAllParallel.

vjovanov · 2025-11-22T06:04:27Z

tests/tck-build-logic/src/main/java/org/graalvm/internal/tck/utils/MetadataGenerationUtils.java

+    /**
+     * Creates index.json inside the given version directory, listing all files present in that directory.
+     */
+    public static void createIndexJsonSpecificVersion(Path metadataDirectory) throws IOException {


Very useful :) \cc @jormundur00

vjovanov · 2025-11-27T09:50:05Z

AGENTS.md

 - Write type annotations in all functions and most variables.
 - Document code without being too verbose.
 - In java, always import classes and use them without qualified names.
+- Always use GradleException if possible.


Hmm, this should be in subprojects I think.

Added AGENTS.md to the tests/src.

vjovanov · 2025-11-27T09:51:29Z

build.gradle

            }

+            // Test metadata generation only for single coordinates (not batched like "1/64")
+            if (!coordinate.contains("/")) {


Guard against all as well.

Actually, don't guard. Just run this task above for a single coordinate.

Done, and reformatted the whole test so it is more clear and readable.

vjovanov · 2025-11-27T09:52:47Z

build.gradle

+                def artifact = parts[1]
+                def version = parts[2]
+
+                def versionDir = new File(tck.metadataRoot.get().asFile, "${group}/${artifact}/${version}")


We do this so many times, please reuse.

vjovanov · 2025-11-27T09:54:09Z

build.gradle

+                        throw new GradleException("No metadata files found in ${versionDir} (besides index.json)")
+                    }
+
+                    def indexList = (List) new groovy.json.JsonSlurper().parse(indexFile)


We need more rules for qualified names.

Fixed, and added to the AGENTS.md.

vjovanov · 2025-11-28T14:30:23Z

AGENTS.md

 - Write type annotations in all functions and most variables.
 - Document code without being too verbose.
- In java, always import classes and use them without qualified names.
+- In Java and Groovy, always import classes and use them without qualified names.


Hahaha, I did the same.

kimeta requested a review from vjovanov as a code owner November 19, 2025 17:04

kimeta force-pushed the mm/introduce-fixTestNIRun-task branch from 8454626 to 0ae9765 Compare November 19, 2025 17:05

kimeta changed the title ~~FixTestNIRun task which fixes test Native Image run fails~~ FixTestNIRun Gradle task to resolve test failures on Native Image run Nov 19, 2025

vjovanov reviewed Nov 22, 2025

View reviewed changes

kimeta force-pushed the mm/introduce-fixTestNIRun-task branch from 0ae9765 to af9be21 Compare November 26, 2025 20:35

kimeta requested a review from ban-mi as a code owner November 26, 2025 20:35

kimeta changed the title ~~FixTestNIRun Gradle task to resolve test failures on Native Image run~~ FixTestNativeImageRun Gradle task to resolve test failures on Native Image run Nov 26, 2025

kimeta force-pushed the mm/introduce-fixTestNIRun-task branch 4 times, most recently from fef6e98 to 8ded7e5 Compare November 27, 2025 00:58

vjovanov reviewed Nov 27, 2025

View reviewed changes

kimeta force-pushed the mm/introduce-fixTestNIRun-task branch from 8ded7e5 to 51fb75a Compare November 28, 2025 12:36

kimeta requested a review from a team as a code owner November 28, 2025 12:36

kimeta requested a review from msupic November 28, 2025 12:36

kimeta force-pushed the mm/introduce-fixTestNIRun-task branch 3 times, most recently from 7e213d9 to 9e7bdb6 Compare November 28, 2025 14:28

vjovanov reviewed Nov 28, 2025

View reviewed changes

vjovanov approved these changes Nov 28, 2025

View reviewed changes

Introduce fixTestNativeImageRun gradle task.

9dfa88e

kimeta force-pushed the mm/introduce-fixTestNIRun-task branch from 9e7bdb6 to 9dfa88e Compare November 28, 2025 14:42

kimeta merged commit 7e9e33b into oracle:master Nov 28, 2025
43 checks passed

Conversation

kimeta commented Nov 19, 2025 • edited by vjovanov Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kimeta commented Nov 19, 2025 •

edited by vjovanov

Loading