Add versioned schemas and PR validation for index.json files by jormundur00 · Pull Request #883 · oracle/graalvm-reachability-metadata

jormundur00 · 2025-12-11T14:13:54Z

What does this PR do?

This PR introduces versioned schemas for all existing index.json files in the repository and adds a workflow to validate any changed index.json files in a pull request against their respective schemas.

Currently, there are three types of index.json files governed by these schemas:

Global metadata index.json - Located at metadata/index.json
Library metadata index.json - Located at metadata/<groupId>/<artifactId>/index.json for each library in the repository
Optional test sub-project index.json - Located at tests/src/<groupId>/<artifactId>/<version>/index.json

New Commands & Tasks

A new Java-based Gradle task validateIndexFiles has been added to the harness. This task is coordinate-aware (supporting -Pcoordinates and --coordinates) and handles space-separated lists for batch validation.

Validate all index files locally:

./gradlew validateIndexFiles -Pcoordinates=all

Validate specific library and its test indexes:

./gradlew validateIndexFiles -Pcoordinates=org.postgresql:postgresql:42.7.3

Infrastructure Integration: The validateIndexFiles task is now a core part of the testInfra and testAllInfra verification suites. It runs in parallel with other checks (like checkstyle and spotless) to ensure data integrity without increasing CI runtime significantly.

Additional Changes

CI Workflow Update: The GitHub Action now utilizes a space-separated string of changed coordinates. This allows for highly targeted validation on Pull Requests, significantly reducing CI overhead.
Documentation Improvements: Updated CONTRIBUTING.md to include specific validation steps within the local testing recipes and a new entry in the Quick Reference section for faster developer onboarding.
Release Packaging: To ensure compatibility and detect version mismatches in Native Build Tools, the following schemas are now bundled with all future releases of GraalVM Reachability Metadata:
- schemas/metadata-root-index-schema-v1.0.0.json
- schemas/metadata-library-index-schema-v1.0.0.json
- schemas/library-and-framework-list-schema-v1.0.0.json

Fixes: #813

vjovanov · 2025-12-11T14:54:00Z

.github/workflows/index-file-validation.yml

+          set -e
+
+          # Treat changed_files as a proper array
+          FILES=(${{ steps.filter.outputs.changed_files }})


We should always strive to do this in gradle. That way I can also do it locally to verify everything is OK.

Makes sense. I've removed this file-producing part of the GitHub action and created a generateChangedIndexFileMatrix gradle task that will do this in gradle (and is locally test-able). As these tasks require a ci.json entry, I've added it as only tested on java: latest-ea and os: ubuntu-latest as not to produce multiple jobs (because there shouldn't be any change with different jdks for this functionality as we don't run native-image).

vjovanov · 2025-12-18T14:55:01Z

.github/workflows/index-file-validation.yml

+        id: set-matrix
+        if: steps.filter.outputs.changed == 'true'
+        run: |
+          ./gradlew generateChangedIndexFileMatrix -PbaseCommit=${{ github.event.pull_request.base.sha }} -PnewCommit=${{ github.event.pull_request.head.sha }}


Why do we need a matrix for this? It takes 1 s to do, so we can do it all at once.

Yeah, this was a bit misleading, as we generated a one-dimensional matrix (so an array) which we fully parsed in one job. I renamed this to generateChangedIndexFileList (and made it so we generate a list, no longer needing ci.json input either), and we still validate all changed index.json files in one job run (as we don't need parallelism here).

vjovanov · 2025-12-18T14:55:54Z

schemas/metadata-library-index-schema-v1.0.0.json

+      },
+      "default-for": {
+        "type": "string",
+        "description": "Java regular expression describing the version range for which this entry should be used by default (e.g. '7\\\\.1\\\\..*')."


And why do we need these escapes?

In the description, we need four backslashes to 'show' two to the user (and the need for two is explained in the other comment).

The IDE documentation viewer parses this JSON string and 'collapses' the four backslashes into two. This ensures the user sees 7\\.1\\..* in their tooltip, which is exactly what they need to copy-paste into their own index.json to be valid.

vjovanov · 2025-12-18T14:56:37Z

schemas/metadata-library-index-schema-v1.0.0.json

+        "metadata-version": "0.0.1",
+        "module": "org.example:library",
+        "tested-versions": [ "0.0.1", "0.0.2" ],
+        "default-for": "0\\.0\\..*"


Also here, why do we need these escapes here?

Because this is a Regex inside a JSON string.

In Regex, the dot is a wildcard, so we need \. to match a literal version dot. But in JSON, a backslash is a special character that must be escaped by another backslash.

So, \\. in the file tells the JSON parser to pass \. to the Regex engine, which then correctly matches the literal version number.

vjovanov · 2025-12-18T14:56:55Z

schemas/metadata-library-index-schema-v1.0.0.json

+        "latest": true,
+        "metadata-version": "1.0.0",
+        "module": "org.example:library",
+        "tested-versions": [ "1.0.0", "1.1.0-M1", "1.1.0" ]


I would not put the milestone and CRs here because we don't want to encourage that.

Makes sense, replaced the milestone version with another full release version.

vjovanov · 2025-12-18T14:57:34Z

schemas/metadata-library-index-schema-v1.0.0.json

+        "tested-versions": [ "1.19.0" ],
+        "override": true,
+        "skipped-versions": [
+          { "version": "1.0.5", "reason": "Known incompatible API change." }


Usually this is the case because native image does not build due to some parsing issues.

I've replaced this example with a "real world" example where tests fail due to an un-parsable reflect-config.json.

This reverts commit bbf7a28.

vjovanov · 2025-12-22T15:58:50Z

docs/DEVELOPING.md

    ```console
   ./gradlew pullAllowedDockerImages -Pcoordinates=org.postgresql:postgresql:42.7.3
    ```
+2. Validate index schemas:


index.json

jormundur00 self-assigned this Dec 11, 2025

jormundur00 requested a review from vjovanov as a code owner December 11, 2025 14:13

jormundur00 added the enhancement New feature or request label Dec 11, 2025

jormundur00 requested a review from kimeta December 11, 2025 14:39

vjovanov reviewed Dec 11, 2025

View reviewed changes

jormundur00 requested a review from a team as a code owner December 12, 2025 09:04

jormundur00 force-pushed the jormundur00/gh-813 branch from 74b3af7 to 89f8e23 Compare December 12, 2025 09:07

jormundur00 mentioned this pull request Dec 16, 2025

Remove global tests index.json file #894

Merged

jormundur00 force-pushed the jormundur00/gh-813 branch 3 times, most recently from abda7a2 to ee85463 Compare December 19, 2025 08:08

vjovanov reviewed Dec 22, 2025

View reviewed changes

kimeta approved these changes Dec 22, 2025

View reviewed changes

jormundur00 force-pushed the jormundur00/gh-813 branch 2 times, most recently from 8fa33b1 to 5968b2e Compare December 22, 2025 15:22

jormundur00 added 10 commits December 22, 2025 16:24

Add schemas and schema validation

beb4089

Update schemas

43d8db9

Remove milestone version

8749e36

Change matrix generation to list

8d494fc

[TEST] Test changing index.json with success and fail

f31eab5

Hotfix

4472b30

Revert "[TEST] Test changing index.json with success and fail"

be80aef

This reverts commit bbf7a28.

Update schema documentation

e9ad776

Run validation fully in gradle

b7af366

[TEST] Only metadata/index.json test

4b9a8b8

jormundur00 force-pushed the jormundur00/gh-813 branch from 5968b2e to 4b9a8b8 Compare December 22, 2025 15:24

jormundur00 added 4 commits December 22, 2025 16:25

Revert tests

8e09a98

Move the gradle task to pure java

e3c5aea

Add docs and license

07ab79a

Test with infra

442bffd

vjovanov reviewed Dec 22, 2025

View reviewed changes

vjovanov approved these changes Dec 22, 2025

View reviewed changes

Hotfix docs

c9cce05

jormundur00 merged commit 712c02c into master Dec 22, 2025
29 checks passed

Conversation

jormundur00 commented Dec 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

New Commands & Tasks

Additional Changes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jormundur00 commented Dec 11, 2025 •

edited

Loading