Add spring aot smoke testing workflows#971
Conversation
jormundur00
force-pushed
the
jormundur00/gh-710
branch
from
1884e14 to
4142dd7
Compare
|
So, we've added a |
I've added a workflow for |
mhalbritter
left a comment
mhalbritter
left a comment
There was a problem hiding this comment.
I've looked at the PR and found some things. Besides that looks good.
docs/CI.md
Outdated
| - Triggers: PRs to master touching [tests/tck-build-logic/](tests/tck-build-logic/), [gradle/](gradle/), [build.gradle](../build.gradle), [settings.gradle](../settings.gradle), or [gradle.properties](../gradle.properties). | ||
| - Uses: generateInfrastructureChangedCoordinatesMatrix. Pulls allowed images, disables Docker networking, validates config, then runs tests. | ||
| - Test affected Spring AOT smoke tests ([.github/workflows/test-affected-spring-aot-3.5.x.yml](../.github/workflows/test-affected-spring-aot-3.5.x.yml), [.github/workflows/test-affected-spring-aot-4.0.x.yml](../.github/workflows/test-affected-spring-aot-4.0.x.yml), [.github/workflows/test-affected-spring-aot-main.yml](../.github/workflows/test-affected-spring-aot-main.yml)) | ||
| - Triggers: PRs to master touching [metadata/](metadata/) or [tests/src/](tests/src/). |
There was a problem hiding this comment.
Should we reference here just metadata/?
There was a problem hiding this comment.
Yes we should, good catch. I've recently removed the tests/ check (as it makes no sense to run these when only changing tests), but forgot to remove it from the documentation.
| import org.gradle.api.Project | ||
|
|
||
| allprojects { | ||
| tasks.register('__internalImpactScan') { |
There was a problem hiding this comment.
Documentation here would be very useful.
There was a problem hiding this comment.
Added a javadoc explaining what the script does to the top of the script.
| } | ||
| } | ||
|
|
||
| // gradle generateAffectedSpringTestMatrix -PbaseCommit=<base> -PnewCommit=<head> -PspringAotBranch=<branch> -PspringAotPath=/path/to/spring-aot-smoke-tests |
There was a problem hiding this comment.
Can you explain here a bit better how this mapping is done?
There was a problem hiding this comment.
Added a step-by-step comment on how the task generates this matrix.
|
Very cool! |
jormundur00
force-pushed
the
jormundur00/gh-710
branch
from
b0d3e46 to
71619ed
Compare
4 participants
What does this PR do?
In this PR we introduce workflows for automated testing of all PR-affected Spring AOT smoke tests. We add 3 separate workflows for
3.4.x,3.5.xandmain(4.0.x) branches of the smoke test repository. Whenever a committer changes an existing libraries metadata on a pull request, the workflows will run all existing smoke tests that have that library as a dependency.This is done in the following way (same process for each branch):
spring-aot-impact-scan.gradlegradle task into that repository. The task is then ran on the specific branch denoted by the workflow, and all tests that use eithernativeTestornativeTestCompileand have a transitive dependency on changed-metadata-libraries are output to the workflow (through thegenerateAffectedSpringTestMatrixtask).expectedToFailflag set in itsbuild.gradle, or if the last run of the test fails on the smoke test repositories CI. If that is the case, the job will be reported as successful (due to the issue with it being outside ofreachability-metadata'scontrol), otherwise, the job fails (meaning committers can't break existing tests with metadata changes).All tests run on specific graalvm versions based on their branch (GraalVM17 for
3.4.x/3.5.xand GraalVM25 for4.0.xby default), and this version mapping is specified inci.json. The java version used can be overridden in a specific testsbuild.gradle(by specifyingjavaVersion), and this is also taken into consideration by the workflows.The reasons we have a separate workflow for each branch, as opposed to one workflow for all branches are that: all branches use different configurations (so batching the same test, if it exists, on multiple branches in one job would be very difficult as the tests would differ and use different configurations) and to bypass the GitHub actions CI job limit (that would be easily reached in a one-workflow scenario). Multiple workflows also allow for easy removal or addition of a branch that we want to test.
An example of the workflows running for a PR that changes metadata for a library used by all tests can be found here:
jormundur00#81
Fixes: #710