Reduce servlet smoke test matrix from 166 to 52 cases#16810
Conversation
|
this will make our list of supported application servers less impressive 😄 https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md#application-servers wdyt of only reducing the number that we run on PRs? (as opposed to on merge to main) |
3ce0e3e to
a967421
Compare
good idea - done |
|
I pushed a couple of changes, and CI is failing now but I think unrelated
EDIT: just made the connection to #16814 |
634e945 to
990d59f
Compare
Instead of permanently removing test cases, run a reduced subset (49 of 166) on pull requests and the full matrix on merge to main. This keeps the supported libraries list accurate while speeding up PR builds. The reduced set covers every server version, all JDK versions (via Tomcat/hotspot and TomEE/openj9), and both VMs. Adding a "test full smoke" label to a PR runs the complete matrix.
Replace the hardcoded REDUCED_TESTS set with rule-based logic that derives the reduced subset from @appserver annotations at runtime, matching the reduceTargets() rules in the Gradle build. Also fix duplicate Gradle task name for Payara 6.x by deduplicating after reduction.
…otReduced - Remove reduceTargets() and reducedSmokeTests from servlet image build (images are always built in full; reduction only applies at test time) - Remove -PreducedSmokeTests=true from pr-smoke-test-servlet-images.yml - Pass reducedSmokeTests boolean into skipIfNotReduced() from caller - Use findProperty() == "true" in smoke-tests/build.gradle.kts
ae9f99f to
c6be8d3
Compare
Signed-off-by: Gregor Zeitlinger <[email protected]>
|
closed in favor of #18953 |
2 participants
Blocked by #17721
Summary
Reduces the servlet smoke test matrix by 69% (166 → 52 test cases) by eliminating redundant server × JDK × VM combinations while preserving all coverage guarantees:
Context: SIG meeting feedback on #16436 — the full targeted-CI approach was too complex, but smaller improvements like reducing smoke tests are welcome.
Test plan
./gradlew :smoke-tests:test -PsmokeTestSuite=nonecompiles successfully./gradlew spotlessApplypasses