Repository navigation
fix(ci): retry Maven Central downloads and restore E2E cache - #7781
Conversation
Aias00
left a comment
There was a problem hiding this comment.
Approved.
This directly addresses the failure mode in #7767, and the approach is conservative in the right places. I traced the classifier and the wrapper end to end:
Classifier (is_transfer_failure) — verified conservative, not permissive:
- Requires
errors > 0 && references > 0 && !invalid && model_errors == classified_model_errors, so a log with any second, unrelated terminal error is rejected. Confirmed against thecompile,tests,not-found,other-repository,mixed-failures,mixed-failures-reversed,mixed-dependencies,mixed-transfersandinvalid-modelfixtures — all correctly refuse to retry. - Only
central (https://repo.maven.apache.org/maven2)transfers matching reset / 429 / 502 (plus the WagonTransfer failed for … 502 Bad Gatewayshape) qualify;Could not find artifactshort-circuits to invalid, so the realapache.snapshotsmiss seen in #7184 is not retried. - Handles mvnd prefixes, ANSI colors and CRLF before classification, and the Help-reference allow-list (
DependencyResolution|PluginResolution|ProjectBuilding|UnresolvableModel) keepsMojoFailureExceptionout. - Each attempt re-classifies only the current attempt log, so a retried transfer error cannot be "reused" by a later attempt (covered by
current-attempt-only).
Wrapper — control flow is correct:
- At most 3 executions; 10s then 20s backoff;
statuses[1] -ne 0(tee failure) andstatuses[0] -ne 1both abort instead of retrying. - Statuses are captured from
PIPESTATUSimmediately after the pipeline underset +e, thenset -eis restored — no masking of the original exit code. INT -> 130/TERM -> 143with the EXIT trap still removing the temp log; matches the cancellation case.execis only ever the last statement of its branch, so no following step is silently skipped.- Argument fidelity is proven by the NUL-separated
printf | cmpcheck withtwo wordsandliteral $(command); *— no re-splitting and no eval.
Cache change in e2e-k8s.yml — restore and save use the identical ${{ runner.os }}-maven-${{ hashFiles(**/pom.xml) }} key, cache-hit != true skips only exact hits, and fallback/miss still publish. Restore-before-build is what removes the ~2.9k cold Central downloads measured in the benchmark.
Callers — the three compose scripts call the wrapper as a plain command and keep their $? / if ! checks; none of them use set -e, so error handling is preserved. native-sql-matrix.yml folded scalar renders correctly on one line.
CI is green on this branch (27 pass / 7 path-filtered skips), including the new Test Maven transfer retries step.
Non-blocking notes:
- Rollout sequencing matters.
pull_requestworkflows are read from the PR head, so merging this does not retroactively repair the ~49 currently-red PRs — each needs to merge/rebase master to pick up the new script. Operationally: merge this first, then batch-resync the red queue and re-run; otherwise it will look like nothing improved. - Scope reminder for release planning. This retries Maven Central transfer failures only. The e2e failures I sampled also involve Docker/Compose and other repositories, which stay unretried. Worth stating explicitly so the queue is not expected to fully clear.
- Cache save race (minor). Several jobs/workflows on the same ref compute the same key; if a reservation conflict ever surfaces as a step failure,
continue-on-error: trueon the Save step would absorb it. - Docs (optional). A short table in the script header listing which Maven error shapes are eligible would help future maintainers extend the allow-list when Maven changes its output wording.
Good to merge.
Fixes #7767
The final approach is retry + restore + conditional save, supported by controlled fork benchmarks and a read-only analysis of the official CI environment.
Problem
Maven Central transfer failures abort CI builds without a workflow-level retry. The E2E image-build job also saves the Maven repository cache but does not restore it before building.
Impact
Dependency-download failures stop build jobs and fail their aggregate checks, blocking PRs before the affected validation completes.
A seven-day review of 198 official failed-job logs confirmed at least 17 Maven Central reset/502 terminal failures, across 14 commit SHAs and three dates. This is a recurring CI reliability problem rather than an isolated failed run.
Changes
Transfer failed for … 502 Bad Gatewayformat. Allow at most three executions, with 10-second and 20-second delays.Benchmark results
The fork benchmark completed 12 serial runs across four configurations, plus seven concurrent runs. Every valid run used the same source revision and root Maven/Docker workload. Each serial configuration had three samples.
Downloadedlog entriesThe build mean decreased by 39.0 seconds (24.5%). Accounting for restore time gives approximately 32.7 seconds of net build-plus-restore savings.
Exact-hit save attempts spent 2–5 seconds performing archive/reservation work, with zero new cache-content upload. Skipping these attempts removes redundant work while retaining cache publication after fallback hits and misses.
Resource findings
These measurements support restoring dependencies before building, skipping redundant exact-hit saves, and retaining miss/fallback saves for downstream reuse.
Validation
The fork benchmark measured warm-cache performance; it did not naturally encounter reset/429/502 failures. Real transient-recovery success rates were therefore not measured. The subsequent conditional-save and bare-502 refinements were verified separately through local tests, source checks, and historical-log replay.
Detailed measurements and supporting evidence:
Non-blocking note
Artifact expiration during partial reruns and unrelated test failures are outside this PR’s scope.
Unsupported registry build-extension failures also remain outside this change.
Checklist
Make sure that:
./mvnw clean install -Dmaven.javadoc.skip=true.