Skip to content

Commit d883430

Browse files
tokokocloud-fan
andcommitted
[SPARK-59293][INFRA] Fix update_build_status.yml to map non-standard conclusions
### What changes were proposed in this pull request? This PR fixes `update_build_status.yml` so that it coerces non-standard workflow run conclusions to a value accepted by the PATCH /repos/{owner}/{repo}/check-runs/{check_run_id} endpoint. run.data.conclusion is mapped to itself when it is one of the conclusions the endpoint accepts, and to failure otherwise, instead of passing the raw value through. This mirrors the mapping of non-standard run statuses in the else branch a few lines below, added by SPARK-57124. It also isolates per-PR sync failures, so that a single uncaught error can no longer end the whole scheduled pass: - The per-PR check-run listing is wrapped in a `try/catch`; a failure skips only that PR, which is retried on the next pass. - Both check-run PATCH calls go through an `updateCheckRun` helper that logs and records a failure instead of throwing. - Backfill failures, which already skipped the PR, are now recorded too. - After the full pass, the step calls `core.setFailed` naming every recorded PR, so a recurring error like this 422 still turns the scheduled run red instead of hiding in the log of a green run. Transient fork-run lookup failures (5xx, rate limiting) still skip the PR without failing the step, as before. Successful syncs send the same PATCH bodies as before. ### Why are the changes needed? The scheduled job has been failing on every run for quite a while with `startup_failure is not a member of ["success", "failure", "neutral", "cancelled", "timed_out", "action_required", "skipped"].` Because the error was uncaught, it ended the pass and left every open PR later in the iteration unsynced, so their Build checks could stay `queued` after their fork runs had completed. Isolating failures per PR keeps one rejected value from stalling the whole job again. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Extracted the script from the YAML and ran it in node against a mocked GitHub API whose PATCH rejects invalid values with a 422, like the real endpoint. Injected a 502 on one PR's check-run listing, a 422 on another's PATCH, and a 500 on a third's backfill: the script synced every other PR, including the one after all three failures, then failed the step naming just those three. With no injected failures it sent the same PATCHes as before and did not fail the step. Not run against live data. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5 Closes #58579 from tokoko/fix-build-status-invalid-conclusion. Lead-authored-by: Tornike Gurgenidze <[email protected]> Co-authored-by: Wenchen Fan <[email protected]> Signed-off-by: Wenchen Fan <[email protected]>
1 parent 522140f commit d883430

1 file changed

Lines changed: 52 additions & 8 deletions

File tree

‎.github/workflows/update_build_status.yml‎

Lines changed: 52 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -112,14 +112,44 @@ jobs:
112112
const isActionRequiredBuildCheck = (cr) =>
113113
cr.name == 'Build' && cr.conclusion == 'action_required';
114114
115+
// PRs whose sync failed on this pass: listing their check runs, PATCHing a Build
116+
// check, or backfilling one. A failed call skips only that PR (it is retried on the
117+
// next pass) instead of aborting the whole scheduled run and leaving every PR later
118+
// in the iteration unsynced. The step still fails after the pass, so an error that
119+
// keeps recurring (e.g. a 422 for a value the endpoint rejects) stays visible instead
120+
// of hiding in the log of a green run. Transient fork-run lookup failures (see
121+
// isTransientError) are not recorded: they also skip the PR until a later pass, but
122+
// leave the step green.
123+
const failedPRs = new Set();
124+
125+
// PATCH a Build check-run, recording a failure against the PR instead of throwing.
126+
const updateCheckRun = async (pr, body) => {
127+
try {
128+
await github.request('PATCH /repos/{owner}/{repo}/check-runs/{check_run_id}', body);
129+
} catch (error) {
130+
console.error(' Updating check run ' + body.check_run_id + ' failed; '
131+
+ 'will retry next pass');
132+
console.error(error);
133+
failedPRs.add(pr.number);
134+
}
135+
};
136+
115137
// Iterate open PRs
116138
for await (const prs of github.paginate.iterator(endpoint,params)) {
117139
// Each page
118140
for await (const pr of prs.data) {
119141
console.log('SHA: ' + pr.head.sha)
120142
console.log(' Mergeable status: ' + pr.mergeable_state)
121143
if (pr.mergeable_state == null || maybeReady.includes(pr.mergeable_state)) {
122-
const checkRuns = await listCheckRuns(pr.head.sha)
144+
let checkRuns
145+
try {
146+
checkRuns = await listCheckRuns(pr.head.sha)
147+
} catch (error) {
148+
console.error(' Listing check runs failed for this PR; will retry next pass')
149+
console.error(error)
150+
failedPRs.add(pr.number)
151+
continue
152+
}
123153
124154
// Does this SHA already carry an action_required Build check? notify (or an
125155
// earlier pass of this updater) writes one when no fork run was found. It is
@@ -179,22 +209,30 @@ jobs:
179209
180210
// Keep syncing the status of the checks
181211
if (run.data.status == 'completed') {
182-
console.log(' Run ' + cr.id + ': set status (' + run.data.status + ') and conclusion (' + run.data.conclusion + ')')
183-
const response = await github.request('PATCH /repos/{owner}/{repo}/check-runs/{check_run_id}', {
212+
// PATCH /check-runs accepts only success | failure | neutral |
213+
// cancelled | timed_out | action_required | skipped, but a workflow_run
214+
// may also conclude startup_failure / stale, or report a null conclusion.
215+
// Those all mean the run produced no test result, so they become failure.
216+
const conclusions = ['success', 'failure', 'neutral', 'cancelled',
217+
'timed_out', 'action_required', 'skipped']
218+
const conclusion = conclusions.includes(run.data.conclusion)
219+
? run.data.conclusion : 'failure'
220+
console.log(' Run ' + cr.id + ': set status (' + run.data.status + ') and conclusion (' + run.data.conclusion + ' -> ' + conclusion + ')')
221+
await updateCheckRun(pr, {
184222
owner: context.repo.owner,
185223
repo: context.repo.repo,
186224
check_run_id: cr.id,
187225
output: cr.output,
188226
status: run.data.status,
189-
conclusion: run.data.conclusion,
227+
conclusion: conclusion,
190228
details_url: run.data.details_url
191229
})
192230
} else {
193231
// PATCH /check-runs accepts only queued | in_progress | completed,
194232
// but workflow_run may also report requested / waiting / pending.
195233
const status = run.data.status == 'in_progress' ? 'in_progress' : 'queued'
196234
console.log(' Run ' + cr.id + ': set status (' + run.data.status + ' -> ' + status + ')')
197-
const response = await github.request('PATCH /repos/{owner}/{repo}/check-runs/{check_run_id}', {
235+
await updateCheckRun(pr, {
198236
owner: context.repo.owner,
199237
repo: context.repo.repo,
200238
check_run_id: cr.id,
@@ -301,9 +339,9 @@ jobs:
301339
// even though its output still parses as syncable, so it too is excluded.
302340
//
303341
// The recheck listing and the create below are wrapped so a failure on one
304-
// PR degrades to "skip this PR, retry next pass" rather than aborting the
305-
// whole scheduled run and starving every PR later in the iteration - the same
306-
// reason the sync loop above guards its calls.
342+
// PR degrades to "skip this PR, retry next pass" (see failedPRs) rather than
343+
// aborting the whole scheduled run and starving every PR later in the
344+
// iteration - the same reason the sync loop above guards its calls.
307345
try {
308346
const recheck = await listCheckRuns(pr.head.sha)
309347
const existingBuild = recheck.some(cr =>
@@ -379,9 +417,15 @@ jobs:
379417
} catch (error) {
380418
console.error(' Backfill failed for this PR; will retry next pass')
381419
console.error(error)
420+
failedPRs.add(pr.number)
382421
continue
383422
}
384423
}
385424
}
386425
}
387426
}
427+
428+
if (failedPRs.size > 0) {
429+
core.setFailed('Failed to sync ' + failedPRs.size + ' PR(s), will retry next pass: #'
430+
+ [...failedPRs].join(', #'))
431+
}

0 commit comments

Comments
 (0)