Skip to content

Commit 948d393

Browse files
luisleo526claude
andauthored
ci: skip suite jobs on draft PRs; the gate fails on drafts (LB-387) (#466)
* ci: skip the PR suite on draft PRs (LB-387) Skip preflight, build, sanitizers, kernel-only, native-live, and corpus-parity-subset for drafts; start full runs on ready_for_review. Update ci_preflight.py trigger/condition pins and matching test_ci_preflight.py pin references, preserving check logic. Keep authz, dependencies, runners, time limits, and build-gate unchanged. * ci: re-run on converted_to_draft and split the CI group by draft state (LB-387) Round B of the final read of #466: add converted_to_draft to the pull_request types so a ready run is followed by a failing draft aggregate when the PR is converted, and key the concurrency group on the draft flag so a stale draft re-run cannot cancel the full ready run. Exact pins plus one red control per change in test_ci_preflight.py. Lane: ci-draft-fix1 (Codex gpt-6.1-sol). Co-Authored-By: Claude Sonnet 5.5 <[email protected]> --------- Co-authored-by: Claude Sonnet 5.5 <[email protected]>
1 parent 83c1ecb commit 948d393

3 files changed

Lines changed: 91 additions & 23 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,17 @@ name: CI
22

33
on:
44
pull_request:
5+
# LB-387: suite jobs skip on a draft PR (the required aggregate then fails, never passes);
6+
# ready_for_review starts the full run when the PR is marked ready.
7+
# converted_to_draft re-runs the workflow so its failing aggregate supersedes the earlier ready run.
8+
types: [opened, synchronize, reopened, ready_for_review, converted_to_draft]
59

610
permissions:
711
contents: read
812

913
# Post-merge events live in ci-post.yml; this file only schedules PR jobs.
1014
concurrency:
11-
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}
15+
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}-${{ github.event.pull_request.draft && 'draft' || 'ready' }}
1216
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
1317

1418
env:
@@ -222,7 +226,7 @@ jobs:
222226
223227
preflight:
224228
needs: [authz, changes]
225-
if: ${{ always() && needs.authz.result == 'success' }}
229+
if: ${{ !github.event.pull_request.draft && (always() && needs.authz.result == 'success') }}
226230
runs-on: ["self-hosted","pf-gcp-pr"]
227231
env:
228232
PF_CI_LABELS: '["self-hosted","pf-ci","pf-ci-x64-16"]'
@@ -282,7 +286,7 @@ jobs:
282286
# successful docs-only or GitHub-only answer skips scheduling; the aggregate
283287
# independently reads the PR diff before accepting the latter skip.
284288
needs: [authz, changes]
285-
if: ${{ always() && needs.authz.result == 'success' && (needs.changes.result != 'success' || (needs.changes.outputs.docs_only != 'true' && (github.event_name != 'pull_request' || needs.changes.outputs.github_only != 'true'))) }}
289+
if: ${{ !github.event.pull_request.draft && (always() && needs.authz.result == 'success' && (needs.changes.result != 'success' || (needs.changes.outputs.docs_only != 'true' && (github.event_name != 'pull_request' || needs.changes.outputs.github_only != 'true')))) }}
286290
# 360 minutes (the platform maximum): a full-set leg waits on the
287291
# test_ci_verify CTest row, 976-1288 s on the standard runners at
288292
# 0d76a099, which more cores do not shorten. No bound of ours may fail a
@@ -384,7 +388,7 @@ jobs:
384388
# separate parity verdict. build-gate independently verifies GitHub-only skips.
385389
corpus-parity-subset:
386390
needs: [authz, changes]
387-
if: ${{ always() && needs.authz.result == 'success' && (needs.changes.result != 'success' || (needs.changes.outputs.docs_only != 'true' && (github.event_name != 'pull_request' || needs.changes.outputs.github_only != 'true'))) }}
391+
if: ${{ !github.event.pull_request.draft && (always() && needs.authz.result == 'success' && (needs.changes.result != 'success' || (needs.changes.outputs.docs_only != 'true' && (github.event_name != 'pull_request' || needs.changes.outputs.github_only != 'true')))) }}
388392
uses: ./.github/workflows/corpus-parity-subset.yml
389393

390394
# Advisory aggregate context covering every GitHub Actions lane. The merge

‎scripts/ci_preflight.py‎

Lines changed: 21 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,14 @@
4747
RELEASE_RUNNER = '["self-hosted","pf-gcp-main"]'
4848
CI_EVENTS = '''on:
4949
pull_request:
50+
# LB-387: suite jobs skip on a draft PR (the required aggregate then fails, never passes);
51+
# ready_for_review starts the full run when the PR is marked ready.
52+
# converted_to_draft re-runs the workflow so its failing aggregate supersedes the earlier ready run.
53+
types: [opened, synchronize, reopened, ready_for_review, converted_to_draft]
54+
'''
55+
CI_CONCURRENCY = '''concurrency:
56+
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}-${{ github.event.pull_request.draft && 'draft' || 'ready' }}
57+
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
5058
'''
5159
POST_CI_EVENTS = '''on:
5260
push:
@@ -170,14 +178,16 @@
170178
HEAVY_JOBS = frozenset(('sanitizers', 'kernel-only', 'native-live', 'corpus-parity'))
171179
AUTHZ_SUCCESS = "${{ needs.authz.result == 'success' }}"
172180
AUTHZ_ALWAYS = "${{ always() && needs.authz.result == 'success' }}"
181+
PREFLIGHT_IF = "${{ !github.event.pull_request.draft && (always() && needs.authz.result == 'success') }}"
173182
AUTHZ_FULL_SUCCESS = "${{ needs.authz.result == 'success' && github.event_name != 'pull_request' }}"
174183
DOCS_ONLY_SKIP = ("${{ always() && needs.authz.result == 'success' && (needs.changes.result != 'success' || "
175184
"needs.changes.outputs.docs_only != 'true') }}")
176185
# Only the scheduling hint comes from changes. The aggregate recomputes this
177186
# class from the event-bound API diff inside its own job.
178-
GITHUB_ONLY_SKIP = ("${{ always() && needs.authz.result == 'success' && (needs.changes.result != 'success' || "
187+
GITHUB_ONLY_SKIP = ("${{ !github.event.pull_request.draft && (always() && needs.authz.result == 'success' && "
188+
"(needs.changes.result != 'success' || "
179189
"(needs.changes.outputs.docs_only != 'true' && (github.event_name != 'pull_request' || "
180-
"needs.changes.outputs.github_only != 'true'))) }}")
190+
"needs.changes.outputs.github_only != 'true')))) }}")
181191
HEAVY_SKIP = DOCS_ONLY_SKIP
182192
SLOW_PREFLIGHT_IF = AUTHZ_ALWAYS
183193
BUILD_VERIFY_RUN = ' '.join(r'''|
@@ -1158,7 +1168,12 @@ def authz_workflow_findings(workflow: str, name: str, aggregate: str) -> list[st
11581168
if events is None or 'pull_request_target' in events:
11591169
findings.append(f'{name} must use readable events without pull_request_target')
11601170
pr = re.search(r'(?m)^ pull_request:[^\n]*(?:\n(?: [^\n]*|[ \t]*))*', workflow)
1161-
if name == 'ci.yml' and (not pr or pr[0].strip() != 'pull_request:'):
1171+
if name == 'ci.yml' and (not pr or pr[0].strip() != (
1172+
'pull_request:\n'
1173+
' # LB-387: suite jobs skip on a draft PR (the required aggregate then fails, never passes);\n'
1174+
' # ready_for_review starts the full run when the PR is marked ready.\n'
1175+
' # converted_to_draft re-runs the workflow so its failing aggregate supersedes the earlier ready run.\n'
1176+
' types: [opened, synchronize, reopened, ready_for_review, converted_to_draft]')):
11621177
findings.append('ci.yml must run on every pull_request without event filters')
11631178
if _job_values(workflow, 'permissions') != ['contents: read']:
11641179
findings.append(f'{name} must retain contents: read workflow permissions')
@@ -1379,6 +1394,8 @@ def pipeline_workflow_findings(workflow: str, name: str, *, pr: bool) -> list[st
13791394
events = CI_EVENTS if pr else POST_CI_EVENTS
13801395
if _job_block(workflow, 'on') != _code(events):
13811396
findings.append(f'{name} must keep its exact {"unfiltered PR" if pr else "main push, nightly and manual"} triggers')
1397+
if pr and _job_block(workflow, 'concurrency') != _code(CI_CONCURRENCY):
1398+
findings.append('ci.yml must keep its exact draft and ready concurrency groups')
13821399
jobs = _jobs(workflow)
13831400
proofs = PR_PROOF_JOBS if pr else PROOF_JOBS
13841401
expected_jobs = {'authz', 'changes', 'preflight', *proofs, 'build-gate'}
@@ -1404,7 +1421,7 @@ def pipeline_workflow_findings(workflow: str, name: str, *, pr: bool) -> list[st
14041421
findings.append("changes must publish the base's scripts/ci_verifier_paths.py answer, exactly")
14051422
preflight = jobs.get('preflight', '')
14061423
if (sorted(_needs(preflight)) != ['authz', 'changes']
1407-
or _job_values(preflight, 'if') != [AUTHZ_ALWAYS]):
1424+
or _job_values(preflight, 'if') != [PREFLIGHT_IF if pr else AUTHZ_ALWAYS]):
14081425
findings.append('preflight must follow authz and changes and run even if classification fails')
14091426
if (f' PINEFORGE_VERIFIER_SLOW: {PREFLIGHT_VERIFIER_SLOW}\n' not in preflight
14101427
or preflight.count('PINEFORGE_VERIFIER_SLOW:') != 1):

‎scripts/test_ci_preflight.py‎

Lines changed: 62 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
DOCS_ONLY_SKIP, GITHUB_ONLY_SKIP, HEAVY_SKIP, HOSTED_MATRIX_RUNNER, JOB_COMMANDS,
2222
PR_RUNNER, POST_RUNNER, MATRIX_LINUX_RUNNER, MATRIX_RUNNER,
2323
NATIVE_EXCLUSION, NATIVE_CACHE_SAVE, CURL_CACHE_SAVE,
24-
PROOF_JOBS, PR_PROOF_JOBS, RELEASE_RUNNER, _jobs, check_commands,
24+
PREFLIGHT_IF, PROOF_JOBS, PR_PROOF_JOBS, RELEASE_RUNNER, _jobs, check_commands,
2525
ci_workflow_findings, gated_workflow_findings,
2626
native_workflow_findings, run_checks)
2727

@@ -89,6 +89,13 @@ def in_job(workflow, job, before, after):
8989
return workflow[:at] + after + workflow[at + len(before):]
9090

9191

92+
def ready_pr(condition):
93+
"""Keep the draft guard when mutating another PR job condition."""
94+
if not condition.startswith('${{ ') or not condition.endswith(' }}'):
95+
raise AssertionError(f'not a workflow condition: {condition!r}')
96+
return '${{ !github.event.pull_request.draft && (' + condition[4:-3] + ') }}'
97+
98+
9299
def in_step(workflow, job, name, before, after):
93100
"""Mutate the named real step, even when another step shares a line."""
94101
step = ci_preflight._step_named(ci_preflight._steps(_jobs(workflow)[job]),
@@ -631,8 +638,9 @@ def test_all_status_functions_require_refusal_and_repository_guards_from_mutated
631638
guard = f' if: {ci_preflight.AUTHZ_SUCCESS}\n'
632639
unnamed_run = ' - run: python3 scripts/ci_preflight.py --output-dir build-ci-preflight\n'
633640
for function in ('always()', 'cancelled()', 'failure()', 'success()'):
634-
conditioned = in_job(body, None, f' if: {AUTHZ_ALWAYS}\n',
635-
' if: ${{ ' + function + " && needs.authz.result == 'success' }}\n")
641+
condition = '${{ ' + function + " && needs.authz.result == 'success' }}"
642+
conditioned = in_job(body, None, f' if: {PREFLIGHT_IF}\n',
643+
f' if: {ready_pr(condition)}\n')
636644
mutations = (
637645
(in_job(conditioned, None, AUTHZ_REFUSE_STEP, ''),
638646
'ci.yml job preflight with a status function must refuse authz first'),
@@ -1338,6 +1346,41 @@ def test_ci_split_requires_every_home_trigger_without_path_filters(self):
13381346
+ ('unfiltered PR' if index == 0 else 'main push, nightly and manual') + ' triggers',
13391347
findings)
13401348

1349+
def test_ci_split_requires_ready_for_review(self):
1350+
self.assertEqual(ci_workflow_findings(*self.ci_sources(), others=self.other_sources()), [])
1351+
findings = self.mutated_file_findings(0, None,
1352+
' types: [opened, synchronize, reopened, ready_for_review, converted_to_draft]\n',
1353+
' types: [opened, synchronize, reopened, converted_to_draft]\n')
1354+
self.assertIn('ci.yml must keep its exact unfiltered PR triggers', findings)
1355+
self.assertIn('ci.yml must run on every pull_request without event filters', findings)
1356+
1357+
def test_ci_split_requires_converted_to_draft(self):
1358+
self.assertEqual(ci_workflow_findings(*self.ci_sources(), others=self.other_sources()), [])
1359+
findings = self.mutated_file_findings(0, None,
1360+
' types: [opened, synchronize, reopened, ready_for_review, converted_to_draft]\n',
1361+
' types: [opened, synchronize, reopened, ready_for_review]\n')
1362+
self.assertIn('ci.yml must keep its exact unfiltered PR triggers', findings)
1363+
self.assertIn('ci.yml must run on every pull_request without event filters', findings)
1364+
1365+
def test_ci_split_requires_separate_draft_and_ready_concurrency_groups(self):
1366+
self.assertEqual(ci_workflow_findings(*self.ci_sources(), others=self.other_sources()), [])
1367+
findings = self.mutated_file_findings(0, None,
1368+
' group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}'
1369+
"-${{ github.event.pull_request.draft && 'draft' || 'ready' }}\n",
1370+
' group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.run_id }}\n')
1371+
self.assertIn('ci.yml must keep its exact draft and ready concurrency groups', findings)
1372+
1373+
def test_ci_split_requires_draft_guards_on_every_pr_suite_job(self):
1374+
self.assertEqual(ci_workflow_findings(*self.ci_sources(), others=self.other_sources()), [])
1375+
for job in ('preflight', *PR_PROOF_JOBS):
1376+
with self.subTest(job=job):
1377+
findings = self.mutated_file_findings(0, job, '!github.event.pull_request.draft && ', '')
1378+
finding = ('preflight must follow authz and changes and run even if classification fails'
1379+
if job == 'preflight' else
1380+
f'{job} must start alongside preflight, after authz and changes, '
1381+
'and skip only its classified path policy')
1382+
self.assertIn(finding, findings)
1383+
13411384
def test_ci_split_refuses_deleting_or_renaming_every_job(self):
13421385
ci = self.ci_sources()[0]
13431386
self.assertEqual(set(_jobs(ci)), set(CI_JOB_IDS))
@@ -2173,13 +2216,19 @@ def test_legacy_metadata_cannot_replace_runner_pins(self):
21732216
self.assertIn(f'ci.yml job preflight must run on {PR_RUNNER}', findings)
21742217

21752218
def test_ci_contract_pins_the_docs_only_skip(self):
2176-
"""Each proof job waits for authz and changes and skips only its docs-only
2177-
answer; preflight also waits for the class decision and always runs."""
2219+
"""Keep each pipeline's path policy and dependencies; PR suite jobs
2220+
also require a ready PR, and post-merge preflight always runs."""
21782221
original, others = self.ci_sources(), self.other_sources()
21792222
self.assertEqual(ci_workflow_findings(*original, others=others), [])
2180-
docs_skip = f' if: {DOCS_ONLY_SKIP}\n'
2223+
docs_skip = f' if: {ready_pr(DOCS_ONLY_SKIP)}\n'
21812224
mutations = []
21822225
for index, proofs, policy in ((0, PR_PROOF_JOBS, GITHUB_ONLY_SKIP), (6, PROOF_JOBS, DOCS_ONLY_SKIP)):
2226+
wrong_skips = [f' if: {ready_pr(condition) if index == 0 else condition}\n'
2227+
for condition in (
2228+
"${{ needs.changes.outputs.docs_only != 'true' }}",
2229+
"${{ !cancelled() && needs.changes.outputs.docs_only == 'false' }}",
2230+
"${{ !cancelled() && needs.changes.outputs.docs_only != 'true' }}",
2231+
)]
21832232
for job in proofs:
21842233
skip = f' if: {policy}\n'
21852234
refused = f'{job} must start alongside preflight, after authz and changes'
@@ -2188,21 +2237,19 @@ def test_ci_contract_pins_the_docs_only_skip(self):
21882237
(index, job, ' needs: [authz, changes]\n', ' needs: [changes, preflight]\n', refused),
21892238
(index, job, ' needs: [authz, changes]\n', ' needs: preflight\n', refused),
21902239
(index, job, skip, '', refused),
2191-
(index, job, skip, " if: ${{ needs.changes.outputs.docs_only != 'true' }}\n", refused),
2192-
(index, job, skip, " if: ${{ !cancelled() && needs.changes.outputs.docs_only == 'false' }}\n",
2193-
refused),
2240+
(index, job, skip, wrong_skips[0], refused),
2241+
(index, job, skip, wrong_skips[1], refused),
21942242
(index, job, skip, skip.replace("!= 'true'", "!= 'true' && github.event_name != 'push'"),
21952243
refused),
2196-
(index, job, skip, " if: ${{ !cancelled() && needs.changes.outputs.docs_only != 'true' }}\n",
2197-
refused)]
2244+
(index, job, skip, wrong_skips[2], refused)]
21982245
mutations += [
21992246
(0, 'preflight', ' needs: [authz, changes]\n', '',
22002247
'preflight must follow authz and changes'),
22012248
(0, 'preflight', ' needs: [authz, changes]\n', ' needs: build\n',
22022249
'preflight must follow authz and changes'),
2203-
(0, 'preflight', f' if: {AUTHZ_ALWAYS}\n', docs_skip,
2250+
(0, 'preflight', f' if: {PREFLIGHT_IF}\n', docs_skip,
22042251
'preflight must follow authz and changes'),
2205-
(0, 'preflight', f' if: {AUTHZ_ALWAYS}\n', '',
2252+
(0, 'preflight', f' if: {PREFLIGHT_IF}\n', '',
22062253
'preflight must follow authz and changes'),
22072254
(0, 'changes', ' steps:\n', ' needs: preflight\n steps:\n',
22082255
'changes must start after authz'),
@@ -2592,8 +2639,8 @@ def test_github_only_contract_refuses_previous_job_trust_and_reader_bypasses(sel
25922639
('build-gate', 'github_only = False', 'github_only = True'),
25932640
('build-gate', 'bool(paths) and all(', 'True or all('),
25942641
('build-gate', 'snapshot() != count', 'False'),
2595-
('build', GITHUB_ONLY_SKIP, DOCS_ONLY_SKIP),
2596-
('corpus-parity-subset', GITHUB_ONLY_SKIP, DOCS_ONLY_SKIP),
2642+
('build', GITHUB_ONLY_SKIP, ready_pr(DOCS_ONLY_SKIP)),
2643+
('corpus-parity-subset', GITHUB_ONLY_SKIP, ready_pr(DOCS_ONLY_SKIP)),
25972644
('preflight', ci_preflight.PREFLIGHT_TIER, "'github-only'"),
25982645
('changes', 'pull-requests: read', 'pull-requests: write'),
25992646
('build-gate', 'pull-requests: read', 'pull-requests: write')):

0 commit comments

Comments
 (0)