Skip to content

Commit 0212cab

Browse files
luisleo526claude
andauthored
CI: PR tier excludes the slow rows on Release and kernel-only, with registration floors, and defers the slow verifier self-test classes (D1, CR-195) (#449)
On pull_request the Release matrix leg and the kernel-only job now pass --exclude-label slow, like Debug and sanitizers already did. Push to main, workflow_dispatch and release keep every row and every class, so coverage is unchanged and only its timing moves. - ci_preflight.py: the two policy checks that forbade this (only PR Debug matrix jobs may exclude slow rows; kernel-only must keep its full population) become one exact rule with must-fail forms; docs/ci.md says the same. This REVERSES the previous policy. The must-fail test for the full-population rule edits the `with:` block of the reusable-workflow job corpus-parity-subset (it has no `steps:`). - ci_verify.py: EXCLUDED_REGISTERED_MIN gains release 895 and kernel 337 (KERNEL_MIN_TESTS), derived on one registered-count basis from the PR-world job data, to be confirmed by a lab ctest -N. 337 keeps today's 34-row slack and stays below the 371 registered in all 16 sampled pull-request and push runs; 'release' stays 895. - test_ci_verify.py classes are partitioned into FAST (13) and SLOW (9) in scripts/ci_verifier_classes.py; the preflight verifier-tests stage runs the FAST classes on every pull request and the SLOW ones only when scripts/ci_verifier_paths.py finds a path the verifier reads; any unknown path or class runs. Non-PR events run everything. HistoricalProviderPins is skipped without the private overlay and is FAST by decision. - preflight has `needs: changes` and `if: ${{ !cancelled() }}` (pinned in ci_preflight.py); the docs say so. - doc anchors: PRIVATE_FIXTURE_PROFILE_FLOORS moved from ci_verify.py:789 to ci_verify.py:802 (CONTRIBUTING.md, docs/pages/contributing-llm.md, docs/pages/native-engine.md, docs/ci.md). - docs/ci.md gains L6's rule for the deferred rows: a red on main in an excluded row or class blocks the next release cut and is fixed or reverted within the hour. AGENTS.md lists Release and kernel-only among the jobs that exclude the slow rows. - New tests assert that push, dispatch and release carry no exclusion and no class filter. Doc sentences this change removes or rewrites ON PURPOSE (doc-reverts check): - "The PR registration floors after lane FIX-E1E2 are 773 for Debug and sanitizers and 782 for native ..." (docs/ci.md), introduced by b2a578c ("docs: make every README and doc page true before the release (DOCS-ENGINE) (#300)"). Those numbers were already stale against scripts/ci_verify.py (876/867/963) and are now the keys of EXCLUDED_REGISTERED_MIN. - "`preflight` waits for nothing." (docs/ci.md), introduced by 133f571 ("1.0 prep: CI flake fix, docs-only CI skip, benchmark refresh (supersedes #303, #304, #307)"): preflight now waits for `changes`. - "Preflight starts at once, and the proof jobs start alongside it" and "`changes` and `preflight` wait for nothing and run on every event" (docs/ci.md), same reason as 133f571 above. - "than documentation runs preflight, both Release jobs, kernel-only and the parity subset with their full sets" and "both Debug jobs, sanitizers and native-live run the registered set excluding the 29 CTest rows labelled `slow`" (docs/ci.md): now stated for Release and kernel-only too. - "A Release leg runs its full population on every event" and "Every event runs the whole kernel set" (the `build` and `kernel-only` rows of the timeout table in docs/ci.md): a pull request now excludes the slow rows. - "The two new keys are derived, not measured: each is the registered count in the sampled public pull-request runs" (docs/ci.md): the kernel key differs from the Release one. Note: scripts/release.yml runs no CTest and no ci_verify, so "release keeps every row" means only that no --exclude-label is present in any release path; the release workflow itself is unchanged. The registration floors for Release and kernel-only (895 and 337) are derived, not measured; a lab run of the release and kernel steps confirms or corrects them. This is the squash of 11cd290, 1b01244 and ef8fc5d (doc-reverts judges every commit with its own message; the three were folded so that one message names every sentence). Co-authored-by: Claude Sonnet 5.5 <[email protected]>
1 parent da9403f commit 0212cab

15 files changed

Lines changed: 762 additions & 64 deletions

‎.github/workflows/ci.yml‎

Lines changed: 23 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -43,8 +43,8 @@ jobs:
4343
# Whether the change is documentation only; scripts/ci_docs_only.py holds the
4444
# rule, and every doubt answers no. On a docs-only pull request or push the
4545
# proof jobs below skip and the build aggregate accepts exactly that skip.
46-
# Preflight, where the documentation guards run, does not wait for it, and
47-
# docs.yml builds the site as before. A few seconds of git: the light pool
46+
# Preflight consumes its independent verifier-class answer, and docs.yml
47+
# builds the site as before. A few seconds of git: the light pool
4848
# class, with the public fragment's hosted fallback.
4949
changes:
5050
runs-on: >-
@@ -59,6 +59,7 @@ jobs:
5959
timeout-minutes: 360
6060
outputs:
6161
docs_only: ${{ steps.classify.outputs.docs_only }}
62+
verifier_slow: ${{ steps.verifier.outputs.verifier_slow }}
6263
steps:
6364
# The tested commit and its parents: a pull request's merge and the base
6465
# it merged into, a push's commit and its parent.
@@ -83,7 +84,23 @@ jobs:
8384
echo "docs_only=false" >> "$GITHUB_OUTPUT"
8485
fi
8586
87+
# This helper uses the base ci_docs_only.py staged above to read the
88+
# same tested diff. A base without the new rule runs every class.
89+
- name: Classify verifier self-tests
90+
id: verifier
91+
env:
92+
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
93+
run: |
94+
if git show HEAD^1:scripts/ci_verifier_paths.py > "$RUNNER_TEMP/ci_verifier_paths.py"; then
95+
python3 "$RUNNER_TEMP/ci_verifier_paths.py" --root . --github-output "$GITHUB_OUTPUT"
96+
else
97+
echo "verifier SLOW: the base holds no scripts/ci_verifier_paths.py"
98+
echo "verifier_slow=true" >> "$GITHUB_OUTPUT"
99+
fi
100+
86101
preflight:
102+
needs: changes
103+
if: ${{ !cancelled() }}
87104
runs-on: >-
88105
${{ github.event_name != 'pull_request_target' &&
89106
github.actor != 'dependabot[bot]' && github.triggering_actor != 'dependabot[bot]' &&
@@ -93,6 +110,7 @@ jobs:
93110
fromJSON('["self-hosted","pf-ci","pf-ci-x64"]') || 'ubuntu-24.04' }}
94111
env:
95112
PF_CI_LABELS: '["self-hosted","pf-ci","pf-ci-x64"]'
113+
PINEFORGE_VERIFIER_SLOW: ${{ needs.changes.result == 'success' && needs.changes.outputs.verifier_slow || 'true' }}
96114
# 360 minutes is GitHub's job maximum and deliberately the only limit:
97115
# its verifier-tests stage runs test_ci_verify.py across parallel unittest
98116
# shards, and no stage of this preflight carries a timeout of its own any
@@ -237,7 +255,7 @@ jobs:
237255
# ratio -- for a runner whose CPU accounting is unusable; record the
238256
# measured reason here and in docs/ci.md before setting it.
239257
PINEFORGE_RUNTIME_BUDGET_CANDIDATE_ONLY: '0'
240-
run: python3 scripts/ci_verify.py ${{ matrix.build_type == 'Release' && 'release' || 'debug' }} --build-dir build --jobs "$(getconf _NPROCESSORS_ONLN)" --ccache ${{ github.event_name == 'pull_request' && matrix.build_type == 'Debug' && '--exclude-label slow' || '' }}
258+
run: python3 scripts/ci_verify.py ${{ matrix.build_type == 'Release' && 'release' || 'debug' }} --build-dir build --jobs "$(getconf _NPROCESSORS_ONLN)" --ccache ${{ github.event_name == 'pull_request' && '--exclude-label slow' || '' }}
241259

242260
- name: Stage and summarize diagnostics
243261
if: always()
@@ -340,7 +358,7 @@ jobs:
340358
fromJSON('["self-hosted","pf-ci","pf-ci-x64-16"]') || 'ubuntu-24.04' }}
341359
env:
342360
PF_CI_LABELS: '["self-hosted","pf-ci","pf-ci-x64-16"]'
343-
# 22.3 min on the standard runner at 0d76a099; every event runs the whole
361+
# PRs exclude the measured slow rows; push and dispatch run the whole
344362
# kernel set. 360 minutes is the platform maximum, the only hang catcher
345363
# (owner rule, 2026-10-12).
346364
timeout-minutes: 360
@@ -380,7 +398,7 @@ jobs:
380398
uses: ./.github/actions/setup-live-curl
381399

382400
- name: Verify (kernel)
383-
run: python3 scripts/ci_verify.py kernel --build-dir build-kernel --jobs "$(getconf _NPROCESSORS_ONLN)" --ccache --curl-dir "${{ steps.curl-deps.outputs.curl-dir }}"
401+
run: python3 scripts/ci_verify.py kernel --build-dir build-kernel --jobs "$(getconf _NPROCESSORS_ONLN)" --ccache --curl-dir "${{ steps.curl-deps.outputs.curl-dir }}" ${{ github.event_name == 'pull_request' && '--exclude-label slow' || '' }}
384402

385403
- name: Stage and summarize diagnostics
386404
if: always()

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ Both C++ unit tests and full corpus verification must pass.
99
The merge ruleset requires `pineforge/verify` and `pineforge/parity` commit
1010
statuses on the PR head's exact tree. The maintainers post them from their own (private)
1111
verification tooling after full verification on their x86-64 build hosts and a
12-
parity verdict; an outside contributor cannot post them. GitHub Actions is advisory: PR Debug, sanitizers and native jobs
12+
parity verdict; an outside contributor cannot post them. GitHub Actions is advisory: PR Release, Debug, kernel-only, sanitizers and native jobs
1313
exclude the measured `slow` CTest rows; push-to-main and manual CI dispatch
1414
run every row, and a documentation-only change runs preflight alone. The commands below run the full sets.
1515

‎CONTRIBUTING.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -159,7 +159,7 @@ What each gate refuses:
159159

160160
`ci_verify.py` counts the CTest rows that actually **ran** and fails below a
161161
floor. The floors' numbers live in one place, the
162-
`PRIVATE_FIXTURE_PROFILE_FLOORS` ci_verify.py:789 table: per profile, `full`
162+
`PRIVATE_FIXTURE_PROFILE_FLOORS` ci_verify.py:802 table: per profile, `full`
163163
is its overlay-present registered population and `pinned` is how many rows of
164164
`tests/private_fixture_rows.txt` that registry registers, so with the private
165165
fixtures absent the floor is `full` minus `pinned`; every other profile

0 commit comments

Comments
 (0)