Repository navigation
Commit 0212cab
CI: PR tier excludes the slow rows on Release and kernel-only, with registration floors, and defers the slow verifier self-test classes (D1, CR-195) (#449)
On pull_request the Release matrix leg and the kernel-only job now pass
--exclude-label slow, like Debug and sanitizers already did. Push to main,
workflow_dispatch and release keep every row and every class, so coverage is
unchanged and only its timing moves.
- ci_preflight.py: the two policy checks that forbade this (only PR Debug
matrix jobs may exclude slow rows; kernel-only must keep its full
population) become one exact rule with must-fail forms; docs/ci.md says the
same. This REVERSES the previous policy. The must-fail test for the
full-population rule edits the `with:` block of the reusable-workflow job
corpus-parity-subset (it has no `steps:`).
- ci_verify.py: EXCLUDED_REGISTERED_MIN gains release 895 and kernel 337
(KERNEL_MIN_TESTS), derived on one registered-count basis from the PR-world
job data, to be confirmed by a lab ctest -N. 337 keeps today's 34-row slack
and stays below the 371 registered in all 16 sampled pull-request and push
runs; 'release' stays 895.
- test_ci_verify.py classes are partitioned into FAST (13) and SLOW (9) in
scripts/ci_verifier_classes.py; the preflight verifier-tests stage runs the
FAST classes on every pull request and the SLOW ones only when
scripts/ci_verifier_paths.py finds a path the verifier reads; any unknown
path or class runs. Non-PR events run everything. HistoricalProviderPins is
skipped without the private overlay and is FAST by decision.
- preflight has `needs: changes` and `if: ${{ !cancelled() }}` (pinned in
ci_preflight.py); the docs say so.
- doc anchors: PRIVATE_FIXTURE_PROFILE_FLOORS moved from ci_verify.py:789 to
ci_verify.py:802 (CONTRIBUTING.md, docs/pages/contributing-llm.md,
docs/pages/native-engine.md, docs/ci.md).
- docs/ci.md gains L6's rule for the deferred rows: a red on main in an
excluded row or class blocks the next release cut and is fixed or reverted
within the hour. AGENTS.md lists Release and kernel-only among the jobs that
exclude the slow rows.
- New tests assert that push, dispatch and release carry no exclusion and no
class filter.
Doc sentences this change removes or rewrites ON PURPOSE (doc-reverts check):
- "The PR registration floors after lane FIX-E1E2 are 773 for Debug and
sanitizers and 782 for native ..." (docs/ci.md), introduced by b2a578c
("docs: make every README and doc page true before the release
(DOCS-ENGINE) (#300)"). Those numbers were already stale against
scripts/ci_verify.py (876/867/963) and are now the keys of
EXCLUDED_REGISTERED_MIN.
- "`preflight` waits for nothing." (docs/ci.md), introduced by 133f571 ("1.0
prep: CI flake fix, docs-only CI skip, benchmark refresh (supersedes #303,
#304, #307)"): preflight now waits for `changes`.
- "Preflight starts at once, and the proof jobs start alongside it" and
"`changes` and `preflight` wait for nothing and run on every event"
(docs/ci.md), same reason as 133f571 above.
- "than documentation runs preflight, both Release jobs, kernel-only and the
parity subset with their full sets" and "both Debug jobs, sanitizers and
native-live run the registered set excluding the 29 CTest rows labelled
`slow`" (docs/ci.md): now stated for Release and kernel-only too.
- "A Release leg runs its full population on every event" and "Every event
runs the whole kernel set" (the `build` and `kernel-only` rows of the
timeout table in docs/ci.md): a pull request now excludes the slow rows.
- "The two new keys are derived, not measured: each is the registered count in
the sampled public pull-request runs" (docs/ci.md): the kernel key differs
from the Release one.
Note: scripts/release.yml runs no CTest and no ci_verify, so "release keeps
every row" means only that no --exclude-label is present in any release path;
the release workflow itself is unchanged.
The registration floors for Release and kernel-only (895 and 337) are derived,
not measured; a lab run of the release and kernel steps confirms or corrects
them.
This is the squash of 11cd290, 1b01244 and ef8fc5d (doc-reverts judges
every commit with its own message; the three were folded so that one message
names every sentence).
Co-authored-by: Claude Sonnet 5.5 <[email protected]>1 parent da9403f commit 0212cab
15 files changed
Lines changed: 762 additions & 64 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
43 | 43 | | |
44 | 44 | | |
45 | 45 | | |
46 | | - | |
47 | | - | |
| 46 | + | |
| 47 | + | |
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
| |||
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
| 62 | + | |
62 | 63 | | |
63 | 64 | | |
64 | 65 | | |
| |||
83 | 84 | | |
84 | 85 | | |
85 | 86 | | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
86 | 101 | | |
| 102 | + | |
| 103 | + | |
87 | 104 | | |
88 | 105 | | |
89 | 106 | | |
| |||
93 | 110 | | |
94 | 111 | | |
95 | 112 | | |
| 113 | + | |
96 | 114 | | |
97 | 115 | | |
98 | 116 | | |
| |||
237 | 255 | | |
238 | 256 | | |
239 | 257 | | |
240 | | - | |
| 258 | + | |
241 | 259 | | |
242 | 260 | | |
243 | 261 | | |
| |||
340 | 358 | | |
341 | 359 | | |
342 | 360 | | |
343 | | - | |
| 361 | + | |
344 | 362 | | |
345 | 363 | | |
346 | 364 | | |
| |||
380 | 398 | | |
381 | 399 | | |
382 | 400 | | |
383 | | - | |
| 401 | + | |
384 | 402 | | |
385 | 403 | | |
386 | 404 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
12 | | - | |
| 12 | + | |
13 | 13 | | |
14 | 14 | | |
15 | 15 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
159 | 159 | | |
160 | 160 | | |
161 | 161 | | |
162 | | - | |
| 162 | + | |
163 | 163 | | |
164 | 164 | | |
165 | 165 | | |
| |||
0 commit comments