Skip to content

Commit 9c96aaf

Browse files
committed
ci_verify: one floor table, the constants alias it, docs point at it
KERNEL_MIN_TESTS (325) and RELEASE_MIN_TESTS (895) were a dead second copy of the floors and lied: the --min-tests help said "the kernel profile defaults to 325, the release profile to 895", while the gate reads PRIVATE_FIXTURE_PROFILE_FLOORS (331/28 and 902/297). Both names now derive from that table, so a landed row is one edit; check_doc_lint reads its floor counts from the table too. The doc places that said "Adding rows means raising the floor in the same commit", "Raise the constant when a row lands", that the kernel and release floors are "KERNEL_MIN_TESTS ci_verify.py:321" and "RELEASE_MIN_TESTS ci_verify.py:595", and ci.yml's "The release profile fails below its CTest row floor (RELEASE_MIN_TESTS in scripts/ci_verify.py)" now send the reader to the table with the real instruction: re-measure the profile's own overlay-present Lab run and edit PRIVATE_FIXTURE_PROFILE_FLOORS. FloorDocsPointAtTheTable controls it: AST teeth that both names subscript the table, the built --help naming the table and not 325/895, and every doc place naming the table with the stale instructions gone. RED on dbf6536e6: 4 failures (Constant(325), Constant(895), 325 != 331, and the help text). The floor sentences this commit replaces are named by their writers: 4fb9db3 wrote CONTRIBUTING.md's "counts the CTest rows that actually ran and fails below a floor" sentence, 8b7965a wrote contributing-llm.md's two (the "KERNEL_MIN_TESTS ci_verify.py, RELEASE_MIN_TESTS ci_verify.py" parenthetical and the "floor — the minimum number of CTest rows a profile must actually run" definition), and e8dff5a wrote native-engine.md's "The profile carries a row floor:" lead. They leave because the numbers they cited were the dead second copy: each place now reads its number from, and points at, PRIVATE_FIXTURE_PROFILE_FLOORS, the one table the gate reads.
1 parent d25f01b commit 9c96aaf

7 files changed

Lines changed: 116 additions & 25 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -202,8 +202,9 @@ jobs:
202202
restore-keys: |
203203
ccache-${{ runner.os }}-${{ runner.arch }}-${{ matrix.build_type }}-
204204
205-
# The release profile fails below its CTest row floor (RELEASE_MIN_TESTS in
206-
# scripts/ci_verify.py); debug registers a subset of those rows.
205+
# The release profile fails below its CTest row floor (its row of the
206+
# PRIVATE_FIXTURE_PROFILE_FLOORS table in scripts/ci_verify.py);
207+
# debug registers a subset of those rows.
207208
- name: Verify (${{ matrix.build_type }})
208209
env:
209210
# A40 rev 7: the relative runtime witness gates on process CPU time,

‎CONTRIBUTING.md‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -158,11 +158,15 @@ What each gate refuses:
158158
### The floors
159159

160160
`ci_verify.py` counts the CTest rows that actually **ran** and fails below a
161-
floor — `KERNEL_MIN_TESTS` ci_verify.py:321 and `RELEASE_MIN_TESTS`
162-
ci_verify.py:595.
163-
A deleted or silently skipped row is a failure, not a quieter run. If your
164-
change adds rows, raise the floor in the same commit and say by how much; if it
165-
legitimately removes one, lower it deliberately and say why. `--min-tests`
161+
floor. The floors' numbers live in one place, the
162+
`PRIVATE_FIXTURE_PROFILE_FLOORS` ci_verify.py:780 table: per profile, `full`
163+
is its overlay-present registered population and `pinned` is how many rows of
164+
`tests/private_fixture_rows.txt` that registry registers, so with the private
165+
fixtures absent the floor is `full` minus `pinned`; every other profile
166+
carries its declared `Profile.min_tests`.
167+
A deleted or silently skipped row is a failure, not a quieter run. When rows
168+
land — or genuinely leave — re-measure the profile's own overlay-present Lab
169+
run and edit that table in the same commit, saying by how much. `--min-tests`
166170
overrides the floor for a local experiment; never in a commit.
167171

168172
## The parity contract

‎docs/ci.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -579,7 +579,7 @@ domain, a relative markdown link whose file or `#anchor` is missing, a cited
579579
exist, a `PF_ABI_VERSION` / `PF_NATIVE_API_VERSION` number the headers do not
580580
define or a "current release" that is not `VERSION`'s, a count the
581581
tree derives stated wrong (the `PF_API` declarations of the two C headers,
582-
`KERNEL_MIN_TESTS` / `RELEASE_MIN_TESTS`, the example manifest of
582+
the `PRIVATE_FIXTURE_PROFILE_FLOORS` floor counts, the example manifest of
583583
`check_native_include_independence.py`), and the marker itself sitting on a
584584
sentence that states a non-live epoch as today's. The
585585
marker on the line above is the escape hatch itself: this sentence names the

‎docs/pages/contributing-llm.md‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -132,9 +132,9 @@ finding to report, not a step to take.
132132
`scripts/check_doc_reverts.py`.
133133

134134
10. **A test row never silently disappears.** Each profile counts the rows that
135-
*ran* against a floor: `KERNEL_MIN_TESTS` ci_verify.py:321 and
136-
`RELEASE_MIN_TESTS` ci_verify.py:595. Adding rows means raising the floor
137-
in the same commit.
135+
*ran* against a floor: the `PRIVATE_FIXTURE_PROFILE_FLOORS`
136+
ci_verify.py:780 table. When rows land, re-measure the profile's own
137+
overlay-present run and edit that table in the same commit.
138138

139139
## The recipe for a lane
140140

@@ -281,8 +281,10 @@ measurement that produced it, so a later change to it is visible as a change to
281281
the record, not as an edit to a literal.
282282

283283
**floor** — the minimum number of CTest rows a profile must actually run
284-
(`KERNEL_MIN_TESTS` ci_verify.py:321, `RELEASE_MIN_TESTS` ci_verify.py:595). It
285-
counts rows that ran, so a skipped row does not pad it.
284+
(the `PRIVATE_FIXTURE_PROFILE_FLOORS` ci_verify.py:780 table: with the private
285+
overlay present, the profile's full registered population; without it, that
286+
population minus the pinned private-fixture rows it registers as SKIP
287+
placeholders). It counts rows that ran, so a skipped row does not pad it.
286288

287289
**receipt** — the recorded evidence an ABI-comparison row needs (a prepared
288290
historical provider). Under `PINEFORGE_REQUIRE_ABI_RECEIPTS`

‎docs/pages/native-engine.md‎

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -4612,19 +4612,24 @@ own features, not only the rows that happened to be source-free.
46124612

46134613
`python3 scripts/ci_verify.py kernel` is the profile that verifies the
46144614
kernel-only build (Release, live runner ON, tutorial OFF, source layer OFF);
4615-
CI runs it as the `kernel-only` job. The profile carries a **row floor**: `KERNEL_MIN_TESTS` in
4616-
`scripts/ci_verify.py` is the count the kernel-only CTest set is
4617-
expected to run, and the `ctest-floor` stage fails the run when fewer rows
4615+
CI runs it as the `kernel-only` job. The profile carries a **row floor**, its
4616+
number in the `PRIVATE_FIXTURE_PROFILE_FLOORS` ci_verify.py:780 table in
4617+
`scripts/ci_verify.py`: the count the kernel-only CTest set is
4618+
expected to run with the private overlay present, minus the pinned
4619+
private-fixture rows of `tests/private_fixture_rows.txt` it registers as SKIP
4620+
placeholders while the overlay is absent. The `ctest-floor` stage fails the
4621+
run when fewer rows
46184622
ran or CTest printed no count it can read, so a test TU that silently becomes
46194623
source-bound (or a filter that empties the suite) is a refusal rather than a
46204624
smaller green. A row counts only when its test ran to a verdict: a row CTest
46214625
skipped (`test_native_live_websocket` exits 77 wherever libcurl lacks
46224626
WebSocket support) or could not start is listed beside the count, in
46234627
`ci-logs/ctest-floor.log` and as `ctestSkipped` / `ctestNotRun` in
46244628
`ci-summary.json`, and never counted, so a row that stops executing fails the
4625-
floor as well. The `release` profile carries the same gate with
4626-
`RELEASE_MIN_TESTS`, so a row that leaves the default build fails CI too.
4627-
Raise the constant when a row lands; `--min-tests N` overrides it for one run
4629+
floor as well. The `release` profile carries the same gate with its own row in
4630+
the table, so a row that leaves the default build fails CI too.
4631+
When a row lands, re-measure the profile's own overlay-present Lab run and
4632+
edit the table; `--min-tests N` overrides the floor for one run
46284633
of any profile (`debug`, `sanitizers` and `native` carry no default floor).
46294634

46304635
### What the kernel-only archive still names

‎scripts/check_doc_lint.py‎

Lines changed: 15 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,9 @@
6868
* the ``PF_API`` declarations of ``include/pineforge/native_c_api.h`` (and,
6969
beside them, of ``pineforge.h``, the runtime set
7070
``scripts/check_c_abi_runtime.py`` pins, and their sum);
71-
* ``KERNEL_MIN_TESTS`` / ``RELEASE_MIN_TESTS`` in ``scripts/ci_verify.py``;
71+
* the kernel and release floor counts, read from the
72+
``PRIVATE_FIXTURE_PROFILE_FLOORS`` table in ``scripts/ci_verify.py`` (the
73+
``KERNEL_MIN_TESTS`` / ``RELEASE_MIN_TESTS`` names alias it);
7274
* the example manifest of ``scripts/check_native_include_independence.py``
7375
(how many sources, how many of them C).
7476
@@ -412,11 +414,18 @@ def read(rel: str) -> str | None:
412414
counts['runtime'] = len(re.findall(r'"[A-Za-z_][A-Za-z0-9_]*"', block.group(1)))
413415
verify = read('scripts/ci_verify.py')
414416
if verify is not None:
415-
for family, name in (('kernel-floor', 'KERNEL_MIN_TESTS'),
416-
('release-floor', 'RELEASE_MIN_TESTS')):
417-
value = re.search(r'^' + name + r'\s*=\s*(\d+)\s*$', verify, re.MULTILINE)
418-
if value:
419-
counts[family] = int(value.group(1))
417+
# KERNEL_MIN_TESTS / RELEASE_MIN_TESTS are aliases derived from the
418+
# one table; the counts come from the table itself, so a doc number
419+
# is checked against the place the numbers really live.
420+
table = re.search(r'^PRIVATE_FIXTURE_PROFILE_FLOORS\s*=\s*\{(.*?)\}',
421+
verify, re.MULTILINE | re.DOTALL)
422+
if table:
423+
for family, profile in (('kernel-floor', 'kernel'),
424+
('release-floor', 'release')):
425+
value = re.search(r"'" + profile + r"':\s*\((\d+),\s*\d+\)",
426+
table.group(1))
427+
if value:
428+
counts[family] = int(value.group(1))
420429
manifest = read('scripts/check_native_include_independence.py')
421430
if manifest is not None:
422431
block = re.search(r'^NATIVE_EXAMPLES\s*=\s*\((.*?)^\)', manifest,

‎scripts/test_ci_verify.py‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
"""
1010
from __future__ import annotations
1111

12+
import ast
1213
import json
1314
import os
1415
from pathlib import Path
@@ -3105,6 +3106,75 @@ def test_red_an_unreadable_pin_fails_the_exclusion_closed(self):
31053106
self.assertEqual(driver.stages[0]['name'], 'ctest-exclusion')
31063107

31073108

3109+
class FloorDocsPointAtTheTable(unittest.TestCase):
3110+
"""N3: the floors have ONE source, and the prose says where it is.
3111+
3112+
KERNEL_MIN_TESTS / RELEASE_MIN_TESTS were a dead second copy: the
3113+
--min-tests help quoted 325 and 895 (false), and six doc places told
3114+
contributors to raise the constants. Both names now derive from
3115+
PRIVATE_FIXTURE_PROFILE_FLOORS, and this control fails when the help
3116+
text or a doc place goes back to a number of its own.
3117+
"""
3118+
3119+
STALE_INSTRUCTIONS = (
3120+
'raise the floor in the same commit',
3121+
'Raise the constant when a row lands',
3122+
'Adding rows means raising the floor',
3123+
'raise the floor',
3124+
)
3125+
DOC_PLACES = ('CONTRIBUTING.md', 'docs/pages/contributing-llm.md',
3126+
'docs/pages/native-engine.md', 'docs/ci.md',
3127+
'.github/workflows/ci.yml')
3128+
3129+
def test_the_two_names_derive_from_the_table(self):
3130+
# AST teeth: the right-hand sides must subscript the table, so
3131+
# writing a number back into either name fails here.
3132+
module = ast.parse((ROOT / 'scripts' / 'ci_verify.py').read_text())
3133+
assigned = {}
3134+
for node in module.body:
3135+
if isinstance(node, ast.Assign) and len(node.targets) == 1:
3136+
target = node.targets[0]
3137+
if isinstance(target, ast.Name) and target.id in (
3138+
'KERNEL_MIN_TESTS', 'RELEASE_MIN_TESTS'):
3139+
assigned[target.id] = node.value
3140+
self.assertEqual(set(assigned), {'KERNEL_MIN_TESTS', 'RELEASE_MIN_TESTS'})
3141+
for name, value in assigned.items():
3142+
with self.subTest(name=name):
3143+
self.assertIsInstance(value, ast.Subscript)
3144+
base = value
3145+
while isinstance(base, ast.Subscript):
3146+
base = base.value
3147+
self.assertIsInstance(base, ast.Name)
3148+
self.assertEqual(base.id, 'PRIVATE_FIXTURE_PROFILE_FLOORS')
3149+
self.assertEqual(ci_verify.KERNEL_MIN_TESTS,
3150+
ci_verify.PRIVATE_FIXTURE_PROFILE_FLOORS['kernel'][0])
3151+
self.assertEqual(ci_verify.RELEASE_MIN_TESTS,
3152+
ci_verify.PRIVATE_FIXTURE_PROFILE_FLOORS['release'][0])
3153+
3154+
def test_the_help_text_names_the_table_and_no_stale_number(self):
3155+
result = subprocess.run(
3156+
[sys.executable, '-B', str(ROOT / 'scripts' / 'ci_verify.py'), '--help'],
3157+
capture_output=True, text=True, timeout=60)
3158+
self.assertEqual(result.returncode, 0, result.stderr)
3159+
self.assertIn('PRIVATE_FIXTURE_PROFILE_FLOORS', result.stdout)
3160+
self.assertNotIn('defaults to 325', result.stdout)
3161+
self.assertNotIn('895', result.stdout)
3162+
3163+
def test_every_doc_place_points_at_the_table(self):
3164+
for rel in self.DOC_PLACES:
3165+
text = (ROOT / rel).read_text()
3166+
with self.subTest(place=rel):
3167+
self.assertIn('PRIVATE_FIXTURE_PROFILE_FLOORS', text)
3168+
for stale in self.STALE_INSTRUCTIONS:
3169+
self.assertNotIn(stale, text)
3170+
for rel in ('docs/pages/contributing-llm.md', 'docs/pages/native-engine.md',
3171+
'CONTRIBUTING.md', '.github/workflows/ci.yml'):
3172+
with self.subTest(place=rel):
3173+
text = (ROOT / rel).read_text()
3174+
self.assertNotIn('KERNEL_MIN_TESTS', text)
3175+
self.assertNotIn('RELEASE_MIN_TESTS', text)
3176+
3177+
31083178
class ExamplesAssertLive(unittest.TestCase):
31093179
run_profile = DriverOrderingAndAggregation.run_profile
31103180

0 commit comments

Comments
 (0)