Skip to content

Add generated CLI coverage for map pair limits - #783

Closed
luisleo526 wants to merge 1 commit into
mainfrom
win/cr559-cli-pair-1108
Closed

luisleo526 wants to merge 1 commit into
mainfrom
win/cr559-cli-pair-1108

Conversation

@luisleo526

Copy link
Copy Markdown
Collaborator

What

Adds an end-to-end regression for the map pair limit. It runs the ordinary command-line runner (docker/run_json.py) against one strategy library built from a committed, genuinely generated C++ fixture. The Pine probe, the generator driver and a provenance file (exact commits, hashes and the command template) sit next to it in tests/fixtures/cr559_cli_pair/.

Seven cases on a four-bar feed. The oracle (tests/test_cr559_map_capacity_cli.py) only reads the files the shell wrote and recomputes what to expect from the inputs; it never runs the CLI:

  • finite limit 50000 with 50000 pairs: succeeds;
  • finite limit 50000 with 50001 pairs: exits 1 with the typed pine_map_capacity_exceeded error (bar 1, 50001 against 50000). This is a control for the finite policy, not evidence of a default-policy bug;
  • limit omitted, and explicit 0, each with 50003 and with 50009 pairs: succeed, with results that depend on the input;
  • a negative limit: exits 2 with run_request_invalid.

No product default changes. src/, include/ and docker/ are untouched.

Registration

The module library is built in the existing cr559-map-capacity fence of tests/CMakeLists.txt. The test row test_cr559_map_capacity_cli and its slow label are in tests/cr150.d/cr559-map-capacity.cmake. Library and row exist only on UNIX with the source layer on and with sanitizers and coverage off, like the other Python-loaded modules.

The row runs seven real CLI cases (about 95 minutes measured on a Linux arm64 host), so it carries the slow label and joins the explicit slow-row pin: 89 to 90 in scripts/ci_verify.py, in its self-test and in docs/ci.md. The pin grows by one name; the documentation assertions in scripts/test_ci_preflight.py follow the rewritten text; no floor or allowlist entry changes.

tests/cr559_cli_pair.sh now prints a message naming the path before exiting 1 when the output directory already holds collection.json or the module is missing or empty. Its commands, inputs, output paths and expected values are unchanged.

Checks

Run on the branch head before it was merged with current main, on an ordinary Linux arm64 build host:

  • The lane-file checker passes, the verifier's own slow-pin arithmetic (set(excluded) - PINNED_SLOW_TEST_ROWS over ctest -N -L slow) leaves nothing unpinned, and the 20 CI policy rows (ci_verify and ci_preflight slices, the lane checker and its controls) pass.
  • Registration read from ctest -N in four configurations (Debug with defaults, with sanitizers, with coverage, and with the source layer off): the row exists only in the first.
  • Mutation control on a scratch copy, not committed: making the default limit 50000 turns the omitted-limit case into exit 1 with the typed capacity error, byte-identical to the finite-limit refusal for the same input; the unmodified code succeeds on that input.
  • The registered row itself, run once through CTest from a fresh Release build of that head (the library is built as libcr559_cli_pair.so and loaded by path): test_cr559_map_capacity_cli passed in 6166 s with all seven cases as listed above (exit codes 0, 1, 0, 0, 0, 0 and 2, in that order), together with its native neighbour test_cr559_map_capacity_error_context: 2 of 2 passed, none skipped. The earlier 93.6 minute run measured the previous registration and shell.

Not covered

  • Hosted CI and the parity gate run on this PR. No historical baseline failure is claimed: the base already defaults to unlimited.
  • On Linux the module is libcr559_cli_pair.so; the test passes its path from the build and the runner loads it by path.

🤖 Generated with Claude Code

Add an end-to-end regression for the map pair limit: a committed, generated
strategy library driven through the ordinary run_json command line in seven
cases (finite boundary and refusal, omitted and explicit zero limit with two
pair counts each, and a negative option), with an oracle that recomputes what
to expect from the inputs. The row is slow-labelled, registered from its
tests/cr150.d lane file with its library in the owning fence, guarded for
sanitizer and coverage builds, and pinned (89 to 90) in the explicit
slow-row policy.

docs/ci.md: the slow-label bound paragraph and the slow-label inventory
paragraph, introduced by 69cddc9 (#679), now state the size of the pinned set
(90 names, including test_cr559_map_capacity_cli) instead of the retired
43 + 28 = 71 breakdown.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
require(stderr.read_bytes() == b"", f"{name}: unexpected stderr")
lines = stdout.read_text(encoding="utf-8").splitlines()
require(len(lines) == 1 and bool(lines[0]), f"{name}: expected one JSON line")
document = json.loads(lines[0], parse_constant=lambda value: reject(value))

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 1 advisory finding(s) below merit a look before merge.

Not checked: tests were not run; no formal proof of the changed code.


Graphify review — findings

Adds a slow-labelled CTest, test_cr559_map_capacity_cli, which builds a committed generated.cpp fixture into a cr559_cli_pair shared library and drives docker/run_json.py through finite-boundary, finite-refusal, omitted/zero-limit and negative-option cases via the new tests/cr559_cli_pair.sh. The test is registered only when sanitizers and coverage are off, and the script refuses to run if the output directory already holds a collection.json, so a rerun in the same build tree fails instead of reusing prior artifacts. Pins the new name in PINNED_SLOW_TEST_ROWS, taking the set from 89 to 90, so ctest-exclusion and shard discovery accept it. docs/ci.md now states the pinned count outright instead of the stale 43 + 28 = 71 breakdown, and the two self-tests that assert the count and doc wording are updated to match. Adds tests/cr559_cli_pair_generate.py, a worker-only driver that runs the real gate/glue.py transpile_json from a declared codegen archive and rejects imports that resolve outside it; it crashes rather than raising its contract RuntimeError when diagnostics are missing or null.

Worth a look

  • CLI pair test fails on every rerun in the same build tree — tests/cr150.d/cr559-map-capacity.cmake:21 · Escalate · medium
    • NOT verified (no proof, no reproducing execution) — needs human review

Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 1420 functions depend on the 1401 functions this change touches.

Health — grade B; 6 existing hotspot(s) in the area this change touches (pre-existing, not introduced here):

  • check_commands() — 22 callers, 5 callees (high)
  • validate_config() — 24 callers, 4 callees (high)
  • main() — 1 callers, 13 callees (high)
  • archive_inputs() — 1 callers, 6 callees (medium)
  • main() — 0 callers, 15 callees (high)
  • validate_ctest_shard_union() — 0 callers, 7 callees (medium)

Verification — 1420 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 1415 function(s) in the blast radius were not formally verified this run

Docs that may be stale (advisory)

…and 10 more.

luisleo526 added a commit that referenced this pull request Nov 8, 2026
…n, trade text and RCI changes (#788)

## Summary

This branch combines seven source heads into one change on top of main (`755e6658`), so that the build, the parity sweep and the review each run once. Every source head was merged with an ordinary merge commit: nothing was rebased, nothing was squashed inside the branch, no conflict occurred in these merges and nothing was resolved by hand. Every source head contains main (`755e6658`) except the trade text channel's, which contains the main before one documentation-only commit (`63f3978a`); the combined branch contains main. When this change merges, pull requests 781, 758, 780 and 783 are closed with a link to it.

A moving average change (head `4c5c65f4`, "Compute SMA from a correctly rounded window sum") was merged into this branch between the trade text channel and #783 and is taken out again by the last commit of the branch, a plain revert of that merge (`git revert -m 1`, no conflict). The parity measurement of the branch with it showed 40 probes whose graded tier falls against their committed TradingView tapes, every one of them calling `ta.sma`; a measurement of main plus only that change gave the same 40 results, and no other probe changed. The change is not part of this merge; whether it returns, and in what form, is decided separately. The tree of this branch equals the tree of the seven heads below merged on main without it.

PR type: code (engine sources, tests, CI scripts and documentation).

Fixes #779

## Sources

| Source | Title | Source head |
|---|---|---|
| #781 | ci: validate split releases and retained smoke output | `130bda2db570bd63bdec57b6b5fdd3cabacab7fc` |
| branch | Validate corpus records and gitlink results in the pull-request checks | `46bb21fb169091722d49d3b2f2e4f1f589148166` |
| #758 | test(ci): register rerun-preempted controller tests | `348a89766e0ff23f88b4837bce80957f7635ef8e` |
| branch | Add an opt-in trade text companion channel | `5930d7f8d12010d67c77a098804eb6cd295095db` |
| #783 | Add generated CLI coverage for map pair limits | `163abf58f1a73c7ec9160120774954fb29f4bd2d` |
| branch | Avoid copying RCI price history during ranking | `022750f8451aadd93066ffad3b54695e23956116` |
| #780 | Make nested consumer state construction portable to Clang | `000e2dac0e8e5e73fe53551adec7fbdf3f251af6` |

The head listed for the corpus validation change already contains the heads listed for #781 and #758, so merging it brings both along and the separate merge of the #758 head changes nothing (it is listed because its pull request closes with this change). The merges that this branch makes of the source heads resolve no conflict by hand. The source histories themselves carry six merges that were resolved by hand in their own branches, before the heads were handed over: three in the history of the corpus validation head and three in the history of the trade text channel head. Corpus validation: `d482eb37` (`scripts/test_ci_preflight.py`: the stage-inventory counts between the registration of the rerun-controller tests and the corpus validation job), `f7519bb8` (`scripts/ci_preflight.py` and `scripts/test_ci_preflight.py`) and `cecedc0e` (`CHANGELOG.md`, `ci/cr150-source-bounds.yaml`, `scripts/test_cr150_source_gate.py` and `tests/private_fixture_reads.evidence`). Trade text channel: `09c9fb52` (`CHANGELOG.md` and `runner/main.cpp`), `8c17d14e` and `5930d7f8` (both in `ci/cr150-source-bounds.yaml` and `scripts/test_cr150_source_gate.py`). The head listed for #758 is that pull request's branch merged once with main; the head published on #758 is older. The head listed for #780 is the head of its branch (the construction fixes and the refreshed source citations in three documentation pages).

## How it was built

The merges were made in the order of the table, with the moving average change between the trade text channel and #783 (see above), except that the separate merge of the #758 head was skipped because the corpus validation head contains it. Relative to main, 52 files change. Besides the files that #781 and #758 share with the corpus validation head, three files are touched by more than one source and were merged automatically without a conflict: `docs/ci.md` (the corpus validation change and #783), `docs/design/native-feature-parity.md` (the trade text channel and #780) and `scripts/test_ci_preflight.py` (#781, #758, the corpus validation change and #783). For every source, the changed lines of the files that only it touches equal the changed lines of this branch, and for every shared file the union of the sources' changed lines equals the branch's; an independent chain of the same merges in the reverse order gives the same tree.

docs/ci.md: the slow-label bound paragraph and the slow-label inventory
paragraph, introduced by 69cddc9 (#679), now state the size of the pinned set
(90 names, including test_cr559_map_capacity_cli) instead of the retired
43 + 28 = 71 breakdown.

docs/ci.md: the three sentences of 42527b6 (#704) that said gke-pr.yml admits exactly two variants, that OLD is the shape without the arm job kept so earlier workflow files still pass, that NEW requires the exact arm classifier output, and that hybrids and extra jobs, outputs, needs or arm steps are refused while OLD retains its original exact rules and templates for a workflow without the arm job, are rewritten to cover the arm and the corpus variants.

## What was done before merge

Hosted CI was green on this head. One parity sweep against the active baseline measured this exact tree: 7,989 probes compared, no probe changed tier, no leaver, no regression, and the gate returned its neutral pass. One review of the combined diff, which cites each source, and of the revert commit gave a GO on this head and tree. The results already recorded on the individual pull requests and branches are not repeated here and do not stand in for these. The moving average change is not in this merge: it was merged into this branch and taken out again by the last commit, a plain revert.

The full post-merge workflow was also run on this head (run 42452226570). It failed exactly the twelve jobs that fail on main `755e6658` (run 42435140307): the eight sanitizer shards and the live-sanitizer job, each stopping at the final link with "No space left on device", and their aggregate jobs. Every other job passed, and no job failed that passes on main.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@luisleo526

Copy link
Copy Markdown
Collaborator Author

This change is merged as part of #788 (squash commit 0c8e1a8), which combined it with the other sources listed there. Closing in favour of that merge.

@luisleo526 luisleo526 closed this Nov 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant