Repository navigation
tests: robust lower-feed shape sampling; release bump leaves replay goldens unchanged - #647
Conversation
…rline one once test_native_intrabar_lower_lookup compares the time of a 4x lower feed with a 6.0 bound. On a shared CI runner each leg is the best of three samples with independent scheduler noise, so a true ratio near 4 occasionally read 6.08 and failed main's post-merge run once (4000 bars 0.1140 s, 16000 bars 0.6934 s), then passed on a re-run. Take the best of five user-CPU samples per leg, and give a ratio in [6, 12) exactly one fresh measured pair, keeping the smaller ratio. A ratio of 12 or more (a quadratic lookup measures about 16) is judged as measured with no retry, and the bound stays 6.0 with CHECK(ratio < kShapeBound) as the only verdict, so a persistently high ratio still fails. The decision is a small header-only helper in tests/ratio_timing.hpp, exercised inside the same test binary on fixed sample sets: the observed noisy pair with a quiet re-measure passes, the same pair with a high re-measure still fails, a quadratic pair is not retried, and a clean linear pair is not retried. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
…the replay goldens' masked bytes unchanged The replay-off goldens went red at the v1.8.0 and v1.9.0 release commits because bytes that depend on the engine version were pinned. The masks now remove that dependence, but the unit rows only bump synthetic versions. Add rows that read the real VERSION file, bump it to the next patch, minor and major, and require the masked default report, its sha256 and its length, and the masked sink stream, its sha256 and its length, to equal the ones at the current version. Every version is derived from VERSION; nothing is a literal copy of today's number. The rows are not vacuous: the raw bytes and the raw sink trailer digest differ at each bumped version, and a version string moved into a sink body row is still visible after masking, so a mask that hid body bytes would fail. The shipped docker/replay_off_golden.json is checked for structure only (three scenarios, a 64-hex sha256 and a positive byte count each); its digests belong to real runner output and are not compared with synthetic bytes. Reverting the numeric engine masking, or reverting the sink trailer recomputation, turns these rows red. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Graphify reviewed this change.
No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.
Not checked: tests were not run; no formal proof of the changed code.
Graphify review — findings
Adds a one-shot re-measurement to the lower-feed linearity timing test: a ratio in [6, 12) gets one fresh pair and keeps the smaller ratio, while a clearly quadratic ratio (≥12) or a passing one stands as measured. This targets shared-runner CI flakiness without loosening the x6 bound. The timing samples per leg also go from best-of-three to best-of-five user-CPU, and the new ratio_with_remeasurement helper in ratio_timing.hpp has unit tests for the noisy, persistent, quadratic and linear cases. Adds replay-off golden tests that pin release-version bumps (patch, minor and major, derived from the repo VERSION) as invisible in the masked default report and in the masked primary sink, including the recomputed trailer digest. A version change inside the sink body stays visible after masking, so the mask can't hide real differences.
No blocking issues surfaced. 1 lower-confidence candidate did not survive cross-model review.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 66 functions depend on the 66 functions this change touches.
Health — grade A; 1 existing hotspot(s) in the area this change touches (pre-existing, not introduced here):
main()— 0 callers, 6 callees (medium)
Verification — 66 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 66 function(s) in the blast radius were not formally verified this run
Two test-only changes, on current main (supersedes #644 and #645):
tests: sample the lower-feed shape ratio more, and re-measure a borderline one once
test_native_intrabar_lower_lookup compares the time of a 4x lower feed with a 6.0 bound.
On a shared CI runner each leg is the best of three samples with independent scheduler
noise, so a true ratio near 4 occasionally read 6.08 and failed main's post-merge run once
(4000 bars 0.1140 s, 16000 bars 0.6934 s), then passed on a re-run.
Take the best of five user-CPU samples per leg, and give a ratio in [6, 12) exactly one
fresh measured pair, keeping the smaller ratio. A ratio of 12 or more (a quadratic lookup
measures about 16) is judged as measured with no retry, and the bound stays 6.0 with
CHECK(ratio < kShapeBound) as the only verdict, so a persistently high ratio still fails.
The decision is a small header-only helper in tests/ratio_timing.hpp, exercised inside the
same test binary on fixed sample sets: the observed noisy pair with a quiet re-measure
passes, the same pair with a high re-measure still fails, a quadratic pair is not retried,
and a clean linear pair is not retried.
tests: prove under the repository VERSION that a release bump leaves the replay goldens' masked bytes unchanged
The replay-off goldens went red at the v1.8.0 and v1.9.0 release commits because bytes that
depend on the engine version were pinned. The masks now remove that dependence, but the unit
rows only bump synthetic versions. Add rows that read the real VERSION file, bump it to the
next patch, minor and major, and require the masked default report, its sha256 and its length,
and the masked sink stream, its sha256 and its length, to equal the ones at the current
version. Every version is derived from VERSION; nothing is a literal copy of today's number.
The rows are not vacuous: the raw bytes and the raw sink trailer digest differ at each bumped
version, and a version string moved into a sink body row is still visible after masking, so
a mask that hid body bytes would fail. The shipped docker/replay_off_golden.json is checked for
structure only (three scenarios, a 64-hex sha256 and a positive byte count each); its digests
belong to real runner output and are not compared with synthetic bytes.
Reverting the numeric engine masking, or reverting the sink trailer recomputation, turns these
rows red.
No product source changes: only scripts/test_check_replay_off_golden.py, tests/ratio_timing.hpp and tests/test_native_intrabar_lower_lookup.cpp.
🤖 Generated with Claude Code