Repository navigation
tests: a release version bump must not move the replay goldens - #645
luisleo526 wants to merge 1 commit into
Conversation
…the replay goldens' masked bytes unchanged The replay-off goldens went red at the v1.8.0 and v1.9.0 release commits because bytes that depend on the engine version were pinned. The masks now remove that dependence, but the unit rows only bump synthetic versions. Add rows that read the real VERSION file, bump it to the next patch, minor and major, and require the masked default report, its sha256 and its length, and the masked sink stream, its sha256 and its length, to equal the ones at the current version. Every version is derived from VERSION; nothing is a literal copy of today's number. The rows are not vacuous: the raw bytes and the raw sink trailer digest differ at each bumped version, and a version string moved into a sink body row is still visible after masking, so a mask that hid body bytes would fail. The shipped docker/replay_off_golden.json is checked for structure only (three scenarios, a 64-hex sha256 and a positive byte count each); its digests belong to real runner output and are not compared with synthetic bytes. Reverting the numeric engine masking, or reverting the sink trailer recomputation, turns these rows red. Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
![graphify-labs[bot]](https://avatars.githubusercontent.com/in/4356270?s=60&v=4){kind=small}
There was a problem hiding this comment.
Graphify reviewed this change.
No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.
Not checked: tests were not run; no formal proof of the changed code.
Graphify review — findings
Adds regression tests that pin down how the replay-off golden check treats repository release bumps. A patch, minor, or major bump derived from the real VERSION file must leave the masked default report bytes, the masked sink bytes, and their SHA-256 digests unchanged for the fifo, any, and ordinary shapes. The sink tests also require that the trailer digest is recomputed after masking, and that version changes in the sink body stay visible rather than being hidden by the mask. A further test checks that the recorded docker/replay_off_golden.json has the expected structure without pinning its digest values.
No blocking issues surfaced.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 20 functions depend on the 20 functions this change touches.
Health — grade A; no new coupling hotspots.
Verification — 20 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 20 function(s) in the blast radius were not formally verified this run
|
Superseded by #647 (same change on current main, folded with its sibling). Branch kept. |
Adds rows to the replay-golden check's own tests so that a release VERSION bump can never move the sink goldens again.
The rows read the real
VERSION, bump the patch, minor and major parts, and assert that the masked bytes and digests are equal to the unbumped ones, with non-vacuity and leak controls. This is the class that turned main red after the last two release commits; with these rows it fails before a tag instead.Test script only (
scripts/test_check_replay_off_golden.py).🤖 Generated with Claude Code