Skip to content

test: run-failure diff tests follow the diff's own start (hold across release resets) - #509

Merged
luisleo526 merged 1 commit into
mainfrom
cx/relreset-test
Oct 25, 2026
Merged

luisleo526 merged 1 commit into
mainfrom
cx/relreset-test

Conversation

@luisleo526

Copy link
Copy Markdown
Collaborator

The v1.7.0 tag's own preflight is red at run-failure-diff-tests: scripts/test_gen_run_failure_catalog_diff.py (test_report_sink_failed_is_engine_fault) still expected report_sink_failed as an addition, but the release reset docker/run_failure_codes_diff.json to "from 1.7.0 to unreleased" with equal catalog shas. Every post-tag engine PR inherits that red.

The run-failure diff tests now follow the diff's own start (its from/to versions), so they hold before and after every release reset; no release pair is hard-coded. One commit on the v1.7.0 tag ed9e08ec05; test files only (scripts/test_gen_run_failure_catalog_diff.py, tests/test_s4e_misc_contracts.py).

Evidence: the plain-tag preflight control fails at that stage only (P2-CX rj-20261025t040223-4edbd9; WIN-CX's CLOUD preflight 57/57 as a second witness); this commit is green on the post-reset tree and on the pre-reset state. REV-CX Opus read: GO, no P0/P1 (three small follow-ups in a later commit).

🤖 Generated with Claude Code

… hold before and after a release reset

A release resets docker/run_failure_codes_diff.json to "from that release, to
unreleased" with equal catalog shas, so its `added` list is empty.
test_report_sink_failed_is_engine_fault still demanded that the diff add
report_sink_failed (true only while the diff starts at 1.6.3): ci_preflight's
run-failure-diff-tests stage was red on the v1.7.0 tag tree and on every branch
cut from it.

- scripts/test_gen_run_failure_catalog_diff.py: assert_diff_states_code compares
  the diff's from.version with the code's `since` as numeric release tuples
  (the generator's own version_key; a candidate sorts below its release). A code
  whose release is after the start must be one whole addition and no change; a
  code already in the start's catalog must be no addition, never removed, and
  listed as changed only for real changes that match this tree's catalog. No
  release pair is written into the logic. A new test drives a real git cycle
  across a reset with 12 negative controls.
- tests/test_s4e_misc_contracts.py: test_upcoming_failure_codes_are_announced
  asserted a non-empty `added`, red at every reset; right after a release no
  release is upcoming, so only the announcement check remains.

Proof: preflight on the plain tag is red at run-failure-diff-tests; on this
tree preflight passes (57 stages) and the s4e misc ctest rows pass; the same
change on the pre-release tree (diff from 1.6.3) is run as the second state.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
@luisleo526
luisleo526 merged commit 7a7d099 into main Oct 25, 2026
13 checks passed
luisleo526 added a commit that referenced this pull request Oct 25, 2026
… the six ownership manifest tests

Brings in the release-reset test fix (#509) so this branch's preflight is no longer red on
the tag's catalog-diff test. Merge, no rebase, no force. The diff against main stays the
one-line tests/CMakeLists.txt change.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
luisleo526 added a commit that referenced this pull request Oct 25, 2026
…it and #509

Brings in the release commit ed9e08e (VERSION and the run-failure catalog diff stamp) and #509
(the run-failure diff tests follow the diff's own start), which cures the red preflight test
test_gen_run_failure_catalog_diff on the tag. No change to this PR's own files.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
luisleo526 added a commit that referenced this pull request Oct 25, 2026
…it and #509

Brings in the release commit ed9e08e (VERSION and the run-failure catalog diff stamp) and #509
(the run-failure diff tests follow the diff's own start), which cures the red preflight test
test_gen_run_failure_catalog_diff on the tag. No change to this PR's own files.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
luisleo526 added a commit that referenced this pull request Oct 25, 2026
…it and #509

Brings in the release commit ed9e08e (VERSION and the run-failure catalog diff stamp) and #509
(the run-failure diff tests follow the diff's own start), which cures the red preflight test
test_gen_run_failure_catalog_diff on the tag. No change to this PR's own files.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
luisleo526 added a commit that referenced this pull request Oct 25, 2026
…it and #509

Brings in the release commit ed9e08e (VERSION and the run-failure catalog diff stamp) and #509
(the run-failure diff tests follow the diff's own start), which cures the red preflight test
test_gen_run_failure_catalog_diff on the tag. No change to this PR's own files.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
luisleo526 added a commit that referenced this pull request Oct 25, 2026
Two textual conflicts, one semantic follow-up:

- .github/workflows/native-live.yml: main added strace to the native build
  dependencies, this PR added git-lfs; the install line keeps both.
- scripts/ci_verify.py EXCLUDED_REGISTERED_MIN: main's native floor is 963,
  this PR adds one native runner row, so the floor stays 964; main's reworded
  D1 comment is kept.
- scripts/test_ci_verify.py: main's test_the_native_profile_is_not_under_this_rule
  (not in the PR's base) ran the native profile at 963 registered and 963 rows;
  with this PR's row the tree registers 964, so the test runs at 964/964.

Doc deletions that arrive from main with this merge, named for
check_doc_reverts (they are main's own edits to docs/ci.md, not changes made
here): six sentences deleted or reworded by 717c75d (promote-baseline refuses
untrusted heads and actors, #482) and by 3a0d150 (the CR-233 split of the PR
wall from full-event verification, #477): "Only after those checks can an
unmerged close pass without a merger lookup and skip `promote`", "The trigger
has no `branches:` filter and the guard does not require a `main` base ref",
"Release and promotion are unchanged:", "The same stage pins the complete shared
`promote-guard` block byte for byte", "Decision tests execute the guard script
extracted from the real workflow with bash and an executable `gh` stub first on
PATH", "Mutation tests alter the real promotion workflow to replace the script
with `exit 0`".

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
luisleo526 added a commit that referenced this pull request Oct 25, 2026
The tree is identical to the previous head of the PR branch (46fe81f, which
already contained main). This merge only reverses the parent order so that the
first-parent chain of the head reaches main's last pull request commit
(7a7d099, #509) at once. check_doc_reverts finds its base on a checkout
with no branch refs (the lab hosts) from that chain; with the PR branch as the
first parent it fell back to c32c732 (#478) and judged main's own commits
(717c75d, 3a0d150, ...) as if this PR had made their doc deletions.

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
luisleo526 added a commit that referenced this pull request Oct 25, 2026
…ab doc-reverts walk starts at main's last PR commit

Brings in #505 (the release ruleset probe listing: docs/ci.md, scripts/ci_release_gate.py,
scripts/test_ci_release_gate.py) on top of the v1.7.0 release commit and #509. No change to this PR's
own files; the first parent is main on purpose (CR150's merge shape).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
luisleo526 added a commit that referenced this pull request Oct 25, 2026
…ab doc-reverts walk starts at main's last PR commit

Brings in #505 (the release ruleset probe listing: docs/ci.md, scripts/ci_release_gate.py,
scripts/test_ci_release_gate.py) on top of the v1.7.0 release commit and #509. No change to this PR's
own files; the first parent is main on purpose (CR150's merge shape).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant