Skip to content

Add the shared evidence-bound gap closeout loader - #982

Merged
seathatflowsinourveins merged 7 commits into
mainfrom
foundation/ns-movers-gap-closeouts-loader-20261010
Oct 11, 2026
Merged

seathatflowsinourveins merged 7 commits into
mainfrom
foundation/ns-movers-gap-closeouts-loader-20261010

Conversation

@seathatflowsinourveins

@seathatflowsinourveins seathatflowsinourveins commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator

Scope

Historical gap counts can remain open after later receipts supply evidence. This PR adds one shared Draft 2020-12 schema and one component-matrix loader for dated layer closeouts, using registered file hashes and two-family review credit. It adds no layer closeout records: the frozen crosswalk, matrix, host exports and main-base counts remain unchanged.

  • Base: main eb31d233234cdc552fc81672087ac8769e3696e0; seven linear commits. The five reviewed commits were replayed, correction IBKR paper orders through NautilusTrader 1.231.0's IB engine, with an after-hours plan #147 remains its own sixth commit, and the FIRST GPT correction is one forward seventh commit.
  • Published head: 416e5da30ceb63f4ae983faaab2f6843d5c4ecf5; the final commit refuses conflicting optional receipt hash/byte mirrors and unsupported installed jsonschema versions. Its direct parent is ad2e1b1ac4fc9c71a9a8a18a052813fa5f72b14c; no further rebase.
  • Lane: lane:shared; other-lane acknowledgment and designated reads remain required. Runtime-workers' prior baseline ACK carries over under the CC's byte-preservation ruling.
  • Scope: the shared loader/schema and their fixtures/docs, hash-pinned CI dependencies, the freshness-suite install, regenerated worker advisory snapshot and its direct comparison test, retained native evidence and explicit manifest registrations.

Each record names the original gap index/text and crosswalk hash, disposition/reason, retained evidence, supported vendor paths/pins and registered review references. The hash registry reads files[] only: real receipts[] rows contain descriptive metadata without hashes or byte counts.

A settled entry requires passing GPT and Claude JSON verdicts at one reviewed head, with reference metadata matching their registered bytes. A residual or blocked entry without that same complete credit records as proposed and preserves category/counts, including prior reviewed credit. Reviewed residuals remain open in their explicit category.

The conditional schema dependency uses the existing module-level SkipTest convention when jsonschema is missing. The freshness job keeps its calendar install step byte-identical to the existing pinned contract and adds a separate hash-pinned jsonschema step, using the existing CI requirements set, before Python 3.13 full discovery. Jsonschema 4.26.0 requires Python >=3.10; the no-record path retains the broader recording-script compatibility floor.

The worker advisory snapshot is regenerated on main c4f49907 through its unchanged native GitTree/derive_ci_protected path, following #849, and its existing test compares directly with the snapshot. The new snapshot has 330 locations and 98 shape entries, SHA-256 8752740817bb41c1a582e33fa475e4ce4064f2ccfecadb706bce40ecb4512157. The prior candidate also had 330/98, with SHA-256 04a245baf20ec34bfb33e76f0692bab6047331723a5c4f8da38f2da5fcf9a64c; those exact bytes are retained. Thirteen blind-checkout locations shift by two lines, with per-script counts and shape multiplicities preserved. The original pre-fix snapshot is retained too. Worker gate implementation, coverage bounds and enforcement assertions are unchanged. Layer PRs conform after this shared loader lands; #976 remains held for that sequence.

The architecture inventory test now uses :(glob)catalogs/landscape/*.json, matching both readers' top-level Path.glob('*.json'). The default Git pathspec had included the nested schema and failed CI on the real merge. The exact existing native method fails one test before and passes after changing only that literal. The complete requested modules pass on the integrated implementation. The test and its registered hash are the only authored changes beyond Git's clean integration of landed #984.

SOTA sources

Evidence-class table

Claim Evidence class Retained evidence
Original refusal/public count controls synthetic Four observed failures before initial implementation
Actual receipt shape and review-count correction synthetic cc-fix/before.log: real receipt positive case errors; isolated residual/blocked assertions fail actual counts
Five existing guards are pinned synthetic Five named failing-case fixtures and five caught isolated in-memory guard-removal controls
Installed schema API and missing-package discovery local_integration Jsonschema 4.26.0, matching source bytes; discovery exits 0 with one declared module skip when absent
Unchanged numeric data and generated bytes local_integration Native matrix --check and five byte comparisons against the main base
Worker advisory snapshot local_integration New-base native regeneration: 330 locations, 98 shape entries; prior snapshots archived; exact-head assertion passes and the original snapshot is caught
Final source/evidence bindings local_integration Registered cc-fix/receipt.json and final FULL validation

Local commands run

Latest correction at 416e5da3:

<CI_ENV>/bin/python -m unittest tests.test_gap_closeouts tests.test_component_matrix tests.test_validate
exit 0; 174 tests, zero skips (26 closeout + 64 matrix + 84 validator).
<CI_ENV>/bin/python scripts/validate.py
FULL exit 0; 70 components, 11,466 hashed files, 4 profiles, 239 receipts.
python3 scripts/component_matrix.py --check
exit 0; 32 rows, with the no-record path using an interpreter without jsonschema.
gitleaks git --no-banner --redact --log-opts='eb31d233234cdc552fc81672087ac8769e3696e0..HEAD'
Gitleaks 8.30.1 exit 0; seven linear PR commits, zero findings.

The two P2 regressions are retained in evidence/artifacts/shared-gap-closeouts-loader-20261010/gpt-fix/receipt.json: seven focused controls give six assertion failures and zero errors on the unchanged reviewed loader, then seven pass after the fix. Matching optional mirrors, metadata-only receipts and no-record behavior remain valid. An actual installed jsonschema 4.19.2 accepted the fixture before the correction and now refuses before returning a document; the actual missing-package refusal and no-record compatibility controls also pass. The version check uses the standard distribution metadata API before loading schema or closeout documents. The native worker assertion caught four extra advisory locations on an unpublished preparation candidate; naming the receipt metadata key receipt_path keeps it distinct from the filesystem iterator. The final committed-head worker assertion passes one test at 330 locations/98 shapes and zero unresolved; unchanged snapshot bytes retain the prior baseline ACK. These are fixture and integrity measurements. The earlier commands below retain their historical scope.

env PATH=<ci-env>/bin:<existing-PATH> nice -n 10 ionice -c 3 <ci-env>/bin/python -m unittest tests.test_gap_closeouts tests.test_component_matrix tests.test_runtime_worker_openhands_push_gate tests.test_start_closure_sampler tests.test_landscape_sweep_skills tests.test_saturation_ledger tests.test_lane_packets
Before the final freshness correction: exit 0; 552 tests, two existing conditional skips.

env PATH=<ci-env>/bin:<existing-PATH> nice -n 10 ionice -c 3 <ci-env>/bin/python -m unittest tests.test_workflow_hardening
At reviewed parent 8f933e57: exit 1; 142 tests, one StopIteration error, one existing conditional skip.

env PATH=<ci-env>/bin:<existing-PATH> nice -n 10 ionice -c 3 <ci-env>/bin/python -m unittest tests.test_workflow_hardening tests.test_gap_closeouts
At the preceding freshness correction: exit 0; 163 tests (142 workflow + 21 closeout), one existing conditional skip.
The workflow regression file remains unchanged.

env PATH=<ci-env>/bin:<existing-PATH> nice -n 10 ionice -c 3 <ci-env>/bin/python -m unittest tests.test_local_pages_architecture_landscape_exclusions tests.test_gap_closeouts tests.test_workflow_hardening
Before the landing rebase: exit 0; 170 tests (7 architecture + 21 closeout + 142 workflow), one existing conditional skip.

<ci-env>/bin/python <isolated guard-removal controls>
exit 0; five of five controls caught. These edit isolated module objects only.

python3 -m unittest discover -s tests -p test_gap_closeouts.py
exit 0; one declared module skip when jsonschema is absent.

<ci-env>/bin/python -m unittest <freshness/workflow modules>
Before the final landing rebase: 261 tests; one then-stale-registration publication assertion failed, the other 260 passed.
After supported registration refresh, the same publication assertion passed: one test, exit 0.

<ci-env>/bin/python scripts/component_matrix.py --check
exit 0; 32 rows, base counts 238 executable / 355 open unchanged.

env PATH=<ci-env>/bin:<existing-PATH> nice -n 10 ionice -c 3 <ci-env>/bin/python scripts/validate.py
Before the landing rebase: FULL exit 0; 70 components, 11,451 hashed files, 4 profiles, 239 receipts.

git diff --cached --check
exit 0.

gitleaks git --no-banner --redact --log-opts='eb31d233234cdc552fc81672087ac8769e3696e0..HEAD'
At ad2e1b1a: Gitleaks 8.30.1 exit 0; six linear PR commits, zero findings.

Fail-before interpretation is explicit: the original registry bug preempts the five existing guards' expected refusal messages in the real-receipt fixtures. Separate guard-removal controls prove that each named fixture detects its missing guard. The isolated residual/blocked cases measure the actual unauthorized count changes. No guard-removal control edits repository source.

The final freshness receipt is freshness-fix/receipt.json, with the actual fail-before and passing logs. The snapshot bytes remain SHA-256 8752740817bb41c1a582e33fa475e4ce4064f2ccfecadb706bce40ecb4512157, and the native derivation on the corrected workflow yields the same advisory locations and shapes. The new-base receipt is cc-fix/landing-rebase.json; its required-module log, original-baseline control and prior-candidate snapshot are registered too. The final committed-head worker assertion passes one test, and substituting only the original pre-fix snapshot makes that assertion fail as expected. The required logs, original snapshot and correction receipt are registered under evidence/artifacts/shared-gap-closeouts-loader-20261010/cc-fix/. The earlier synthetic equal-hash receipt fixture did not model the real metadata-only rows; the correction states that limit. Personal trace prefixes and final blank lines are normalized with actual assertions/results retained. The historical public-count trace keeps its execution-time line 200 and now cites current source line 298. Its linked receipt hash is refreshed.

The current branch contains seven linear commits on main eb31d233: 2c10ba79, c1d4439d, 7e768d69, bd42faf7, e1c6df76, ad2e1b1a, and 416e5da3. The first five replay the reviewed commits; ad2e1b1a preserves correction #147 as its own commit, containing only the one architecture test line and its native registry entry. The final forward commit addresses the two P2s in the FIRST GPT read. The only replay conflict was in manifests/evidence.json: the exact main copy was retained and the affected own rows were regenerated through the supported registration commands. The remaining registry deltas match the original commits row-for-row. All five replayed commits have identical range-diff output outside manifests/evidence.json; the fold carries the original regression evidence and records seven architecture tests with one failure before the correction and seven passing afterward. Main registrations and non-file registry arrays are preserved. No hand-merged manifest was used. Local testing follows touched modules plus FULL validate; full discovery remains hosted. The upstream full nox suite was not run, and these fixtures establish no vendor runtime, production-data or host acceptance.

At ad2e1b1a, the landing-rebase validation recorded FULL exit 0 with 70 components, 11,456 hashed files, 4 profiles and 239 receipts. The CI-parity venv ran all four required modules: 296 tests, OK, three skips (the existing conditional skip and two PyYAML-gated tests). Default Python, with PyYAML available, ran the other three modules: 275 tests, OK, one existing conditional skip; both PyYAML-gated tests passed. No single interpreter ran all 296 tests with one skip. The exact-head worker assertion passed separately, and snapshot SHA-256 8752740817bb41c1a582e33fa475e4ce4064f2ccfecadb706bce40ecb4512157 remains unchanged at 330 locations and 98 shapes. The range-diff and fold proofs have SHA-256 ab96d7593db0df206af4d678e897b595c999eea5fdf3853ca98e231e961a79ce and 9b5bfa3bf97674a86d77488cd94481f6cf3e1be500bcae67ec1b382bac527de9, respectively.

Decision record

docs/decisions/2026-10-10-shared-gap-closeouts.md and catalogs/landscape/gap-closeouts/README.md describe the authoritative registry, dependency floor, review-credit rules and supported update commands.

Host evidence

No host receipt or platform-status change. Per-layer host, production-data and login boundaries retain their own requirements. Runtime-workers independently regenerated and acknowledged the snapshot at 8f933e57 at 20:25:27Z. The CC's carryover ruling preserves that ACK: snapshot bytes remain unchanged, and the final committed-head native assertion passes. The CC acknowledged the peer inventory edit as correction #147, as #984's owner. The CC's 2026-10-11 00:23Z ruling carried its prior Claude PASS to ad2e1b1a using change-id 66088a3f1e92. The FIRST GPT read then requested these two P2 corrections. This forward commit changes the loader, so the CC determines whether that earlier Claude PASS carries to 416e5da3; carryover is not assumed. A GPT micro at the new head, current-head hosted CI and remaining landing gates are due.

Checklist

  • Workflow action pins and permissions remain unchanged.
  • Dependencies use existing exact versions and SHA-256 hashes.
  • No credential values, expanded personal home paths or new required secrets.
  • Peer-owned/protected paths and frozen historical crosswalk preserved.
  • Required fixtures, source citations, native snapshot and final evidence bindings are retained.
  • Worker baseline acknowledgment carried for the byte-identical snapshot.
  • New-head hosted CI and designated reads.
  • Other-lane acknowledgment and CC landing.

@seathatflowsinourveins seathatflowsinourveins added the lane:shared Touches files owned by both lanes; needs both lanes' acknowledgement label Oct 10, 2026
@socket-security

socket-security Bot commented Oct 10, 2026 •

Copy link
Copy Markdown

Dependency limit exceeded — report not shown.

This pull request scan exceeded the 10,000-dependency limit applied to this scan, so the results are incomplete and may be inaccurate. To avoid reporting false positives, Socket has not posted a report.

Upgrade your plan to raise the dependency limit and get complete reports, or view the partial scan in the dashboard.

Socket is always free for open source. If this is a non-commercial open source project, contact us to request a free Team account.

@seathatflowsinourveins
seathatflowsinourveins force-pushed the foundation/ns-movers-gap-closeouts-loader-20261010 branch 2 times, most recently from a08b616 to ad2e1b1 Compare October 11, 2026 00:18
Validate dated layer closeouts through the pinned jsonschema 4.26.0
Draft 2020-12 API. Bind original crosswalk identity, registered evidence,
supported vendor sources and two passing GPT/Claude reports at the same head.
Preserve proposed/residual/blocked gaps and all current-main counts.

Retain the required failing-before controls, receipt/files compatibility
correction and final native fixture evidence. Make the existing validator
pin available through hash-pinned CI requirements.

Sources: python-jsonschema/jsonschema v4.26.0 at
a7277432b0f7bcd0551f6e589d30457017125df4, validators.py:307,348,812;
native-agent-stack main 9d4c00c,
catalog_decisions.py:57,85 and host_receipts.py:710.

Validation: 84 touched-module/inventory tests, zero skips; unchanged native
matrix check and final registered-tree FULL validation passed.
Account for the shared loader's eight additional input-read locations and
one measured dynamic shape in the existing repository workflow test.
Retain the historical inventory and every other script, shape, coverage
and enforcement expectation; no loader, schema or worker-policy change.

Source: native-agent-stack main 9d4c00c,
blueprints/runtime-workers/openhands/resolver/gate_reads.py:913-930.

Validation: the original assertion fails before correction; the touched
module passes 125 tests with one existing optional PyYAML skip using the
hash-pinned CI environment. Final registered-tree FULL validation passes.
Remove the final empty line in each native advisory test log to satisfy
the repository whitespace contract. Refresh the explicit file bindings
and retain the prior hashes. Test assertions and results are preserved.

Validation: final registered-tree FULL validate passed after normalization;
the cached whitespace gate must pass before this forward commit.
Use files[] as the hash authority and model real metadata-only receipt
rows in the fixtures. Keep unreviewed residual/blocked records proposed
without changing counts or prior credit. Pin each missing closeout guard.

Use the existing missing-library SkipTest convention and install the
hash-locked schema validator before freshness discovery. Document the
Python 3.10 dependency floor and correct the upstream/native citations.

Regenerate the worker advisory snapshot through the native derivation and
remove hard-coded increments. Retain the prior snapshot and fail-before
measurements. Source: main 03a986e,
validate.py:665-666 and the native worker GateReads derivation;
jsonschema 4.26.0/a7277432b0f7bcd0551f6e589d30457017125df4.

Validation: required/schema-enabled modules 552 tests, 2 existing skips;
freshness-related modules 261 tests with the stale-registration case
rechecked after supported refresh; final FULL validation passed.
Peer-path correction acknowledged by the CC as #984's owner in
correction #147 (2026-10-10T21:40Z). This commit carries, as its own
linear commit on main eb31d23, the fold that a08b616 authored inside
its forward merge of d965908 and a7c15bd: the single inventory line
and its files[] registration, re-registered through
host_receipts.register_file and evidence_manifest.py --write.

Git 2.53.0's default pathspec '*' can match directory separators;
':(glob)' uses FNM_PATHNAME, so its wildcard cannot cross '/'.
Primary source: git/git v2.53.0, Documentation/glossary-content.adoc,
the pathspec and glob sections, re-read from the installed Git 2.53.0
gitglossary(7).

Reader references, unchanged from a7c15bd to main eb31d23:
tools/local-pages/architecture_sources.py:375 and
tools/local-pages/architecture_builder.py:241 both use top-level
Path.glob('*.json'). The test now uses the equivalent supported
Git pathspec, ':(glob)catalogs/landscape/*.json'.

Regression evidence carried from a08b616, measured at main a7c15bd:
the exact native inventory method from merged main, compiled in memory
with its policy and committed catalog inputs, fails one test before on
the nested gap-closeouts/schema.json and passes one test after
changing only the pathspec literal; zero errors in either probe.

Re-measured on this linear branch at parent e1c6df7
(2026-10-11T00:08Z): git ls-files lists 22 paths for
'catalogs/landscape/*.json', including
catalogs/landscape/gap-closeouts/schema.json, and 21 top-level paths
for ':(glob)catalogs/landscape/*.json'.
tests.test_local_pages_architecture_landscape_exclusions ran 7 tests
with 1 failure on that nested schema.json before this change, and
7 tests OK after it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Fix the two P2s from the FIRST GPT read of ad2e1b1. files[] alone
supplies evidence bindings; optional receipt sha256/bytes mirrors must
agree with those bindings. Metadata-only rows remain valid. Nonempty
closeouts require installed jsonschema exactly 4.26.0 before schema or
document loading, with an actionable hash-pinned installation command.
The no-record path does not inspect package metadata.

Use CPython v3.13.16 Lib/importlib/metadata/__init__.py:980 (version)
and :49 (PackageNotFoundError), and python-jsonschema/jsonschema
 a7277432b0f7bcd0551f6e589d30457017125df4 jsonschema/__init__.py:27.
Both fetched primary source files match the installed bytes; retain
source hashes and the supported metadata API citation in the correction
receipt and decision record. No replacement validator is introduced.

The seven focused controls fail against the unchanged reviewed loader:
six assertion failures, zero errors, while equal mirrors, descriptive
receipts and the no-record behavior pass. All seven pass after the fix.
Actual Python 3.14.4/jsonschema 4.19.2 accepts the fixture before the fix
and refuses it afterward without returning a document. Actual missing-
package refusal and no-record compatibility controls also pass.

Keep the receipt key named receipt_path, distinct from the existing
filesystem iterator named path. The unchanged native worker assertion
caught four advisory locations on the unpublished preparation candidate;
the native alias control restores 330/98 with identical per-script counts
and shape multiplicities. Preserve that failure and the candidate-only
control; the final committed head is checked before the first push.

Required modules: 174 tests OK, zero skips (26 closeouts, 64 matrix,
84 validator). FULL validate: 70 components, 11466 hashed files,
4 profiles, 239 receipts, exit 0. Matrix --check: 32 rows, exit 0.
The worker snapshot remains SHA-256 8752740817bb41c1a582e33fa475e4ce4064f2ccfecadb706bce40ecb4512157,
330 locations and 98 shapes; frozen crosswalk and generated reports
remain unchanged. Preserve historical traces and refresh their source
locator and file registrations through the native registry function.

One forward commit on ad2e1b1. The GPT micro and the CC's disposition
of its prior Claude PASS remain required at this new loader head.
@seathatflowsinourveins
seathatflowsinourveins force-pushed the foundation/ns-movers-gap-closeouts-loader-20261010 branch from 416e5da to c49f08a Compare October 11, 2026 02:38
@seathatflowsinourveins
seathatflowsinourveins marked this pull request as ready for review October 11, 2026 03:01
@seathatflowsinourveins
seathatflowsinourveins merged commit 878f289 into main Oct 11, 2026
33 checks passed
@seathatflowsinourveins
seathatflowsinourveins deleted the foundation/ns-movers-gap-closeouts-loader-20261010 branch October 11, 2026 03:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:shared Touches files owned by both lanes; needs both lanes' acknowledgement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant