Integrate PRs #55–#58 into one grand-catalog base (2026-09-22) - #59

…evidence

…paper practice

…ntegration

…ration

…ollector

Protocol is committed before any real-bar evaluation. Collector retains every
response page with hash and ledger for pagination evidence.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…any evaluation

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…n tests

Session state derived from the provider clock+calendar (never the local
clock alone); universe = active/tradable us_equity assets excluding OTC and
placeholder symbols (reuses collect_daily.DATA_SYMBOL/read_credentials via
importlib); SIP snapshots batched under a 4KB URL budget; observation_class,
prev_bar_mismatch, stale/invalid quote flags; protocol eligibility and S1-S5
signal flags computed from daily.parquet prior sessions (duckdb window
functions) plus today's snapshot bar when --history is given, else eligibility
is reported unknown; bounded own news fetch (not market_research.collect,
whose 3-GET/30-symbol shadow-watchlist contract cannot cover a universe scan)
with sha256'd headlines and no bodies; private full-row output vs a
public output redacted to symbol/list-membership/rank/rounded-pct/dv-bucket/
flags only. 38 synthetic-fixture local integration checks (no network, no
credentials, no real dataset) all pass. Live read-only run executed once
against the paper-api/data hosts (no --history: daily.parquet was not yet
materialized by the coverage builder at run time).

Co-Authored-By: Claude Fable 5.1 <[email protected]>

… for broad-universe

coverage.py materializes daily.parquet from the raw/all bar CSVs (refusing on an
incomplete ledger or duplicate rows) and builds an offline coverage-manifest.json
covering collection integrity (pagination proof, full sha256 re-verification,
row-total reconciliation), universe/calendar coverage, quality flags, identity
collisions, protocol eligibility counts and a bounded live entitlement probe.
corporate_actions.py collects the market-wide /v1/corporate-actions feed in
calendar-year windows with the same page/ledger evidence pattern as
collect_daily.py and summarizes counts per type/year.

Live runs against the completed broad-market-20260921 dataset: 21,332,097
materialized rows, 0 pagination violations across 310 batches, 4,424/4,424
pages sha256-verified with 0 mismatches, row totals matched exactly, and
independently reproduced asset-master symbol selection matched plan.json
byte-for-byte (symbols_sha256_match=true). Corporate-actions collection
retrieved 17,217 items across 11 years with 0 failures.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…ation profile into the portable practice

The twin moves from three fixed concurrent agents to the per-host rule
(start at 8 under the client's default ceiling of 16; the min(16, CPUs - 2)
formula is a local convention) and sets CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH=1.
The workflows README adds the Adopt step for a permission source covering the
Workflow tool and the contract bullets for opt-in and limits, failure and
replay, agent allowlists and skill preloads, profile invariants and version
gates, with the official-doc excerpts retained as an artifact. source-scout
and review-changes run acceptance commands raw through `rtk proxy` where
installed (byte-identical to agent-lab). The Codex example registers each role
under [agents."<name>"] with config_file and description, labelled unverified
on an installed CLI. The routing guide's cache key name is corrected.

New dated records: docs/harness-rules-convergence-20260922.md (layer
manifest, 49 + 67 verified rules with statuses, placements and overturns) and
docs/new-workstation-runtime-profile-20260922.md (Threadripper PRO 5975WX /
RTX 4090 / 128 GB projection: sizing, services, verifier-corrected
clean-install sequence, Codex configuration rules, sim-paper-live gates),
with evidence/artifacts/harness-rules-convergence-20260922/ (run usage rows,
the blocked Codex Lane C attempt, official-doc excerpts).

Checks: validate.py (68 components, 1,475 files, 4 profiles, 130 receipts),
validate_catalogs, validate_foundation ok, validate_convergence, build --check,
unittest OK (41 skipped), token-report suite, zizmor, git diff --check, the
portable suites (169/23/16/18/PASS); an independent Opus read-only review
returned 14 findings, all resolved. No private content; manifest rows match.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…comer dispositions, engine reconciliation

16 foundation and 12 consolidated trading layers confirmed against GitHub metadata for
396 repositories; 34 candidate records (33 repositories) adversarially verified, one
targeted candidate (snyk/agent-scan); engine card reconciled with the NautilusTrader
2.0.0rc5 destination; Codex cross-family review blocked by the exhausted usage limit
and recorded as not established. Rebased onto the concurrent sync commits.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…source

codex-rs/agent-roles/src/{agent_role_config,loader,discovery}.rs at tag
rust-v0.155.1, read through the GitHub contents API and retained in
evidence/artifacts/harness-rules-convergence-20260922/codex-agent-roles-source.json:
[agents.<name>] entries carry description, config_file and nickname_candidates;
role files hold name, description, nickname_candidates and config keys such as
developer_instructions with unknown keys rejected; every .toml under the agents
directory is discovered without a table entry; a declared config_file is not
loaded twice; the file's description wins over the table's; duplicate names in
one layer warn; a description is required. The Codex example, its README and
the workstation profile now say so instead of "unverified"; the registration is
still not exercised in a session. The second Lane C attempt failed on the same
Codex usage limit and is recorded in runs.json.

Checks: validate.py, validate_catalogs, validate_foundation ok, build --check,
unittest OK, zizmor, git diff --check.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…I check)

Co-Authored-By: Claude Fable 5.1 <[email protected]>

Replace the two ad hoc host-path prototype scripts with tools/sota-convergence/
{extract_layers,github_freshness,build_manifest}.py: deterministic no-network
extraction of the working files from the two maintained catalogs, resumable
authenticated GitHub freshness, and a manifest merge with a tested pin-vs-
upstream rule, a never-promote disposition rule, and a sanitize-then-refuse
writer. Add synthetic-fixture tests, the checked-in default reconciliations
file, the tool README, and the operating recipe with its recipes/README.md
index row.

Regression run against the real 2026-09-22 working files reproduces the
published manifest's (layer, repository, review_status) identity set,
candidate dispositions and components_confirmed exactly; the small remaining
pins_behind_upstream difference and its cause are documented in both the
tool README and the recipe.

Co-Authored-By: Claude Sonnet 5 <[email protected]>

Fourth tracked job on codex-cli 0.155.1 (365 s, four findings, none high; the
first two attempts failed on a Codex usage limit and the third on a stale
companion-broker token after re-login): config_file in the Codex example is
relative to the config folder (agents/<name>.toml, not .codex/agents/...);
min(16, CPUs - 2) is the client's default cap per the bundled
/workflow-authoring reference and the setting accepts 1-256, so the twin
CLAUDE.md and recipe say so; .wslconfig processors counts logical processors
(5975WX: 32 cores / 64 threads, plan moved to 60 with a topology check); the
Codex thread cap is a separate client limit, not a mirror of the Claude
setting. runs.json records all four attempts; the cooperation recipe records
the broker-restart boundary after codex login.

Checks: validate.py, validate_catalogs, validate_foundation ok, build --check,
unittest OK, zizmor, git diff --check, portable suites.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

- classify_pin() now excludes OS-distribution package pins (systemd's
  real row: a GitHub repository with an Ubuntu-package pin) via an
  -Nubuntu/-Ndeb/+debN pin-suffix regex and a --os-package-ids
  allow-list (default: systemd), instead of only excluding non-GitHub
  repositories.
- github_freshness.py: gh_api() now catches subprocess failures
  (timeouts, missing binary) per call instead of letting one abort the
  whole run; results are checkpointed to --out every 25 fetched
  repositories and again in a finally block; a resumed run now retries
  records that previously carried "error" instead of treating them as
  covered.
- disposition() validates proposed_label against the fixed proposable
  set ({not_adopted, keep_but_compare, targeted_candidate}) before
  applying the survives logic, so an unrecognised label can no longer
  pass through unchanged as a promotable-looking value.
- build_manifest.py now sorts per-layer components/entries by
  (decision-rank, id) and candidates by (disposition-rank, repository)
  before writing, so reruns against reordered lane/catalog input
  produce byte-identical row order.
- recipes/sota-convergence-practice.md step 3 replaced the
  nonexistent "run-saved-workflow" command with the actual contract:
  the Claude Code saved workflow "sota-convergence" (args: work_dir,
  repo, lanes?, refuters?, budgets?, max_proposals_per_lane?) returns
  {lanes, critic, lost} and writes nothing; the coordinator persists
  that object to $WORK_DIR/lanes.json.

Each finding is reproduced by a test that fails against the pre-fix
source and passes after the fix (verified by stashing the two source
files and rerunning tests.test_sota_convergence).

Co-Authored-By: Claude Fable 5.1 <[email protected]>

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…rer rebuild, no host path in the tooling README

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…validator and progress service pass

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…e tooling

- URL normalization: compute_upstream/repository_known fall back to a
  normalized GitHub slug lookup, and github_freshness.py records every alias
  URL seen for a slug, so a release/tag/tree catalog URL resolves to the same
  freshness record as the canonical URL (finding 1).
- Sanitize decoded manifest values (walking dict/list/str) before
  json.dumps, then json.loads the result to prove it is valid JSON before
  the leak check and write; broadened host-path coverage to macOS and
  Windows forms, reusing scripts/validate.py's pattern intent (findings 2, 6).
- github_freshness.py keeps releases/tags/commit sub-request failures as
  per-record partial_errors, counts them, treats them as retryable on
  resume, and distinguishes an expected 404 ("no releases") from a
  timeout/429/5xx (finding 3).
- github_freshness.py now reports fetched_this_run, retained_from_prior_runs
  and an observation_window (min/max observed_at); build_manifest.py's
  method.freshness text cites that window instead of generated_at, with a
  fallback for older freshness files that predate the top-level fields
  (finding 4).
- merge_lanes handles a selected verdict's survives=False and survives=None
  explicitly, so an unknown outcome stays "<status>_unverified" instead of
  being promoted to confirmed_default with a false refutation note
  (finding 5).
- Carry evidence_level from the catalog card into manifest trading entries,
  and rewrite the recipe/tools README/catalog "Method and limits" text to
  separate review_status (selection/pin confirmation) from evidence_level
  (execution classification) -- review_status never establishes native
  execution (finding 7).

tests/test_sota_convergence.py: 25 new tests covering all seven findings
(52->53... actually 41->53 tests). Verified: unittest suite passes (53/53);
py_compile clean; git diff --check clean; scripts/validate.py fails only on
expected SHA-256/byte-count hash mismatches for the six files this commit
edits (manifests/evidence.json is out of scope and was not touched); rerun
of build_manifest.py against the private 2026-09-22 working files under
/home/seath/codex-ecosystem/state/sota-convergence-20260922 (read-only,
output under /tmp) confirms the slug-normalization fix newly resolves 8
previously-null freshness lookups (repomix, markitdown, qmd, rtk,
codebase-memory-mcp, toon x3 layers), flipping pins_behind_upstream_unique_components
from 15 to 17 (repomix and markitdown newly behind); components_confirmed
and candidates_by_disposition are unchanged (106 / 34) since the real
2026-09-22 lanes.json has zero survives=null selected verdicts, so finding
5's fix is defensive here rather than count-changing on this dataset.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

Co-Authored-By: Claude Fable 5.1 <[email protected]>

… in the convergence record

Co-Authored-By: Claude Fable 5.1 <[email protected]>

… statuses excluded from confirmed totals

Co-Authored-By: Claude Fable 5.1 <[email protected]>

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…nce and practice

Co-Authored-By: Claude Fable 5.1 <[email protected]>

… catalog integration branch

Shared generated files (explorer HTML, evidence hashes, build_ecosystem inputs) were unioned and rebuilt; validators pass (68 components, 1,529 hashed files, 133 receipts).

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…ing #56) into the grand catalog integration branch

Unioned the shared generated indexes (explorer HTML, evidence hashes and receipts, stack evidence ids, saturation audit receipts, build_ecosystem inputs) and kept the newer PR #57 wording for the worker-cap prose and examples. Validators pass (68 components, 1,688 hashed files, 137 receipts; landscape 843 repositories); unittest 1,153 tests OK.

Co-Authored-By: Claude Fable 5.1 <[email protected]>

…e corrected default/override explanation

PR #55's paragraph called min(16, CPUs − 2) the runtime ceiling; PR #57 had corrected it to the client default that the setting overrides (1–256). Found by the bounded Codex cross-family review of the integration.

Co-Authored-By: Claude Fable 5.1 <[email protected]>