Skip to content

Read the newest sweep manifest in the new-host grand list - #1000

Merged
seathatflowsinourveins merged 3 commits into
mainfrom
cc/w4-currency
Oct 11, 2026
Merged

seathatflowsinourveins merged 3 commits into
mainfrom
cc/w4-currency

Conversation

@seathatflowsinourveins

@seathatflowsinourveins seathatflowsinourveins commented Oct 11, 2026 •

Copy link
Copy Markdown
Collaborator

Scope

The new-host grand list reads the newest committed sweep through the existing component-matrix selector, replacing its September 23 constant. Its currency flag now compares the displayed winner pin with that sweep's upstream release using the producer's own comparison API, correcting 14 falsely current rows and the distinct-component summary from 20 to 31. The preceding guard repair validates the selected document at the report's new join boundary: incomplete sections, missing report layers, malformed consumed component fields, and a future UTC checked_at stop the build with the offending path before output or registration. It never falls back to an older sweep.

  • Base: 65cf2730068429a1c48c8c18fb24aadef8120817.
  • Head: c7ab11920d5bff32f50ce43a8ef3f763db0b1e04, ONE CC-adjudicated correction of 74721739be522a1bcfdd972c6544c411c813a5b5, after the earlier guard repair of adca78fb4e6f14408de815a71c6435c932882abd.
  • Lane: lane:foundation.
  • Original owned paths: the grand-list script, tests, generated JSON/Markdown and their existing evidence bindings. The pin correction changes the script, its fixture test, both generated reports and their three existing evidence bindings. The shared selector, native producer and bootstrap pin files remain byte-identical.

The maintained selector still chooses the greatest parsed checked_at and uses the later filename for a tie. The report checks the shape of fields it consumes, including the manifest's coverage of report layers. Null pin comparisons remain unknown; candidate records retain their native repository/disposition shape, and the existing five unjoined winners remain explicit. This does not claim whole-document sweep-schema validation.

SOTA sources

  • [mechanism] 65cf273:scripts/component_matrix.py:L659-L699 provides the existing JSON/date selection and duplicate-layer refusal. Its Git blob is ecc083fff39c87853fc675b46729cff766e2d273, identical at reviewed adca78f. No second selector is introduced.
  • [mechanism] 65cf273:tools/sota-convergence/build_manifest.py:L1818-L1862 assembles the foundation components and trading entries with id, repository, pin and upstream/comparison fields; L1971-L1978 defines complete layer coverage; L2021-L2024 preserves null for uncomputed comparisons. Git blob e3afbcb445d2719a44b6601c1993cd7a6b0649ef is identical at reviewed adca78f. The consumer guard follows this maintained producer contract.
  • [mechanism] python/cpython v3.13.16, locked commit cbc944f4bc59639a444dd971c737788ba2283a91, Doc/library/datetime.rst:L925-L939 and L951-L958 documents timezone-aware datetime.now(timezone.utc). The installed mirror was read at that pin; the consumer compares the sweep's parsed date with the current UTC date, using a fixed clock in regressions.
  • [reference] 65cf273:catalogs/saturation/ledger.json:L10918-L10922 records the completed October 9 sweep; docs/lanes.md:L94-L116 specifies report regeneration and registration of files already listed on main; .github/workflows/validate.yml:L151 checks the grand list.

Evidence-class table

The table and original command block below record the preceding 7472173 guard repair. The adjudicated displayed-pin correction has separate current validation below.

Claim Evidence class Result
Both GPT findings are in scope at the newly changed join synthetic The same newer partial/future inputs are ignored by base 65cf273's fixed older path but consumed by reviewed adca78f's added selector call. Base returns 1.0.0/joined; head returns null/unjoined for partial and 99.0.0/behind for future.
Repair regressions fail before the fix synthetic Four permanent methods at exact adca78f: seven intended assertion failures, zero errors, no skips. They cover partial input, three malformed component records, complete future input, and both --write refusal cases.
Refusal precedes publication synthetic Corrected --write regressions preserve both pre-existing output sentinels and make zero register_file calls.
Current report behavior and selection rules are preserved local_integration / synthetic Generated reports remain byte-identical: 32 layers, 66 winners, 20 behind and five unjoined. Independent inverse-name/date and filename-tie controls pass; fixed UTC today passes and tomorrow refuses.
Touched and adjacent module gates pass local_integration 112 tests, zero skips, exit 0: grand-list 43, component-matrix 64, evidence-manifest five.
FULL validation and PR-range secret scan pass local_integration 70 components / 11,472 hashed files / four profiles / 239 receipts; two PR commits scanned with 100% redaction and no leaks.

Local commands run at preceding 7472173 guard repair

python3 -E -s -B -m unittest tests.test_new_host_grand_list tests.test_component_matrix tests.test_evidence_manifest
112 tests, no skips; exit 0
python3 -E -s -B scripts/new_host_grand_list.py --write
32 layers / 66 winners; both reports byte-identical to reviewed head
python3 -E -s -B scripts/new_host_grand_list.py --check
exit 0
python3 -E -s -B scripts/component_matrix.py --check
32 rows; exit 0
python3 -E -s -B scripts/evidence_manifest.py --check
11,472 files; exit 0
nice -n 10 ionice -c 2 -n 7 python3 -E -s -B scripts/validate.py
70 components / 11,472 hashed files / four profiles / 239 receipts; exit 0
git diff --check
exit 0
gitleaks git --redact=100 --log-opts='65cf2730068429a1c48c8c18fb24aadef8120817..74721739be522a1bcfdd972c6544c411c813a5b5'
two PR-range commits, no leaks; exit 0

Decision record

Both reproduced GPT P2s are blockers on the new caller; their underlying shared-selector omissions are not repaired across unrelated consumers. The adopted selector, date ordering and filename tie behavior remain the recorded mechanism. No new library, candidate selection, verdict or catalog pin is introduced.

The earlier 12-to-20 sweep-pin count did not describe the displayed winners. The CC's newly reproduced real-data failure supersedes its prior P3 classification: 14 joined rows for 11 components were falsely marked current. The correction reuses the producer's own classify_pin and pin_comparison_fields APIs for each displayed winner pin against the joined upstream release. The true distinct-component count is 31, and the generated page now discloses this exact comparison basis beside its separate bootstrap-pin note.

Adjudicated displayed-pin correction

Primary mechanism: 65cf273:tools/sota-convergence/build_manifest.py:L39-L48 and L147-L176 owns version, commit-tracking and OS-package exclusion behavior; L1366-L1372 owns null/not-compared output. The file's Git blob e3afbcb445d2719a44b6601c1993cd7a6b0649ef is unchanged here. The report loads that existing module by exact code path using the producer's own supported importlib pattern at L113-L123; fixture input ROOT does not select executable code. No second comparison rule is written.

Fail-first at exact 7472173: the new fixture deliberately sets the sweep's pin to 2.0.0/current while the displayed winner varies. One method gives two intended assertion failures and zero errors: displayed 1.0.0 should be behind 2.0.0; a displayed commit-tracking pin should remain null/not compared. The equal-version control already passes. After repair all cases pass, including the rendered behind note and summary.

On committed data the correction changes exactly 14 row flags from false to true and the summary from 20 to 31 distinct components; no other JSON property changes. There are still 66 winners, 61 joined and five unjoined. Restored components are agent-browser, ccusage, claude-code, headroom, markitdown, mcporter, opentelemetry-collector-contrib, prometheus, rtk, sandbox-runtime and worktrunk. Both JSON and Markdown are regenerated; the page's behind notes now describe the displayed pins.

Current gates: 270 tests without skips (grand-list 44, component-matrix 64, evidence five and the maintained producer's SOTA-convergence module 157), FULL 70 components / 11,472 hashed files / four profiles / 239 receipts, report/matrix/evidence checks and diff check all pass. Native registration updates only the test and two report bindings. The actual three-commit PR range scans with 100% redaction and no leaks.

Follow-ups

  • P3: add direct shared-selector tests whose filename order disagrees with checked_at and which exercise the filename tie rule; these are currently independently probed.

  • Wider scope: bring the component matrix and other shared-selector consumers under an agreed sweep-admission/whole-document schema contract; the report guard covers its own consumed fields only.

  • CC-adjudicated follow-up 4242517442: the producer selects a prerelease stream for the sweep pin's major, which can differ from the displayed pin's major. Zero committed rows are affected; carrying release-list evidence or selecting the displayed pin's stream remains a producer/consumer follow-up.

  • CC-adjudicated follow-up 4242517447: equivalent non-canonical GitHub URLs can join but fail the producer classifier's URL recognition. Zero committed rows are affected; align repository normalization with the native producer.

  • CC-adjudicated follow-up 4242517453: custom --os-package-ids cannot be recovered from a sweep manifest that omits generation options. Zero committed rows are affected; the producer must record its options before consumers can preserve a custom exclusion policy.

  • CC-adjudicated producer follow-up 4242517457: Promptfoo is compared with code-scan-action-0.2.2 from the producer's upstream stream selection. These values and the flag are unchanged since 74721739; correct the producer's stream selection separately.

  • Cosmetic P3: align the indentation of the two added page-note strings at scripts/new_host_grand_list.py:572-573 with their neighbours.

  • Astra P3: improve the shared selector's malformed-layer diagnostic so it includes the offending manifest path.

  • Astra P3: refresh docs/decisions/2026-10-05-dagu-mise-pin-move.md:128, whose live-row citation still targets the old scripts/new_host_grand_list.py:44 constant line.

Host evidence

No host receipt, installation, service, broker, model or provider operation. Synthetic fixtures and local repository gates do not establish hosted acceptance. The CC owns review and landing.

Checklist

  • Both in-scope blockers reproduced on the exact reviewed head and repaired in one forward commit.
  • Permanent fail-first tests protect refusal, outputs and registration.
  • Both JSON and Markdown reports were regenerated; three existing evidence bindings changed (the two reports and the test). The shared selector and native producer retain their reviewed bytes.
  • Touched modules, FULL validation, evidence and diff checks pass.
  • PR-range-only gitleaks uses 100% redaction; no credential or settings read.
  • Confirmation reviews, actual-head hosted CI and CC landing.

scripts/new_host_grand_list.py hard-coded
MANIFEST = "catalogs/sota-convergence/manifest-20260923.json" (:46 at
65cf273), so every winner's pin was compared with the 2026-09-23 sweep
although the newest committed manifest by checked_at is
manifest-20261009.json (2026-10-09, sweep landscape-sweep-20261009-pine,
catalogs/saturation/ledger.json:10918-10922, landed in #956). The constant
fed the pin data (:218), the JSON sources block (:305) and the rendered
text (:400, :487); the module docstring named it (:15).

build() now takes the manifest from
component_matrix.newest_sweep_manifest(ROOT) and render_md() names the one
the data came from. Source of the mechanism, first party at 65cf273:
scripts/component_matrix.py:659-683 newest_sweep_manifest picks the
manifest with the latest ISO checked_at (ties: the later file name), never
by file name or ledger order, and stops on an unreadable or undated one;
SWEEP_MANIFEST_DIR and SWEEP_MANIFEST_NAME are :88 and :91, parse_iso_date
:629-636, load_convergence_input :639-656. The component matrix already
selects its sweep manifest the same way (:856). A tree with no sweep
manifest now stops the build with a message instead of a
FileNotFoundError.

Regression: tests/test_new_host_grand_list.py NewestSweepManifestTests
builds over a temp root with two manifests, the later checked_at having
the opposite pin_behind_upstream. Against 65cf273's script it fails (the
build reads manifest-20260923.json: ('1.0.0', False) != ('2.0.0', True));
with this change it passes.

Regenerated with python3 scripts/new_host_grand_list.py --write: 66
winners, sources.manifest 20260923 -> 20261009, 42 upstream_latest and 20
pin_behind_upstream values changed, pins_behind_upstream 12 -> 20
components, join coverage unchanged (the same 5 winners without a manifest
row). manifests/evidence.json re-registers the two outputs and the test
file (docs/lanes.md:116).

Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
@seathatflowsinourveins seathatflowsinourveins added the lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers label Oct 11, 2026
@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-11T17:31:41.157454Z c7ab119 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: adca78fb4e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/new_host_grand_list.py Resolved
@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: 74721739be

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@seathatflowsinourveins
seathatflowsinourveins marked this pull request as ready for review October 11, 2026 17:25

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c7ab11920d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/new_host_grand_list.py Resolved
Comment thread scripts/new_host_grand_list.py Resolved
Comment thread scripts/new_host_grand_list.py Resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant