Skip to content

Register 10 P2/P3 SOTA candidates from the 2026-09-26 gap audit (metadata only) - #315

Merged
seathatflowsinourveins merged 1 commit into
mainfrom
claude/p2-p3-candidates-20260926
Sep 26, 2026
Merged

seathatflowsinourveins merged 1 commit into
mainfrom
claude/p2-p3-candidates-20260926

Conversation

@seathatflowsinourveins

@seathatflowsinourveins seathatflowsinourveins commented Sep 26, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Registers the 10 SOTA candidates that the 2026-09-26 gap audit confirmed are missing from every catalog. They are recorded in manifests/candidates.json, so the next landscape sweep's discovery and refuter steps pick them up. The 3 P1 repos (MIRIX, ByteRover CLI, ColPali) are not here; they are being trialled and get their own PR.

This is metadata-level registration only. No install was attempted, no adoption status is given, and every entry says so. The upstream facts were fetched live with gh api, and each entry records the time of that check.

Repository Priority Status Layer(s) License Reviewed commit Latest release
langchain-ai/langmem P2 registered for next sweep foundation/durable-memory MIT 9d033b47d9ce none
OSU-NLP-Group/HippoRAG P2 registered for next sweep foundation/semantic-rag MIT 1438aba3fc44 v1.0.0
lightonai/pylate P2 registered for next sweep foundation/semantic-rag MIT 3389837ce465 v1.6.0
FlagOpen/FlagEmbedding P2 registered for next sweep foundation/semantic-rag MIT fd1a2bdf6948 v1.4.2
EleutherAI/lm-evaluation-harness P2 registered for next sweep us-equities/evaluation-experiments, foundation/quality-evaluation MIT d6de81643928 v0.4.13
traceloop/openllmetry P2 registered for next sweep foundation/observation-inference Apache-2.0 f7082c8a99ef 0.62.3
NVIDIA/TensorRT-LLM P3 watch foundation/semantic-rag NOASSERTION 923c24fdd0be v1.2.1
Cinnamon/kotaemon P3 watch foundation/document-retrieval Apache-2.0 9ad3e4e49aa3 v0.12.0
Helicone/helicone P3 watch foundation/observation-inference Apache-2.0 067d9290acb4 v2025.08.21-1
open-telemetry/semantic-conventions-genai P3 watch foundation/observation-inference Apache-2.0 e57c543b4889 none

catalogs/us-equities/decision-index.json and its manifest counts were regenerated with scripts/catalog_decisions.py --write: legacy_candidate goes from 20 to 30 and repositories from 861 to 871. blueprints/token-native-focus/saturation-audit.json gets the matching identity-partition counts: catalog_identities 861 → 871, nonselected 792 → 802 and legacy_candidate 2 → 12. tests.test_stack_lifecycle enforces them. catalogs/saturation/ledger.json is untouched. It is an append-only hash chain written only by a completed sweep.

Review

A GPT-6 (Codex, gpt-6-astra, effort max) cross-family review found one defect:

  • Finding: open-telemetry/semantic-conventions was registered as the GenAI schema, but its pinned docs/gen-ai/README.md says the conventions moved to open-telemetry/semantic-conventions-genai.
  • Fix: the entry now registers the maintained repository, with its own live facts, and cites the pinned "Moved" notice.
  • Re-check: all 10 reviewed_commit values resolve and still equal their default-branch HEAD, and all 21 source URLs return HTTP 200.

Judgment calls, recorded in the entries' decision text:

  • TensorRT-LLM is filed under foundation/semantic-rag as the closest existing layer, because there is no local-inference layer.
  • TensorRT-LLM's license was read from its LICENSE file (Apache-2.0), because GitHub's API reports NOASSERTION.

Verification

  • scripts/validate.py, validate_catalogs.py, evidence_manifest.py --check, catalog_decisions.py --check, saturation_ledger.py --check, component_matrix.py --check and new_host_grand_list.py --check: all pass.
  • Full python -m unittest: 5784 tests OK (skipped=632) before the rebase onto Install receipts for promptfoo and inspect-ai on the workstation (GPT-6 reviewed and repaired) #317. After the rebase, the related modules (tests.test_stack_lifecycle and others) pass: 128 OK.
  • python -m unittest tests.test_catalog_decisions tests.test_saturation_ledger tests.test_catalogs tests.test_validate tests.test_evidence_manifest: 181 tests OK.

SOTA sources

🤖 Generated with Claude Code

@seathatflowsinourveins seathatflowsinourveins added the lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers label Sep 26, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

…missing

Metadata-level registration in manifests/candidates.json with live upstream facts and
pinned sources, for the next landscape sweep. The OpenTelemetry GenAI entry points at the
maintained semantic-conventions-genai repository (GPT-6 cross-family review finding).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@seathatflowsinourveins
seathatflowsinourveins force-pushed the claude/p2-p3-candidates-20260926 branch from afca389 to fef53bf Compare September 26, 2026 03:15
@seathatflowsinourveins
seathatflowsinourveins merged commit 9c1669c into main Sep 26, 2026
24 of 28 checks passed
@seathatflowsinourveins
seathatflowsinourveins deleted the claude/p2-p3-candidates-20260926 branch September 26, 2026 03:32
seathatflowsinourveins pushed a commit that referenced this pull request Sep 26, 2026
Hot-file commit (docs/lanes.md): manifests/evidence.json starts from
main's copy (with #318's and #315's entries) and re-registers every
changed or new file of this branch (indent 2, ensure_ascii); receipts[]
mirrors the rtk and markitdown receipts' new limitations and keeps every
other entry. component_matrix --write (no change) and
new_host_grand_list --write regenerate the reports.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins added a commit that referenced this pull request Sep 26, 2026
…bootstrap reminder, blind-packet leak, retained markitdown evidence (#314)

* Retain markitdown's installed-byte verifier, run directories and raw checker output

Codex post-merge review of #291, finding 3.

- verify_installed.py is retained byte-identical. Its 2026-09-25 stdout
  was never written to a file, so results.json says so (not_retained)
  and adds a 2026-09-26 read-only re-run on the production prefix
  (45 of 45 identical) plus a per-file listing (installed_files.py) of
  the wheel RECORD, wheel member and installed digests in RECORD form.
- run-candidate/, run-baseline/ and run-postswitch2/ (converted
  documents, PDF stderr with the home directory replaced by ~, package
  listings) are retained; their files hash to the recorded values.
  run_arm.py kept only checker counts, so check_outputs.py is re-run on
  them (out/checker-rerun-20260926.out), and the independent verifier's
  own checker.out, legs logs and integrity check are retained.
- results.json gains corrections, not_retained, steps and files; the
  receipt gains a dated limitation and regenerated provenance.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
(cherry picked from commit 9ec4c33)

* Keep the durable-memory incumbent's build and releases out of blind packets

Codex post-merge review of #269, finding 1 (P2).

The dated durable-memory limitations (the 19b6429/Mac/2.3.2/VelaNext
entry, the open_gaps[0] entry and the 2.4.0/2.4.1 cutover entry), the
older #234 "Production now runs ai-memory 19b6429" entry and the #234
"keeps ai-memory as the production control" sentence named the
incumbent's build, releases and role without a selection word next to a
candidate name, so build_all_packets(..., withhold=True) kept them.
Each sentence carrying those identifiers now calls the component the
incumbent, a choice phrase the prose filter (lane_packets.py
CHOICE_PHRASE, reduce_prose) drops; the ledger keeps every fact.
Verdict fields and checked_at are unchanged. A regression test builds
the real withheld durable-memory packet (default and blind-export
parameters) and asserts that 19b6429, 2.3.2, 2.4.0, 2.4.1 and
"production control" are absent while the ledger still records them.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
(cherry picked from commit 418c0f0)

* Grand list legend: the JSON carries winners' basis and only open-gap counts

Codex post-merge review of #269, finding 2 (P3). The "What a winner
means" legend said the JSON carries the layer's limitations and open
gaps; the generator emits each winner's why_selected and evidence_refs
and only gap counts. The legend now says so and points to the layer
ledgers for the limitations and the open-gap text (without paths: the
page is a new-host document).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
(cherry picked from commit caeaa4d)

* rtk hook exclusions: adopt #318's four-entry recipe and add rtk's own check to its reminder

Reconciles this branch's Codex post-merge fixes for #291 (findings 1-2)
and the #314 P2 with #318, which merged first.

- Recipe: #318's four-entry exclude_commands block stays the one set of
  guidance in recipes/README.md, bootstrap.md step 4a, token-practice.md,
  the rtk pin note and the SOTA refresh record. The single regex this
  branch had tested ('^git(\s+\S+)*\s+show(\s+\S+)*\s+(:\S|[^\s-]\S*:)'
  with "diff") is kept only as dated evidence, labelled as an alternative
  that was tested (mitigation-20260926, readback-20260926). The recipe
  also explains rtk's anchoring (registry.rs:1540-1569, 1599-1604) and
  that explicit `rtk` commands bypass the exclusions (registry.rs:1690).
- Reminder: rtk_config_reminder keeps #318's text check (the exact
  four-entry key, exactly once) and, once it passes, runs the installed
  "$bin_dir/rtk" hook check (timeout 30, stdin closed, the caller's
  environment) on `git show HEAD:x | tail -n 5`, `git -C . show
  --no-color HEAD:x | tail -n 5`, `diff a missing`, `git branch -a` and
  `git -C . branch`. Either check failing prints the reminder; a text
  pass with an rtk failure adds a hint naming the probe, because rtk
  ignores a TOML-valid file that does not deserialize (TrackingConfig,
  src/core/config.rs:152-158 at 1d87b8e7). Print-only.
- Tests: all of #318's reminder tests are kept (the silent ones now run
  against a stub rtk that honours the file); new stub tests cover an
  exact text that rtk rewrites, the [tracking] counterexample, an
  unsupported subcommand, a rewritten branch probe, extra output, the
  #291 comments counterexample, the five probes and XDG_CONFIG_HOME
  reaching rtk. An optional real-binary test (skipped without an
  installed rtk 0.50.0) covers the four-entry block, the [tracking]
  rejection with and without history_days, the old two-entry line, the
  single-regex alternative and a missing file.
- Evidence: combined-reminder-20260926.{sh,out} (hook check and the
  combined reminder on scratch configs; silent on this host's #318
  block). The rtk receipt keeps #318's addenda untouched; its added
  limitation now references #318's recipe and the combined reminder;
  results.json gains a step, files and relabelled corrections; the
  receipt's provenance is regenerated.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

* Register the post-merge fixes' files and regenerate the grand list

Hot-file commit (docs/lanes.md): manifests/evidence.json starts from
main's copy (with #318's and #315's entries) and re-registers every
changed or new file of this branch (indent 2, ensure_ascii); receipts[]
mirrors the rtk and markitdown receipts' new limitations and keeps every
other entry. component_matrix --write (no change) and
new_host_grand_list --write regenerate the reports.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

---------

Co-authored-by: Scout <scout@local>
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant