Repository navigation
Laptop evidence: ultracode-subagent E2E of 13 token/memory/RAG tools; vLLM 0.30.0 embedder switch - #316
Merged
Conversation
… vLLM 0.30.0 embedder switch Mirrors the workstation's #296 method on wsl-authoring-20260923 (13 of #296's 16 tasks; 29 children, 15 Sonnet 5 + 14 Opus 5.5 at effort max). Publishes each tool's check result and independent verifier verdict, including the refuted rtk claim (branch -a '+' marker; the bare git show exclusion missing the -C form), Serena's missed cross-file references and repomix --compress dropping a def. Adds a baseline_kind field so only counterfactual_raw pairs read as reductions. Adds the laptop's vLLM 0.25.0 -> 0.30.0 qualification and switch receipt, reporting set identity, strict-order failures and the production-vs-production control totals. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
enabled auto-merge (squash)
September 26, 2026 02:45
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
seathatflowsinourveins
deleted the
claude/laptop-e2e-evidence-20260926
branch
September 26, 2026 02:58
seathatflowsinourveins
added a commit
that referenced
this pull request
Sep 26, 2026
…ion form and git branch The laptop's ultracode E2E verifier (#316) found two gaps on the pinned rtk 0.50.0: "^git show [^ ]*:" misses `git -C DIR show REV:path` (the blob is windowed), and filter_branch_output (src/cmds/git/git_cmd.rs 3185-3244) keeps git's '+ ' worktree marker, so `git branch -a` misreports local branches as remote-only when a same-name remote exists. The recipe now adds two subcommand-anchored patterns whose global-option group mirrors rtk's GIT_GLOBAL_OPT (-C, -c, --git-dir, --work-tree, --flags), tells hosts to replace the existing exclude_commands line, and bootstrap-linux.sh reminds unless the key appears exactly once with all four entries. Retained raw hook-check on the pinned binary; dated receipt addendum. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
added a commit
that referenced
this pull request
Sep 26, 2026
…ion form and git branch (#318) The laptop's ultracode E2E verifier (#316) found two gaps on the pinned rtk 0.50.0: "^git show [^ ]*:" misses `git -C DIR show REV:path` (the blob is windowed), and filter_branch_output (src/cmds/git/git_cmd.rs 3185-3244) keeps git's '+ ' worktree marker, so `git branch -a` misreports local branches as remote-only when a same-name remote exists. The recipe now adds two subcommand-anchored patterns whose global-option group mirrors rtk's GIT_GLOBAL_OPT (-C, -c, --git-dir, --work-tree, --flags), tells hosts to replace the existing exclude_commands line, and bootstrap-linux.sh reminds unless the key appears exactly once with all four entries. Retained raw hook-check on the pinned binary; dated receipt addendum. Co-authored-by: seathatflowsinourveins <[email protected]> Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
seathatflowsinourveins
added a commit
that referenced
this pull request
Sep 27, 2026
…ta from the 2026-09-27 verdict wave (#401) * E1/E2 token E2E receipts: dated adjudications, errata and reproduced answer oracles Applies the 2026-09-27 cross-family GPT-6 review of the E1 (#296) and E2 (#316) receipts, append-only: every recorded field keeps its value, and each receipt keeps its own serialization. - Every tools[] row gains adjudication {as_of, task_acceptance, bad_input_control_recorded, basis, evidence}, using the review's status table and vocabulary (partial, retracted_later, untested, unsupported). - E1: QMD retracted_later (wrong_document, corrections_to_296); Repomix retracted_later (count=47 PASS on a 48-function file; repomix v1.18.1 PythonParseStrategy.ts L81-86 tests only a def's first line); ai-memory untested (base_empty=True is a vacuous pass); the other thirteen partial with no recorded failing control. A dated errata block names the pre-errata sha256 the preregistration records as a historical identity. - E2: context-mode, jcodemunch, qmd and ast-grep stay partial with no recorded failing control; SocratiCode's comparison counts an abridged transcription; RTK retracted by its verifier; Repomix's FAIL accurate. - READMEs carry dated corrections; the review is retained, sanitized, beside E1. - tests/test_token_e2e_receipt_checks.py: receipt contracts (red before the errata) and answer oracles rebuilt from pinned Git blobs, hash-matched to the recorded baselines, each with failing controls (red under a weaker historical-style check). Sources: docs/acceptance-evidence-policy.md (Discriminating controls); https://github.com/yamadashy/repomix/blob/v1.18.1/src/core/treeSitter/parseStrategies/PythonParseStrategy.ts#L81 evidence/artifacts/token-e2e-codex-20260926/receipt.json#/corrections_to_296. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Token layer: routing rules and recipe limits for known upstream defects Closes the documentation, recipe and routing gaps from the 2026-09-27 per-tool verdict wave for the token-efficiency layer. Every rule cites the pinned upstream source or a committed receipt; no pin changes. recipes/README.md (token-tool rows and sections): - agentsview: v0.44.0 unqualified; worker populations need --include-children, --include-automated and --include-one-shot, and --fts reads message bodies only (v0.43.0 docs/session-api.md, docs/commands.md); archive answers are observation. - ast-grep: every 0.45.3 command registers customLanguages from any sgconfig.yml in the working directory or a parent (lib.rs L107-139, config.rs L98-131/L280-299); PR #2960's opt-in is unreleased; outline (#2957) and YAML rule (#2963) limits. - ccusage: token-only while any model is unpriced (v20.0.24 json-output.md Unpriced Models; config-files.md pricingOverrides). - codebase-memory: trace_path excludes tests and evidence by default, search_code hits can be mentions (v0.11.0 src/mcp/mcp.c schemas); caller-set procedure. - context-hub: check metadata.versions (FastAPI guide 0.136.3 vs 0.141.1). - context-mode: npm tarball digest and plugin commits are separate identities. - headroom: v0.39.1 changes only the proxy limiter, so the omission-count defect stands. - markitdown: HTML as .txt passes through; -x html / -m text/html; selective extras. - mcporter: --name does not select the server; a positional token becomes the selector (v0.14.1 call-arguments.ts L164-192, ephemeral-flags.ts L97-105); the jCodeMunch example gains --server. - qmd: MCP query expands and reranks by default (v2.8.3 README, server.ts); typed lex + rerank:false + bounded get; catalog scope; refresh guard for #989/#991 (src/store.ts reindexCollection). - repomix: multi-line Python signatures are dropped (v1.18.1 PythonParseStrategy.ts L81-86); exact-definition tasks go to an uncompressed pack or the original. - toon: BOM root-string round trip (#339); no record-count threshold upstream; nested-uniform columns are tabular (tabular.ts L61-78). docs/token-session-handbook.md: a "Known upstream limits behind the lanes" section outside the carrier text (every carrier line unchanged), and row pointers. docs/token-practice.md: counts, comparisons and acceptance rules (invocation counts are not success rates; comparisons need task acceptance; count recovery reads; gpt-tokenizer's special-token contract; receipts carry adjudications). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Apply the GPT-6 review of this branch: exact inventory oracle, TOON eligibility wording One cross-family review round (GPT-6 through codex-omniroute, read-only) returned DEFECTS FOUND with two findings; both are fixed: - P2 tests/test_token_e2e_receipt_checks.py: inventory_oracle checked only the claimed count plus subset membership, so a correct count with missing, duplicated or no names passed. It now requires the exact name set with one entry per function. New controls keep the correct count while dropping, duplicating or inventing a name, or listing none; they failed against the old oracle first (red) and pass now. - P3 docs/token-session-handbook.md (and the matching recipes/README.md toon row): "accepts any non-empty uniform array" overstated TOON 4.1.1's tabular eligibility. The encoder needs non-empty objects sharing one key set, with columns of primitives or, recursively, of non-empty objects sharing one key set; an empty object or an array-valued column falls back to list form (packages/toon/src/encode/tabular.ts L6-78 at v4.1.1; confirmed with the installed 4.1.1 CLI on [{}], [{"a":[]}], [{"a":{}}] and [{"a":{"b":1}}]). Also: the ast-grep oracle keys counts by path under scripts/ rather than file stem, and E1's Context Mode adjudication states that attempt 1's marker-grep FAIL on a refused call is not shown to be a run of attempt 2's answer check (the receipt does not record that command). Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> * Apply the independent verifier's findings: TOON source line, citation anchors, E2 Serena erratum The independent verifier of this branch reported five defects; all are fixed: - docs/token-session-handbook.md, the TOON Sources line under the carrier (on main before this branch) still said "The encoder accepts a non-empty uniform array", the wording GPT-6 finding P3 refuted. It now states the encoder's rule from packages/toon/src/encode/tabular.ts L6-78 at v4.1.1 and links the handbook's "Known upstream limits behind the lanes" section. It is not a "- " carrier line, so tests/test_token_lanes_subagent_start.py's verbatim-line check is unaffected. - Citation anchors: SocratiCode's INCLUDE_DOT_FILES default is documented under README.md "Indexing Behaviour" (v1.14.0 L1574-1579), not "Ignore Rules", so the handbook links #indexing-behaviour. codebase-memory-mcp v0.11.0 search_code gets its own link to src/mcp/mcp.c#L632-L659 in the handbook, and the recipe gives index_repository (L466-481), trace_path (L532-560) and search_code (L632-659) one link each. Both upstream files matched the retained copies by sha256 on 2026-09-27. - E2 receipt errata.items[4] and the E2 README erratum now name Serena's accurate reference FAIL (find_referencing_symbols missed 6 of 8 call sites; the definition check passed) and give each group of partial rows its own reason. Only that one finding string changed in receipt.json (same serialization, 1 line); written_at is unchanged because this refines the unmerged 2026-09-27 erratum on the same day. - tests/test_token_e2e_receipt_checks.py: the require_commit docstring now says the guard is modelled on RetainedEvidenceTests' Git check and adds the pinned-commit probe (tests/test_adoption_status.py L1721-1724 checks only git and the checkout). Sources: toon-format/toon v4.1.1 packages/toon/src/encode/tabular.ts; giancarloerra/SocratiCode v1.14.0 README.md; DeusData/codebase-memory-mcp v0.11.0 src/mcp/mcp.c; the E2 receipt's own serena row (tools[5]) and README. Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]> --------- Co-authored-by: Scout <scout@local> Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
5 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
wsl-authoring-20260923) ultracode-subagent E2E of 13 token, memory and RAG tools, reusing the method and tasks of Token stack inside Ultracode subagents: 16/16 tools E2E with lifetime counters, exact comparisons and five new host receipts #296.2ed0b8cc(main after ai-memory consolidation: dated note on GPT-6 models and upstream #852 #313).lane:foundation.evidence/artifacts/token-e2e-ultracode-laptop-20260926/{receipt.json,README.md}evidence/receipts/vllm-030-switch-laptop-20260926.jsonmanifests/evidence.json: 3 files and 1 receipt registered.SOTA sources
evidence/artifacts/token-e2e-ultracode-20260925/{receipt.json,README.md}(Token stack inside Ultracode subagents: 16/16 tools E2E with lifetime counters, exact comparisons and five new host receipts #296, workstation) at this repository's main.exact_comparison: null).evidence/receipts/vllm-030-switch-20260925.json, and the upstream is vllm-project/vllm v0.30.0 (source pinced6857a, as reused from the reference and labelled not re-verified on this host).adoption/pins-linux-x86_64.json. rtk is v0.50.0, binary sha25623433a2a….Evidence-class table
native_provenon this hostgit branch -a'+' worktree marker; the baregit showexclusion misses-C). Serena: check failed (missed cross-file references in.mjs). Repomix: check failed (--compressdropped a def). The other 10: confirmednative_provenbaseline_kind: counterfactual_rawpairs are read as reductions;check_ground_truthpairs are not savingslocal_integrationupstream_estimate: host-wide over the window, not isolatednative_provenReview. An independent evidence-reviewer ran three rounds:
Checks at the pushed head:
validate.py: passed (5997 files, 154 receipts).evidence_manifest --check: passed.tests.test_host_receipts,test_evidence_manifestandtest_validate: 232 OK.validate.py --scan-fileon the new files: passed.origin/main..HEAD: no leaks.The rtk recipe fix these findings led to is in a separate PR.
🤖 Generated with Claude Code