Skip to content

Blind-lane audit and registry lows left after #145 rounds 9–12 #189

Open

Description

@seathatflowsinourveins

These are low findings from the independent review rounds of #145 (rounds 9 to 12, 2026-09-24). The reviewer rated them below the merge bar, so #145 merges without them. Each still needs a fix or a dated disposition.

Transcript audit (tools/sota-convergence/transcript_audit.py)

  • R12-1: the NORM11-1 rule is under-tested. The tests don't pin a padded base component followed by two or more components. A value.rsplit('/', 1) rewrite of the rule, or a first-plus-last-two rewrite, passes every targeted module and reopens round 11's recursive forms. Fix: add a case such as <export>/.. /a/b/*.
  • NORM11-3: three path-modelling gaps in code the delta didn't change.
    • Lexical vs physical ..: _within realpaths the raw path, while the runtime normalises lexically. This is latent: there are 0 tracked symlinks in the export source.
    • The dirname of a wildcard-free Glob pattern is not modelled.
    • A NUL byte crashes the audit at unguarded call sites (claude_lane.py, adjudicate.py). It should flag instead.

Registry (tools/sota-convergence/lane-provenance.json)

  • R12-2 / RR11-2: registry checks cover too few keys.
    • The Claude uniqueness key omits prompt_sha256, so the README's append step for a lane-prompt.md change would fail CI.
    • The per-path guard doesn't require the current agent or prompt hash.
    • The registry-scope text in the README and tests is stale.
  • Enforce lane-provenance.json append-only against the base branch in CI #175 tracks CI enforcement of append-only against the base branch.

Disclosed, not fixed

Related: #168.

🤖 Generated with Claude Code

Activity

  1. seathatflowsinourveins commented on Sep 24, 2026

    @seathatflowsinourveins
    CollaboratorAuthor

    These lows come from independent round 13, which reviewed the JSON-prose P1 fix at 4ab4dfe2. #145 is now merged as ff2ac50a.

    • R13-1: the three newly counted layers are false positives. The layers the fix adds to cited exposure (git-github-automation, portfolio-risk, research-factors-ml) match only on technical uses of selection words, such as "Retained artifact hash: worktrunk-create.json" or "retained candidate records". The disclosure therefore over-reports, which is the safe direction. Still, the README's "Remaining limits" sentence should say these are technical-sense matches, not stated selections.
    • R13-2: two mutants survive. Each of them changes the measure's output, yet no test fails.
    • Hardening option: role-keyed scalars. A key-named selection such as {"selected_tool": "<winner>"} is invisible to both the prose measure and the label-container checks. The real export has none today.

    🤖 Generated with Claude Code

  2. seathatflowsinourveins commented on Sep 24, 2026

    @seathatflowsinourveins
    CollaboratorAuthor

    These lows come from the targeted review of #151's fix at ba57f96f. #151 is merged as 149940af.

    • Weak test pinning of the generic-word list. Only embedding is pinned. Removing the brokerage words, or any single word other than embedding, still passes all 113 lane-packet tests.
      • Pin the default-build restoration with assertIn("Brokerage model defaults include NullSlippageModel", …).
      • Compare the shared fields' text with assertEqual, not just their placeholder counts. The two builds are byte-identical at ba57f96f.
    • http is a no-op. It has 4 characters, and name parts need at least 5.
    • Newcomers mode redacts a candidate's own owner. "QuantConnect product347 entitlement" in LEAN Alpaca brokerage's card becomes " …", because dropping the refuted lean.brokerages.interactivebrokers leaves QuantConnect as a unique owner. Nothing leaks, but a little information is lost. Either extend the no-placeholder test to role/card_limitations with this exception recorded, or accept it.
    • Latent: two candidates differ from a sibling only by whole name. In the packet-wide matcher, this applies to claude-agent-sdk-typescript and LEAN Alpaca brokerage. So a future phrase like "the TypeScript SDK" or "the Alpaca brokerage" would pass unredacted. No such phrase exists today.

    🤖 Generated with Claude Code

  3. seathatflowsinourveins commented on Sep 24, 2026

    @seathatflowsinourveins
    CollaboratorAuthor

    Two pre-existing lows, found by the independent round-1 review of #206:

    • The retrieval-CLI list matches exact names only. A variant such as /usr/local/bin/serena-mcp-server, run as a system-prefix executable token, is neither refused by blind_path_issue nor flagged by the by-path rule. No such variant exists on this host. A possible fix is matching by name prefix or by a list of known entry points.
    • blind_path_issue doesn't scan the directories Codex itself prepends to a child's PATH: its bundled codex-path and the arg0 directory. Today codex-path holds only rg, measured 2026-09-24 with codex-cli 0.155.1.

    🤖 Generated with Claude Code

  4. seathatflowsinourveins commented on Sep 24, 2026

    @seathatflowsinourveins
    CollaboratorAuthor

    Two more lows from the independent round 2 of #206. Both predate #206 or are heuristic classes the README already documents:

    🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions