Skip to content

memory-lifecycle-v2 repairs: ai-memory's own upstream suites at v2.3.2/v2.4.0/v2.4.1, repaired fixture (GPT-6 reviewed) - #330

Merged
seathatflowsinourveins merged 1 commit into
mainfrom
claude/memory-lifecycle-v2-repairs-pr-20260926
Sep 26, 2026
Merged

seathatflowsinourveins merged 1 commit into
mainfrom
claude/memory-lifecycle-v2-repairs-pr-20260926

Conversation

@seathatflowsinourveins

Copy link
Copy Markdown
Collaborator

Summary

This PR repairs the GPT-6 cross-family findings on merged #292 (blueprints/memory-lifecycle-v2/) and adds ai-memory's own upstream test suites at the pinned release tags.

Upstream verification is native_proven: unchanged upstream suites at the release tags, run with the upstream CI command. Each receipt is under blueprints/memory-lifecycle-v2/upstream-20260926/.

Tag Command Exit Passed Failed Ignored
v2.3.2 cargo test --workspace --all-targets 0 3,357 0 11
v2.4.0 cargo test --workspace --all-targets 0 3,515 0 14
v2.4.1 cargo test --workspace --all-targets 0 3,615 0 14
v2.4.1 cargo test --workspace --doc 0 0 (no tests selected) 0 0

The v2.4.1 doc-test run selected no tests, so it is recorded as untested. Round 1 of these runs was superseded and is kept under superseded-round-1/. A discriminating control, filed separately under discriminating_control and labelled local_integration, runs pass, fail, fail, pass against mutated checkouts. control.py now exits nonzero unless that exact sequence occurs and the checkout is restored.

Repaired fixture rerun (local_integration: a locally authored fixture that drives the official binaries over stdio MCP inside bwrap with no network). Each binary passed on its first attempt; the source is results-20260926.json:

Binary Phase pre Phase post
ai-memory 2.3.2 82 / 82 19 / 19
ai-memory 2.4.0 82 / 82 19 / 19
ai-memory 2.4.1 82 / 82 19 / 19

What was wrong, and the fix (GPT-6 findings F1–F8, all verified)

  • Analyzer:
    • A missing hit is no longer scored as empty.
    • Rejections must carry the exact code and full message.
    • Multi-project identity is checked by page id and scope.
    • Swept pages must be unreadable, both after the sweep and after a restart.
    • TTL ordering is checked against the server's expired_at.
  • Retained evidence: sanitized per-phase native MCP responses with hashes for 3 runs (runs-20260926/) and the 2 surviving 2026-09-25 runs (runs-20260925/). Each run has a publication.json binding every public file to its private source's sha256.
  • Failed attempts: all 7 2026-09-25 attempts are listed in history/attempts-20260925.json. The original preregistration (ca81e9b4…, before any run) is kept byte-identical beside the amended one (fb82e0aa…).
  • Timestamps: taken from the retained receipts.
  • Claims: the "live quality gate" overclaim is withdrawn, and the fixture is relabelled local_integration.
  • Tooling fixes:
    • assemble_results.py reports every attempt.
    • run_upstream_tests.py refuses to overwrite an earlier log or receipt.
    • setup.sh's header no longer claims a pre-run rustup-init checksum; the binary was compared after the runs (rustup-init-verification.json).
    • The historical hash log carries a publication record for its <scratch> substitution.

Review

  1. GPT-6 review: found F1–F8 on Memory lifecycle v2: ai-memory 2.3.2 and 2.4.0 pass 89/89 checks each (real-time TTL, 4-project scope, restart durability) #292, posted there.
  2. Opus rounds: an Opus repair, an independent Opus review, then an Opus finisher and an Opus verification.
  3. GPT-6 verification pass: confirmed F1 and F3–F8 resolved, and F2 resolved in the repository. It found two mediums, the upstream runner overwriting logs and the control exiting 0 regardless, both fixed with failing-first tests.
  4. Independent headless review: checked the published files.

The #292 description is corrected in a comment on #292.

Scope notes

  • The grand-dashboard row for memory-lifecycle-v2 changes in a separate lane:shared PR, because observability/grand-dashboard/state.json is a shared hot file. DashboardCheckpointTests checks the row's date once it cites the 2026-09-26 results, and skips until then.
  • The memory-lifecycle gate in catalogs/us-equities/convergence-review.json stays open. Live-store policy, scoped retrieval quality and evaluated learning were not exercised.

Verification

  • Full python -m unittest on this tree: Ran 6041 tests, OK (skipped=628).
  • scripts/validate.py, host_receipts.py validate, component_matrix.py --check, new_host_grand_list.py --check, evidence_manifest.py --check and validate_catalogs.py: all pass.

SOTA sources

🤖 Generated with Claude Code

… repaired fixture, retained evidence

Repairs the GPT-6 cross-family findings on #292: analyzer rules, retained sanitized native
responses with publication records, preserved failed attempts, corrected timestamps and evidence
classes; adds ai-memory's own cargo test --workspace --all-targets at v2.3.2, v2.4.0 and v2.4.1.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
@seathatflowsinourveins seathatflowsinourveins added the lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers label Sep 26, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@seathatflowsinourveins
seathatflowsinourveins merged commit e372369 into main Sep 26, 2026
24 checks passed
@seathatflowsinourveins
seathatflowsinourveins deleted the claude/memory-lifecycle-v2-repairs-pr-20260926 branch September 26, 2026 10:41
seathatflowsinourveins added a commit that referenced this pull request Sep 26, 2026
…und (#340)

memory-lifecycle-v2 and retrieval-quality-v2 cite their repaired 2026-09-26 results (#330, #335);
token-efficiency-wiring states the retained 2026-09-26 coverage run (#333).

Co-authored-by: Scout <scout@local>
Co-authored-by: Claude Opus 5.5 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lane:foundation Foundation lane: Claude/Codex setup, hosts, memory, RAG, research, workers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant