roadmap: track Hindsight comparison and local-first product readiness #725

Objective

Make Cortex a usable local-first memory product, using the Hindsight comparison to identify concrete gaps. This tracks an ongoing investigation and a proposed backlog; it is not a claim that Hindsight is superior on every axis or that the comparison is complete.

Delivery ownership confirmed by the owner, October 10

This workspace owns macOS corrections and native Claude Code/Codex verification against the 18 acceptance journeys. The owner's collaborator performs Windows verification using the same invariants and artifact identities. Linux qualification is deferred and does not block a macOS-qualified release. Windows support claims await that collaborator's receipts; Linux remains explicitly unverified. This assignment preserves the full macOS product expectations, including documentation and visual inspection.

Evidence already collected

Pinned Cortex revision: 02c3cf006e7024811bcb379ce375e7eafd8267a9.
Pinned Hindsight revision: 44d5340148fd47efb34e184d61f87ab5e2d8d1e2.
Companion inspected read-only: cdeust/cortex-viz@2482563172c0f02a1c022b82cb4e38299b7748b4.

Static source comparison, public-API retrieval measurement and live Hindsight documentation rendering were inspected. The retrieval experiment used 500 SciFact documents, 1812 shared facts and 50 questions. Warm mean latency was 1.5674s for Hindsight versus 3.5509s for Cortex; nDCG@10 was 0.8866 versus 0.8769–0.8796. These results concern this retrieval setup only: different embeddings/rerankers, prebuilt Hindsight graph, PostgreSQL stores and no extraction or final-answer measurement. They do not establish default SQLite performance, general product quality or service-level guarantees. Raw response SHA256: 9ca9738a6605440a867334e44b73b35b5f666f3221363c4e49f68f8d14e7e175.

Project evidence is retained locally under rea/_reference/hindsight-cortex-2026-10-10/ (report, manifest, benchmark protocol/results/raw responses, rendered documentation captures). Findings and acceptance evidence are also recorded in each child issue below.

New issues

• feat: preserve source segments and document versions through memory and claim retrieval #713 : feat: preserve source segments and document versions through memory and claim retrieval
• fix: expose partial document ingestion and resume missing writes idempotently #714 : fix: expose partial document ingestion and resume missing writes idempotently
• feat: define snapshot and freshness guarantees for memory-derived syntheses #715 : feat: define snapshot and freshness guarantees for memory-derived syntheses
• feat: specify and verify resumable incremental transcript capture across Codex rollouts #716 : feat: specify and verify resumable incremental transcript capture across Codex rollouts
• feat: preserve event time separately from extraction time in ClaimEvent #717 : feat: preserve event time separately from extraction time in ClaimEvent
• feat: define opt-in private agent memory scope consistently across retrieval paths #718 : feat: define opt-in private agent memory scope consistently across retrieval paths
• proposal: optional cited server-side reflect for clients without an orchestration LLM #719 : proposal: optional cited server-side reflect for clients without an orchestration LLM
• bench: extend Hindsight comparison beyond retrieval to extraction, answers, freshness and recovery #720 : bench: extend Hindsight comparison beyond retrieval to extraction, answers, freshness and recovery
• fix: avoid silently remapping explicit domain identities through substring collisions #721 : fix: avoid silently remapping explicit domain identities through substring collisions
• perf: profile public recall latency with native reranker and replay costs separated #722 : perf: profile public recall latency with native reranker and replay costs separated
• docs: build a guided, searchable and versioned local-first documentation experience #723 : docs: build a guided, searchable and versioned local-first documentation experience
• feat: unify local memory inspection, source tracing and visualization in the product journey #724 : feat: unify local memory inspection, source tracing and visualization in the product journey

Existing work to reuse

• bench: the longitudinal harness measures retrieval on synthetic memories, not agent outcome on a real repository #343: agent outcomes on real repository work.
• feat(wiki): incremental regeneration so the generated wiki does not rot #359: incremental generated-wiki regeneration.
• Install path: the postInstall field is ignored by the host, so no backend is set up and remember fails without PostgreSQL #703, A profile with no configuration connects to a local PostgreSQL it was never configured for #704, First SessionStart installs 1.1 GB inside a 30 s hook, and the downloads are missing from PRIVACY #705: installation, backend selection and first-use reliability.
• Automatic injection finds the target 60 percent of the time against 97 percent for recall, and default recall hides user memories by tag #706: automatic context injection versus explicit recall.
• Headline retrieval numbers describe PostgreSQL; the default SQLite store has no published number #708: evidence for the default SQLite backend.
• The 36 neuroscience mechanisms are not shown to improve retrieval in the committed ablations #709: ablation evidence for the memory mechanisms.

Validation campaigns required before product conformance

• Extraction and normalization on identical source documents, including contradictory and dated facts.
• Final answers: correctness, source fidelity, abstention and reader-model budget.
• Correction/deletion propagation, synthesis freshness and interrupted-ingestion recovery.
• End-to-end first local install, default backend, actual companion rendering and source inspection.
• Operational costs, resource use, offline behavior, isolation and cross-client contracts.
• Documentation usability on a built artifact: discoverability, first success, troubleshooting, accessible rendering and offline assets.

Prioritize a reliable first local memory round-trip, then source inspection and a guided documentation journey. Reuse cortex-viz; its existing visualizations should be assessed, not replaced on the assumption that no UI exists. Resolve backend capability gaps explicitly. The documentation and rendering work must demonstrate real product behavior.

Primary references

Hindsight documentation, Cortex README, cortex-viz README.

Global product audit extension, October 10

The investigation now covers product promise, knowledge lifecycle, native Claude/Codex integration, distribution, platform support, offline operation, backup/restore, model identity, costs, documentation and inspection. Coverage of the analysis does not establish native conformance of every product combination.

Executed during the extension:

• Four production stdio smokes passed (Claude/Codex client names, full/lean, discovery and memory_stats), without native host lifecycle delivery.
• Offline profile replays demonstrate missing Codex bootstrap and naming-dependent behavioral observations.
• A real safe_handler boundary probe shows four cancelled callers leave four workers active; a fifth request starts despite the declared budget of four. Native MCP cancellation delivery is a separate acceptance case.
• GitHub wheel/MCPB checksums match; each of 660 engine Python members matches the v4.23.5 tag. Forty-two differ from current main. Installed uv/PyPI runtime bytes were not inspected.

New child issues:

• fix: bootstrap cognitive profiles from supported Codex conversation history #726: fix: bootstrap cognitive profiles from supported Codex conversation history
• fix: normalize comparable behavioral observations while preserving native tool identity #727: fix: normalize comparable behavioral observations while preserving native tool identity
• fix: retain admission ownership while cancelled handler workers remain active #728: fix: retain admission ownership while cancelled handler workers remain active
• release: bind parity claims to the distributed plugin and runtime artifacts #729: release: bind parity claims to the distributed plugin and runtime artifacts
• test: gate platform support claims with native Claude and Codex journey receipts #730: test: gate platform support claims with native Claude and Codex journey receipts
• feat: provide a verified portable backup and restore journey for local memory #731: feat: provide a verified portable backup and restore journey for local memory
• feat: preserve document attachments and their provenance through local retrieval #732: feat: preserve document attachments and their provenance through local retrieval
• feat: expose local operation receipts with attributable resource consumption #733: feat: expose local operation receipts with attributable resource consumption
• feat: report effective model identities and capabilities for reproducible local runs #734: feat: report effective model identities and capabilities for reproducible local runs

The proposed acceptance contract has 18 journeys across 12 candidate OS/host/backend cells. This is a validation design, not a support claim. Each receipt identifies artifact digest, client/OS/interpreter, backend, model manifest, fixture, native events and observed effects. Current macOS receipts remain bound to their original versions. Windows full-suite debt (#692/#693), macOS SQLite vector support (#691), install/default-backend issues (#703–#705) and measurement gaps (#708/#709) remain linked rather than duplicated.

Release ordering: establish install/runtime/store identity and first successful round-trip; establish document/source/correction recovery; prove native host continuity; publish measured baselines and recovery evidence; integrate documentation and visual inspection. Existing closed repairs to artifact/claim deletion and lineage are acknowledged, not refiled as absent features.

Local report: rea/_reference/hindsight-cortex-2026-10-10/global/rapport-global.md; machine-readable acceptance contract: global/contrat-acceptation.json. These local paths are evidence locations, not downloadable public URLs. Each child contains primary-source permalinks and its acceptance evidence.

Historical evidence refresh: the public v4.23.4 native receipt was downloaded and its published checksum verified. It records macOS shared-PostgreSQL transfer in both directions, project exclusions and automatic startup context, plus the initial tool-discovery failure. It remains evidence for that release and platform, not full native coverage of current main.