perf: profile public recall latency with native reranker and replay costs separated #722

Problem

On the fixed SciFact campaign, Cortex warm public MCP latency averaged 3.550896 s (p95 4.105969 s) versus Hindsight HTTP 1.567441 s (p95 1.857700 s). These are configured product measurements, not an isolated algorithm comparison or a performance regression. Native telemetry confirms 150/150 Cortex recalls succeeded on PG with 300 passages reranked and 100 memories returned.

Proposed solution

Profile retrieval, embedding, cross-encoder inference, enrichments, persistent replay and response serialization before choosing an optimization. Preserve retrieval quality and intended memory-state effects.

Acceptance evidence

• Reproduce with exact corpus, revisions, models, CPU and public-call parameters.
• Provide a measured latency breakdown and actual ONNX/PyTorch thread behavior.
• Any optimization has before/after quality, latency and persistence-correctness evidence under unchanged conditions.
• Keep first-use/setup latency, warm latency and transport differences distinct.

Alternatives considered

Keep the current behavior and document its limits. This remains valid where the added guarantee has no user need; the issue should close with a documented product decision rather than assume that copying Hindsight is always appropriate. Reuse existing Cortex mechanisms before adding a second pipeline.

Scope assessment

MEDIUM. Raised from the October 10 Hindsight/Cortex comparison. Static observations refer to the pinned revisions below. They do not establish successful execution of Hindsight's whole pipeline or a repository-wide absence in Cortex. Priority is a proposal, not a release blocker.

Evidence

mcp_server/core/reranker.py:156; mcp_server/infrastructure/pg_schema.py:1190; mcp_server/handlers/replay_tracking.py:54

Conditions: Apple M4, 10 CPUs, 24 GiB RAM; separate ephemeral PostgreSQL17.9 databases; CPU models. Cortex all-MiniLM-L6-v2 and FlashRank ms-marco-MiniLM-L-12-v2; Hindsight BAAI/bge-small-en-v1.5 and cross-encoder/ms-marco-MiniLM-L-6-v2. 50 queries, first sweep excluded from warm timing, AB then reversed BA, 100 warm calls per arm. nDCG@10 H=.88664, C=.87956/.87695; Recall@10 H=.956, C=.936. Hindsight fixture imports prebuilt entities/relations, output volumes differ, and Cortex recalls mutate memory state. No general speed claim or causal attribution is established.

Related: #709 covers mechanism-benefit ablations; this issue concerns the latency breakdown of actual public recall.