chore(llm): cache-identity churn batch 2 — summary pair-dedup v7, market-brief context quantization, forecast prompt-hash cache, whyMatters v8 cross-read (#4914)#4916
Conversation
…byte-identical prompts (#4914) - summary cache key: dedup identical (headline, body) pairs before the top-5 slice (v6→v7) — the server prompt path dedups AFTER the key was computed, so the same unique story set with a different duplicate composition minted distinct keys for an identical prompt - daily market brief: quantize every float interpolated into the summarizer geoContext (index moves 0.25pp, VIX 1pt, HY 10bps, 2s10s 5bps, yields 0.1pp, sector 0.5pp) — raw 5-min-tick floats became the key's :g segment and defeated the 24h-class TTL per tick and per user - forecast narratives: key the combined/scenario caches on the exact prompt text (the #4911 prompt-hash pattern) instead of raw floats the prompt never renders; TTL 1h→24h (prompt-hash keys self-invalidate, and the old TTL expired between hourly runs by construction) - whyMatters: the cron fallback now reads the analyst endpoint's v8 envelope (same hashBriefStory identity) before paying a direct-Gemini generation; read-only — legacy output stays in the v5 namespace Closes #4914 🤖 Generated with Claude Code Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Greptile SummaryThis PR eliminates four "volatile value folded into cache identity" patterns that were defeating long-TTL LLM caches, following the investigation started in #4892/#4911. All changes are targeted fixes with failing-first tests.
Confidence Score: 4/5Safe to merge; all changes are cache-identity fixes with no write-path side effects. The four targeted changes are well-scoped: the summary dedup is a pre-slice filter with no downstream write impact, the prompt-hash cache for forecasts is correct by construction, the v8 cross-read is strictly read-only, and the quantization logic is straightforward. One FSI value in daily-market-brief was precision-reduced (2dp→1dp) but not given an explicit quantize step like the other fields — a sub-0.1 FSI drift within a market session can still generate a distinct src/services/daily-market-brief.ts — the fsiValue line is the only value that wasn't given an explicit step bucket; all other changed files look clean. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[generateWhyMatters called] --> B{callAnalystWhyMatters\nconfigured?}
B -- yes --> C[Call analyst endpoint]
C -- valid string --> D[Return result]
C -- null / throws / out-of-bounds / stub --> E
B -- no --> E[Read v8 cache: brief:llm:whymatters:v8 - READ-ONLY NEW]
E -- valid envelope --> F[Return v8.whyMatters]
E -- miss / malformed / stub --> G[Read v5 cache: brief:llm:whymatters:v5]
G -- cache hit --> H[Return cached string]
G -- miss --> I[Call Gemini directly]
I -- success --> J[Write to v5, return result]
I -- failure --> K[Return null to caller]
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[generateWhyMatters called] --> B{callAnalystWhyMatters\nconfigured?}
B -- yes --> C[Call analyst endpoint]
C -- valid string --> D[Return result]
C -- null / throws / out-of-bounds / stub --> E
B -- no --> E[Read v8 cache: brief:llm:whymatters:v8 - READ-ONLY NEW]
E -- valid envelope --> F[Return v8.whyMatters]
E -- miss / malformed / stub --> G[Read v5 cache: brief:llm:whymatters:v5]
G -- cache hit --> H[Return cached string]
G -- miss --> I[Call Gemini directly]
I -- success --> J[Write to v5, return result]
I -- failure --> K[Return null to caller]
Reviews (1): Last reviewed commit: "test: update summarize-reasoning CACHE_V..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| if (regime.fsiValue > 0) lines.push(`FSI: ${regime.fsiValue.toFixed(2)} (${regime.fsiLabel})`); | ||
| if (regime.vix > 0) lines.push(`VIX: ${regime.vix.toFixed(1)}`); | ||
| if (regime.hySpread > 0) lines.push(`HY Spread: ${regime.hySpread.toFixed(0)}bps`); | ||
| if (regime.fsiValue > 0) lines.push(`FSI: ${regime.fsiValue.toFixed(1)} (${regime.fsiLabel})`); |
There was a problem hiding this comment.
regime.fsiValue was reduced from toFixed(2) to toFixed(1), but unlike VIX (1-pt bucket), HY Spread (10 bps), yields (0.1pp), and sector moves (0.5pp), FSI has no named quantize step. A sub-0.1 drift — e.g., 1.24 → 1.26 — still produces distinct :g segments ("1.2" vs "1.3") and mints a new key. The test's drift=0.03 happens not to cross a 0.1 boundary, so this edge case isn't covered. If the FSI feeds from the same seeded quotes pipeline as VIX, the remaining variance could still generate spurious misses.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
1 participant
Closes #4914. Follow-up to the 2026-07-05 spend investigation: #4892 fixed the dominant burner (country-intel contextHash), #4911 the regional narratives, #4909 the whyMatters shadow double. This batch removes the same "volatile value folded into cache identity → TTL defeated → paid churn" class from the three remaining sites, plus one cross-namespace blind spot.
Changes
1. Summary cache key dedups identical (headline, body) pairs —
src/utils/summary-cache-key.ts, v6→v7The server prompt path (
summarize-article.ts:171-193) dedups pairs AFTER the key is computed (:156), so the same unique story set with a different duplicate composition minted distinct keys for an identical prompt — every composition variant was a paid cache miss. Dedup now runs pre-slice in the shared builder (client + server stay aligned). Exact pairs only: a repeated headline with a different body stays a distinct key. Version bump retires the old rows cleanly.2. Daily market brief quantizes the summarizer geoContext —
src/services/daily-market-brief.tsVIX: 18.24,HY Spread: 342bps, yields to 2dp, sector moves to 0.1pp, index changes to 0.1pp all fed the key's:gsegment; every 5-min quote/regime tick and every user minted a fresh key against a 24h-class TTL. All prompt-interpolated floats are now bucketed (index 0.25pp, VIX 1pt, HY 10bps, 2s10s 5bps, yields 0.1pp, sector 0.5pp) so the prompt — and therefore the key — only shifts on market-meaningful moves. Prompt/key consistency is by construction (same string).3. Forecast narrative caches key on the exact prompt text —
scripts/seed-forecasts.mjsbuildCacheHashhashed raw probability floats, ALL newsContext entries, and cascade floats while the prompt renders integer percents and the top-3 headlines — hourly drift minted fresh keys for byte-identical prompts, and the 3600s TTL expired between hourly runs anyway. NowbuildNarrativeCacheHash(SYSTEM_PROMPT, buildUserPrompt(preds))(the #4911 pattern; a system-prompt edit self-invalidates), TTL 24h (prompt-hash keys make staleness impossible; TTL is only an eviction bound).4. whyMatters cron fallback reads the endpoint's v8 envelope —
scripts/lib/brief-llm.mjsbrief:llm:whymatters:v5(cron fallback) and:v8(analyst endpoint) sharehashBriefStoryidentity but were disjoint: a transient endpoint failure paid a direct-Gemini generation for a story with a valid v8 envelope already in Redis. The fallback now reads v8 first (bounds-validated, sensitivity-stub rejected). Read-only — the legacy single-sentence output stays in v5, so the two prompt contracts never cross-contaminate in the write direction.Verification
tests/summary-cache-key.test.mts(18),tests/daily-market-brief.test.mts(12, incl. geoContext byte-identity under drift + material-move busting),tests/forecast-narrative-cache-hash.test.mjs(5, new),tests/brief-llm.test.mjs(107, incl. v8 reuse / malformed-envelope fall-through / read-only namespace).summary-cache-capacity(3),forecast-llm-telemetry(4).npm run typecheck+npm run typecheck:api+ biome: clean.Remaining item from the investigation — the digest generator running paid synthesis for every eligible user on every 30-min tick regardless of
isDue— is product-behavioral (dashboard-refresh contract) and tracked for its own PR in #4914's closing note.https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP