chore(llm): regional hygiene — narrative prompt-hash cache + brief retry-budget cap by koala73 · Pull Request #4911 · koala73/worldmonitor

koala73 · 2026-07-05T17:52:32Z

What ships (#4896 items 1–2)

1. Narrative prompt-hash cache. generateRegionalNarrative fired its ~900-token LLM call before any content-identity check: byte-identical world state (7 regions × 4 seed-bundle-regional runs/day) regenerated the same narrative every 6h, and same-15-min-bucket re-runs burned the call just for persistSnapshot's idempotency guard to discard the result. The generator now caches the parsed narrative under intelligence:narrative-cache:v1:<region>:<sha16(prompt)> (24h TTL — the prompt's only volatile input is a day-granular date). Injectable opts.cache for tests; best-effort on Redis failure; only valid parsed narratives are cached so a garbage response can't be pinned.

2. Retry-budget cap on the weekly-brief coverage-fail bypass. The #2989 self-heal bypass re-ran all 7 region briefs on every 6h tick for as long as an OpenRouter outage lasted (~28 LLM calls/day, timed exactly to provider incidents), because each failed attempt refreshes fetchedAt. A rolling 24h budget (INCR + EXPIRE NX, 2 bypasses/window) keeps the fast transient self-heal — first retry on the next tick, one more after — then pauses until the window expires. Fails open on Redis errors (a hiccup can never block self-healing); the scheduled weekly cadence never consults the budget.

Tests

New tests/regional-narrative-cache.test.mjs (5): cache hit serves provider=cache with the producing model and zero LLM calls; changed content misses; invalid generations never cached; throwing cache never blocks generation; global region short-circuits untouched.
tests/seed-bundle-regional-briefs-cooldown.test.mjs +4: bypass fires with budget room, suppressed at exhaustion, fails open on pipeline errors, and the weekly cadence never consults the budget. Full regional suites 115/115 ✓, biome ✓.

Refs #4896 (items 3–5 dispositioned on the issue)

https://claude.ai/code/session_01YTm4GsLG2M7ZuaHF9MpZih

…try-budget cap Two of the #4896 waste patterns: 1. Narrative-before-dedup: generateRegionalNarrative fired its ~900-token call before any content-identity check — byte-identical world state (7 regions x 4 bundle runs/day) regenerated the same narrative every 6h, and same-15min-bucket re-runs burned it just for persistSnapshot's idempotency guard to discard the result. The generator now caches the parsed narrative under a hash of the exact prompt (whose only volatile input is a day-granular date), 24h TTL, injectable for tests, best-effort on Redis failure. Only VALID parsed narratives are cached. 2. Coverage-fail retry storm: the #2989 cooldown bypass re-ran ALL region briefs on every 6h tick for as long as an outage lasted (~28 LLM calls/day, timed exactly to provider incidents) because each failed attempt refreshes seed-meta fetchedAt. A rolling 24h retry budget (INCR+EXPIRE NX, 2 bypasses/window) keeps the fast transient self-heal — first retry on the next tick, one more after — then pauses until the window expires. Fails OPEN on Redis errors so a hiccup can never block self-healing; the scheduled weekly cadence never consults the budget. Refs #4896 Claude-Session: https://claude.ai/code/session_01YTm4GsLG2M7ZuaHF9MpZih

vercel · 2026-07-05T17:52:34Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
worldmonitor	Ignored	Preview	Jul 5, 2026 6:03pm

greptile-apps · 2026-07-05T17:58:38Z

Greptile Summary

This PR introduces two Redis-backed hygiene improvements for the regional intelligence pipeline. It adds a prompt-hash cache to generateRegionalNarrative (SHA-256 over the serialized prompt, 24h TTL, only valid parsed narratives stored) so byte-identical world state within a calendar day reuses a cached narrative instead of re-billing the LLM, and adds a rolling 24h retry-budget cap to the weekly-brief coverage-fail bypass in shouldRunBriefs (INCR + EXPIRE NX pipeline, 2 bypasses/window, fails open on Redis errors) so a sustained provider outage can no longer re-run all 7 region briefs on every 6h tick.

scripts/regional-snapshot/narrative.mjs: new defaultNarrativeCache() factory using Upstash REST; injected cache reads before the LLM call and writes only on valid parse; cache errors are fully isolated.
scripts/seed-bundle-regional.mjs: new consumeBriefRetryBudget() using a pipeline of INCR + EXPIRE NX; the budget gate sits only on the coverage-fail bypass path — the normal weekly cadence is unaffected.
Both new paths are covered by five new narrative-cache tests and four new briefs-cooldown tests.

Confidence Score: 4/5

Safe to merge. Both new paths (narrative cache and retry-budget cap) fail open on Redis errors and are thoroughly covered by the nine new tests. The only finding is a missing User-Agent header in the new Redis fetch helpers.

The cache-miss fallback and open-circuit error handling are solid: a Redis outage can't block narrative generation or self-healing brief retries. The INCR + EXPIRE NX budget pattern is correct for a fixed-window counter, the SHA-256 truncation to 16 hex chars is adequate collision resistance for the 7-region key space, and the test suite directly exercises every new branch including the global-region short-circuit and the throwing-cache isolation. The only gap is a User-Agent omission in the two new Redis fetch helpers, which is a convention violation rather than a functional defect.

The new defaultNarrativeCache factory in narrative.mjs and consumeBriefRetryBudget in seed-bundle-regional.mjs both make Redis REST calls without a User-Agent header.

Important Files Changed

Filename	Overview
scripts/regional-snapshot/narrative.mjs	Adds prompt-hash cache (SHA-256/16-hex key, 24h TTL) to generateRegionalNarrative; cache is dependency-injected, best-effort, and only stores valid parsed narratives. New `defaultNarrativeCache` fetch calls are missing the `User-Agent` header required by AGENTS.md.
scripts/seed-bundle-regional.mjs	Adds `consumeBriefRetryBudget` (INCR + EXPIRE NX pipeline, 2/24h, fails open) gating the coverage-fail bypass in `shouldRunBriefs`. Logic is correct; normal weekly cadence path is unaffected. New pipeline fetch is also missing `User-Agent`.
tests/regional-narrative-cache.test.mjs	Five new tests cover cache hit/miss, invalid-response exclusion, throwing-cache isolation, and global-region short-circuit. Well-structured with injected cache and callLlm for full isolation.
tests/seed-bundle-regional-briefs-cooldown.test.mjs	Four new tests cover budget-available bypass, budget-exhausted suppression, Redis-failure open-circuit, and weekly-cadence bypass. Stub correctly distinguishes GET vs POST calls and budget state.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant S as seed-bundle-regional
    participant SRB as shouldRunBriefs
    participant CB as consumeBriefRetryBudget
    participant GRN as generateRegionalNarrative
    participant Cache as Upstash Cache
    participant LLM as LLM Provider

    S->>SRB: shouldRunBriefs()
    SRB->>Cache: GET seed-meta (fetchedAt, recordCount)
    Cache-->>SRB: meta

    alt "age >= 6.5 days (weekly cadence)"
        SRB-->>S: true (run, no budget check)
    else "recordCount == 0 (coverage-fail bypass)"
        SRB->>CB: consumeBriefRetryBudget(url, token)
        CB->>Cache: POST /pipeline [INCR, EXPIRE NX]
        Cache-->>CB: count
        alt "count <= 2"
            CB-->>SRB: true (budget available)
            SRB-->>S: true (run briefs)
        else "count > 2"
            CB-->>SRB: false (budget exhausted)
            SRB-->>S: false (skip until window resets)
        end
    else "recordCount > 0 (still in cooldown)"
        SRB-->>S: false (skip)
    end

    Note over S,LLM: For each region (when briefs run)
    S->>GRN: generateRegionalNarrative(region, snapshot, evidence, opts)
    GRN->>GRN: build prompt + SHA-256 cache key
    GRN->>Cache: "GET intelligence:narrative-cache:v1:{region}:{hash16}"
    alt Cache hit (valid parsed narrative)
        Cache-->>GRN: "{narrative, model}"
        GRN-->>S: "{narrative, provider:'cache', model}"
    else Cache miss
        Cache-->>GRN: null
        GRN->>LLM: "callLlm(prompt, {validate})"
        LLM-->>GRN: "{text, provider, model}"
        GRN->>GRN: parseNarrativeJson (valid?)
        alt valid narrative
            GRN->>Cache: "SET key {narrative, model} EX 86400"
            GRN-->>S: "{narrative, provider, model}"
        else invalid / empty
            GRN-->>S: "{emptyNarrative, provider:'', model:''}"
        end
    end

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant S as seed-bundle-regional
    participant SRB as shouldRunBriefs
    participant CB as consumeBriefRetryBudget
    participant GRN as generateRegionalNarrative
    participant Cache as Upstash Cache
    participant LLM as LLM Provider

    S->>SRB: shouldRunBriefs()
    SRB->>Cache: GET seed-meta (fetchedAt, recordCount)
    Cache-->>SRB: meta

    alt "age >= 6.5 days (weekly cadence)"
        SRB-->>S: true (run, no budget check)
    else "recordCount == 0 (coverage-fail bypass)"
        SRB->>CB: consumeBriefRetryBudget(url, token)
        CB->>Cache: POST /pipeline [INCR, EXPIRE NX]
        Cache-->>CB: count
        alt "count <= 2"
            CB-->>SRB: true (budget available)
            SRB-->>S: true (run briefs)
        else "count > 2"
            CB-->>SRB: false (budget exhausted)
            SRB-->>S: false (skip until window resets)
        end
    else "recordCount > 0 (still in cooldown)"
        SRB-->>S: false (skip)
    end

    Note over S,LLM: For each region (when briefs run)
    S->>GRN: generateRegionalNarrative(region, snapshot, evidence, opts)
    GRN->>GRN: build prompt + SHA-256 cache key
    GRN->>Cache: "GET intelligence:narrative-cache:v1:{region}:{hash16}"
    alt Cache hit (valid parsed narrative)
        Cache-->>GRN: "{narrative, model}"
        GRN-->>S: "{narrative, provider:'cache', model}"
    else Cache miss
        Cache-->>GRN: null
        GRN->>LLM: "callLlm(prompt, {validate})"
        LLM-->>GRN: "{text, provider, model}"
        GRN->>GRN: parseNarrativeJson (valid?)
        alt valid narrative
            GRN->>Cache: "SET key {narrative, model} EX 86400"
            GRN-->>S: "{narrative, provider, model}"
        else invalid / empty
            GRN-->>S: "{emptyNarrative, provider:'', model:''}"
        end
    end

_{Reviews (1): Last reviewed commit: "chore(llm): regional hygiene — narrative..." | Re-trigger Greptile}

greptile-apps · 2026-07-05T17:58:41Z

+      const resp = await fetch(`${url}/get/${encodeURIComponent(key)}`, {
+        headers: { Authorization: `Bearer ${token}` },
+        signal: AbortSignal.timeout(3_000),
+      });
+      if (!resp.ok) return null;
+      const data = await resp.json();
+      return data?.result ? JSON.parse(data.result) : null;
+    },
+    async set(key, value, ttlSeconds) {
+      const { url, token } = getRedisCredentials();
+      await fetch(url, {
+        method: 'POST',
+        headers: { Authorization: `Bearer ${token}`, 'Content-Type': 'application/json' },
+        body: JSON.stringify(['SET', key, JSON.stringify(value), 'EX', String(ttlSeconds)]),
+        signal: AbortSignal.timeout(3_000),
+      });


Missing User-Agent header in new Redis fetch calls

AGENTS.md mandates "Always include User-Agent header in server-side fetch calls." Both the get (line 475) and set (line 485) methods in defaultNarrativeCache omit it. The same gap appears in consumeBriefRetryBudget in seed-bundle-regional.mjs (line 57). Upstash itself doesn't enforce this, but the convention exists for consistent server-side request attribution — the LLM provider calls in this same file already include the User-Agent via provider.headers(key), so the Redis paths are now inconsistent with the rest of the module.

Context Used: AGENTS.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

…creds are absent getRedisCredentials() exits the process on missing env — the default narrative cache killed the CI unit runner (and any credless local run) the moment generateRegionalNarrative touched it without an injected cache. Read env directly and no-op without creds: no cache, generation proceeds exactly as before the cache existed. Caught by the unit job on this PR; reproduced locally with UPSTASH_* unset. Claude-Session: https://claude.ai/code/session_01YTm4GsLG2M7ZuaHF9MpZih

…ket-brief context quantization, forecast prompt-hash cache, whyMatters v8 cross-read (#4914) (#4916) * chore(llm): cache-identity churn batch 2 — stop paid regeneration of byte-identical prompts (#4914) - summary cache key: dedup identical (headline, body) pairs before the top-5 slice (v6→v7) — the server prompt path dedups AFTER the key was computed, so the same unique story set with a different duplicate composition minted distinct keys for an identical prompt - daily market brief: quantize every float interpolated into the summarizer geoContext (index moves 0.25pp, VIX 1pt, HY 10bps, 2s10s 5bps, yields 0.1pp, sector 0.5pp) — raw 5-min-tick floats became the key's :g segment and defeated the 24h-class TTL per tick and per user - forecast narratives: key the combined/scenario caches on the exact prompt text (the #4911 prompt-hash pattern) instead of raw floats the prompt never renders; TTL 1h→24h (prompt-hash keys self-invalidate, and the old TTL expired between hourly runs by construction) - whyMatters: the cron fallback now reads the analyst endpoint's v8 envelope (same hashBriefStory identity) before paying a direct-Gemini generation; read-only — legacy output stays in the v5 namespace Closes #4914 🤖 Generated with Claude Code Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP * test: update summarize-reasoning CACHE_VERSION pin v6→v7 (missed in the #4914 bump) Claude-Session: https://claude.ai/code/session_01WmhkGjZrb9RbV7pV3muFxP

koala73 enabled auto-merge (squash) July 5, 2026 17:52

koala73 mentioned this pull request Jul 5, 2026

chore(llm): cost-hygiene batch — narrative-before-dedup, regional-brief outage retry storm, analyze-stock per-symbol fan-out, retry amplification #4896

Closed

vercel Bot deployed to Preview July 5, 2026 17:56 View deployment

greptile-apps Bot reviewed Jul 5, 2026

View reviewed changes

koala73 merged commit 2d2d659 into main Jul 5, 2026
24 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore(llm): regional hygiene — narrative prompt-hash cache + brief retry-budget cap#4911

chore(llm): regional hygiene — narrative prompt-hash cache + brief retry-budget cap#4911
koala73 merged 2 commits into
mainfrom
chore/regional-llm-hygiene

koala73 commented Jul 5, 2026

Uh oh!

vercel Bot commented Jul 5, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Jul 5, 2026

Uh oh!

greptile-apps Bot Jul 5, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

koala73 commented Jul 5, 2026

What ships (#4896 items 1–2)

Tests

Uh oh!

vercel Bot commented Jul 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps Bot commented Jul 5, 2026

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Sequence Diagram

Uh oh!

greptile-apps Bot Jul 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented Jul 5, 2026 •

edited

Loading