Skip to content

chore(llm): cost-hygiene batch — narrative-before-dedup, regional-brief outage retry storm, analyze-stock per-symbol fan-out, retry amplification #4896

Description

@koala73

Minor waste patterns from the 2026-07-05 OpenRouter spend investigation — none individually urgent (each ≲$0.5/day today), batched here so they're tracked. All file:line refs verified against origin/main 2026-07-05.

  1. Regional narrative fires before dedup. scripts/regional-snapshot/narrative.mjs (~900-tok call, 7 regions × 4 runs/day via seed-bundle-regional) is invoked in seed-regional-snapshots.mjs BEFORE persistSnapshot performs its identical-snapshot dedup — a byte-identical snapshot still burns the narrative call and then discards it. Move the LLM call behind the dedup check (or content-hash guard it).

  2. Regional weekly-brief cooldown-bypass = retry storm during outages. When briefs are empty (e.g. OpenRouter credit outage), shouldRunBriefs() bypasses the 6.5-day cooldown and re-runs all 7 region briefs every 6h until success (~28 brief calls/day vs intended ~7/week) — a burn spike that triggers exactly during provider incidents. Add backoff or a daily retry cap.

  3. analyze-stock: OpenRouter-first + per-symbol fan-out. server/worldmonitor/market/v1/analyze-stock.ts:1018 pins providerOrder: ['openrouter','generic'] (skips groq/ollama), and the client (src/services/stock-analysis.ts) sends one RPC per watchlist symbol with a 15-min cache. Currently ~111 calls/day, but scales linearly with users/watchlist size. Consider groq-first and/or batching symbols per call.

  4. Retry amplification re-sends full prompts. Forecast stages: MAX_RETRIES=3 → up to 4 attempts × 2 providers, re-POSTing 2,500-token prompts within a 120s stage budget. deduct-situation waits its full 120s timeout before re-sending the entire prompt to the next provider. callLlm's validate() rejection also re-sends the full prompt to the next provider. Worth a shared "spend budget per logical call" ceiling.

  5. classify-event per-headline boilerplate (post-fix(classify): stop free/anon users flooding the premium-gated classify-event endpoint (#4865) #4869 this is small: ~700 OpenRouter calls/day): each call re-sends a ~1.5KB taxonomy system prompt + few-shot examples for a 50-token output. Batching N headlines per call cuts the boilerplate ~N:1.

Context: the dominant burner (anon get-country-intel-brief cache-bust) is tracked separately in #4892; whyMatters shadow 2× in #4893; market_implications cache guard in #4894; LLM telemetry in #4895.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: AI/intelAI analysis, intelligence findings, summarization

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions