Skip to content

chore(digest): BRIEF_WHY_MATTERS_SHADOW is default-on at 100% sampling — every cache-miss runs TWO gemini calls, one discarded #4893

Description

@koala73

Found during the 2026-07-05 OpenRouter spend investigation. This silently ~doubles the whyMatters share of digest-notifications LLM spend.

Evidence

api/internal/brief-why-matters.ts:

  • L82: const shadowEnabled = env.BRIEF_WHY_MATTERS_SHADOW !== '0'default ON.
  • Sample pct defaults to 100 when unset/invalid (L373 warning path confirms).
  • L431: await Promise.allSettled([analystPath, geminiPath]) — two full gemini-2.5-flash completions per cache-miss story; one is used, the other stored only as an observability diff.

digest-notifications runs */30 * * * * on Railway with brief-llm.mjs pinned to OpenRouter (skipProviders:['ollama','groq']), so both calls bill OpenRouter.

Fix

If the shadow comparison window has served its purpose, set BRIEF_WHY_MATTERS_SHADOW=0 on the Railway service (immediate ~50% cut to whyMatters spend, zero correctness impact) — or drop BRIEF_WHY_MATTERS_SHADOW_SAMPLE_PCT to a small value (e.g. 5) if ongoing drift detection is still wanted. Consider flipping the code default to off so a fresh deploy isn't silently 2×.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: AI/intelAI analysis, intelligence findings, summarization

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions