title: LLM Stack DeepSeek Migration - Plan
type: feat
date: 2026-07-06
artifact_contract: ce-unified-plan/v1
artifact_readiness: implementation-ready
product_contract_source: ce-plan-bootstrap
execution: code
deepened: 2026-07-06

LLM Stack DeepSeek Migration - Plan

Goal Capsule

• Objective: Move WorldMonitor's core LLM surfaces off Groq's llama-3.1-8b-instant onto OpenRouter DeepSeek (deepseek/deepseek-v4-flash for the utility/prose tiers, deepseek/deepseek-v4-pro for the reasoning tier), demote Groq to an upgraded free-tier fallback (llama-3.3-70b-versatile), and fix the pending LLM-config bugs (dead LLM_TOOL_MODEL, undocumented env vars, region-403 exposure) while verifying the historical thinking-tag-stripper audit item (already deduped on main) and hardening DeepSeek output shape.
• Authority: This plan > repo conventions > implementer judgment. The brief editorial voice must not cut over without the U3 shadow evaluation passing (stop condition below).
• Execution profile: Multiple small PRs in the sequence given by the Planning Contract. Every merge is manual review by Elie — never enable auto-merge. Env changes land on BOTH planes (Vercel needs a redeploy to bind env; Railway seeder env is per-service).
• Implementation baseline: Start from a fresh worktree off origin/main. The research worktree (45a428d75) predates the telemetry/cache-churn batch (chore(digest): BRIEF_WHY_MATTERS_SHADOW is default-on at 100% sampling — every cache-miss runs TWO gemini calls, one discarded #4893–chore(llm): cache-identity churn batch 2 — summary pair-dedup v7, market-brief context quantization, forecast prompt-hash cache, whyMatters v8 cross-read (#4914) #4916); several cited files differ on main. Verify each file:line cite before editing.
• Stop conditions: (1) U3 shadow evaluation fails the rubric → stop before U4, report findings, keep serving the gemini path, and env-revert LLM_REASONING_MODEL for all reasoning surfaces (chat-analyst, deduct-situation) — not just the brief. (2) Any change would touch a probability-affecting forecast path → stop; only narrative prose may move models before the epic(forecast): verified probabilistic forecasting — resolution engine, public Brier track record, learned calibration, LLM ensemble (MiroFish++ roadmap) #4930 resolver baseline exists. (3) OpenRouter credits not topped up → stop before any cutover deploy.
• Tail ownership: Post-deploy verification (anon console sweep, Axiom queries, cache-version grep proof) is part of each unit, not a follow-up.

Product Contract

Summary

Migrate all server-side LLM traffic to a two-tier DeepSeek stack on OpenRouter with Groq as the upgraded fallback, using the existing shadow A/B mechanism to protect the brief's editorial voice, bumping every model-fed cache version, and closing the audit's config bugs (dead tool-tier model, telemetry-dark seeders, stale docs) — the historical stripper-duplication item is verified rather than fixed, since main already dedupes it.

Problem Frame

The 2026-07-06 audit found the stack inverted relative to intent: the highest-volume stages (classify-event, summarize-article) and the forecast narrative generator run on Groq's 2024-era 8B Llama because the default provider chain puts Groq first, while OpenRouter carries a mid-2025 Gemini 2.5 Flash default and a reasoning tier pinned to a preview SKU (google/gemini-3-flash-preview). The prod tool-tier env points at x-ai/grok-4.1-fast, which has zero OpenRouter endpoints — the exact deprecation failure mode preview SKUs invite. DeepSeek V4 (April 2026) now offers flash-tier quality at $0.09/$0.18 per M tokens across 16 OpenRouter providers and pro-tier reasoning at $0.435/$0.87 across 15, making "cheaper and better" true for every tier except where Gemini 3 Flash still benchmarks ahead — a trade the user has accepted in exchange for cost, open weights, and non-preview stability. Groq stays as the free-tier/outage fallback but on a current model.

Requirements

Model stack

• R1. All Vercel callLlm surfaces (classify-event, summarize-article, country-intel-brief, analyze-stock) run deepseek/deepseek-v4-flash via OpenRouter as primary; Groq is fallback.
• R2. The reasoning tier (callLlmReasoning / callLlmReasoningStream: brief-why-matters analyst path, chat-analyst, deduct-situation) runs deepseek/deepseek-v4-pro via LLM_REASONING_MODEL.
• R3. Railway seeder surfaces (forecast narrative, insights, regional narrative + weekly brief, brief-prose fallback path, and any other seeder-side chains found in the fresh-worktree sweep) run deepseek/deepseek-v4-flash primary.
• R4. Every Groq slot is upgraded from llama-3.1-8b-instant to llama-3.3-70b-versatile (already proven on this account by the regional-snapshot scripts).
• R5. The brief editorial voice (whyMatters/digest/description prose) switches only after a shadow A/B window over the existing BRIEF_WHY_MATTERS_SHADOW mechanism validates DeepSeek output; cutover ships with the cache bumps in R9.

Bug fixes

• R6. The dead LLM_TOOL_MODEL=x-ai/grok-4.1-fast prod env is replaced with a routable model.
• R7. A single shared thinking-tag stripper serves all surfaces — already true on main (summarize-article.ts imports stripThinkingTags at line 18, uses it at line 238; hasReasoningPreamble is a distinct preamble detector, not a duplicate); U2 re-proves this with rg and extends the shared stripper only if DeepSeek's live output shape demands it.
• R8. DeepSeek reasoning output is controlled: disabled on the utility tier, bounded on the reasoning tier, and never leaks into served content.
• R9. Every cache fed by a swapped model gets a version bump in the same PR as the swap (summary v7, whyMatters v5+v8, description v3, digest v8, ci-sebuf:v3, market:analyze-stock:v3, forecast LLM caches after verifying current keying).

Observability and hygiene

• R10. Seeder surfaces that are telemetry-dark today (seed-insights, brief-llm chain, regional-snapshot) emit llm_call events before their model cutover.
• R11. Env documentation and prose docs are trued up: .env.example (missing LLM_TOOL_*, BRIEF_WHY_MATTERS_*; stale reasoning example), docs/ai-intelligence.mdx tier prose, and a docs-wide model-name sweep (rg -i 'gemini-2.5-flash|llama-3.1-8b|llama-3.3-70b' docs/ .env.example) — known additional hits: docs/methodology/news-digest-and-briefing.mdx (brief provider chain, ~line 345) and docs/panels/forecast.mdx (seeder default routes, ~line 26). Tiered-provider prose sections need reading, not just grep.
• R12. Edge functions that call OpenRouter are pinned to non-restricted Vercel regions, and non-stream callLlm failures log the response body (region-403 diagnosability).

Constraints

• R13. No LLM output enters any probability-affecting forecast path before the epic(forecast): verified probabilistic forecasting — resolution engine, public Brier track record, learned calibration, LLM ensemble (MiroFish++ roadmap) #4930 resolver baseline exists — forecast probabilities stay with the rule-based detectors; only narrative prose changes model.
• R14. Self-hosted/local behavior is unchanged: Ollama stays first in the chain when configured; client-side provider toggles and user-supplied Ollama settings are untouched.

Scope Boundaries

Deferred to Follow-Up Work

• Model-validity probing in llm-health.ts (it only checks origin reachability; a dead model ID silently falls through every call). Worth its own issue after this migration.
• Wiring any caller onto callLlmTool (zero callers today; this plan only makes its config valid).
• LLM-forecaster ensemble work (epic(forecast): verified probabilistic forecasting — resolution engine, public Brier track record, learned calibration, LLM ensemble (MiroFish++ roadmap) #4930 Bet 4) — explicitly out until the resolver/scorecard baseline exists.
• An automated model-eval harness; this plan uses the shadow mechanism plus validator-rejection telemetry instead.

Outside this product's identity

• Embeddings migration (text-embedding-3-small and its brief:emb:v1 cache stay).
• BYOK or direct-provider integrations; everything stays behind OpenRouter/Groq.
• Prompt rewrites beyond what model compatibility requires.

Success Criteria

• Axiom wm_api_usage llm_call events show the target model per stage, with fallback_index ≥1 rates no worse than pre-migration.
• validate_reject / stripped_empty reason rates per stage at or below the Groq-8B baseline (quality proxy for the utility tier).
• Shadow rubric passes for the brief voice against a quantitative bar: zero fabricated dates, 100% ranked-hash validity, ≥95% word-count compliance across ≥50 shadow pairs, plus a subjective voice check by Elie.
• Every non-brief cutover (utility stages, seeder prose, reasoning surfaces) passes a sampled pre/post quality read — 10–20 outputs per surface spot-read before that cutover is declared done, because semantically worse-but-parse-valid output is the failure mode validate_reject cannot catch.
• Post-fix run-rate spend stays within the current ~$0.25–$1.50/day envelope (verify via OpenRouter /api/v1/key daily usage).

Planning Contract

Key Technical Decisions

• KTD1 — Tier mapping. deepseek/deepseek-v4-flash ($0.09/$0.18, 1M ctx, 16 OpenRouter endpoints) for utility + seeder prose + brief voice; deepseek/deepseek-v4-pro ($0.435/$0.87, 15 endpoints) for the reasoning tier; llama-3.3-70b-versatile for every Groq fallback slot. Deep multi-provider liquidity is the anti-grok-4.1-fast insurance: no single provider deprecation can zero the model.
• KTD2 — Chain reorder, not Groq removal. Default PROVIDER_CHAIN becomes ollama → openrouter → groq → generic. Ollama stays first so self-hosted deployments are untouched (it is skipped in cloud where OLLAMA_API_URL is unset); Groq stays present as the free/outage fallback.
• KTD3 — Reasoning control mirrors the existing extraBody idiom. OpenRouter credentials gain a default reasoning-off body (as Ollama already does with think: false); callLlmReasoning/callLlmReasoningStream opt back in. The exact OpenRouter parameter shape (reasoning: {enabled:false} vs effort levels, and how each DeepSeek provider normalizes it) is an implementation-time verification, not a plan-time decision.
• KTD4 — Env-flip where env exists, code-default everywhere. Code defaults change in all three config layers so env vars become optional overrides rather than load-bearing config; Vercel LLM_REASONING_MODEL/LLM_TOOL_MODEL and Railway FORECAST_LLM_* env give the reversible rollout lever. Env only binds where the code executes — Vercel flags need a Vercel redeploy; Railway flags don't reach Vercel functions.
• KTD5 — Brief voice via the existing shadow mechanism. During the window: BRIEF_WHY_MATTERS_PRIMARY=gemini (serve the stable path), LLM_REASONING_MODEL=deepseek/deepseek-v4-pro (candidate feeds the analyst path), BRIEF_WHY_MATTERS_SHADOW=1 with a sample pct. Cutover flips PRIMARY back and turns shadow off in the same deploy as the cache bumps — the chore(digest): BRIEF_WHY_MATTERS_SHADOW is default-on at 100% sampling — every cache-miss runs TWO gemini calls, one discarded #4893 lesson is that shadow left on doubles spend indefinitely.
• KTD6 — Cache invalidation is manual version bumps, swept. No chat cache is model-aware, so each swap PR bumps every version tag it feeds and proves with rg that zero old-version references remain. Do not "fix" this by hashing raw inputs into keys — that recreated the intel-brief cache-busting burn; quantize/normalize if keying ever changes.
• KTD7 — Forecast swap is narrative-only and therefore safe now. Probabilities come from rule-based detectors; the LLM writes prose. Swapping before the epic(forecast): verified probabilistic forecasting — resolution engine, public Brier track record, learned calibration, LLM ensemble (MiroFish++ roadmap) #4930 resolver clock starts avoids a mid-track-record voice change without confounding calibration data.
• KTD8 — Telemetry parity before seeder cutover. seed-insights, the brief-llm injected chain, and regional-snapshot emit no llm_call events; instrument them (mirroring the seed-forecasts emitter) before moving their models so the migration is observable.

High-Level Technical Design

Three independent config layers must each be edited — none shares a source of truth:

Rollout sequence with the voice gate:

flowchart TB
  OPS[Ops pre-flight: top up OpenRouter credits + auto-topup, file tracking issues] --> U1
  U1[U1 core chain migration + cache bumps + env fix] --> U3
  U2[U2 DeepSeek output-shape hardening] --> U3
  U3[U3 reasoning-tier flip + shadow window, serve gemini path] --> GATE{Shadow rubric passes?}
  GATE -->|yes| U4[U4 brief voice cutover + brief cache bumps + shadow off]
  GATE -->|no| HOLD[Stay on gemini path, report findings, revisit model choice]
  U5[U5 seeder telemetry parity] --> U6[U6 Railway seeder migration]
  U1 --> U6
  U1 --> U7[U7 region pinning + 403 body logging]

Assumptions

• deepseek/deepseek-v4-flash and deepseek/deepseek-v4-pro remain routable with multi-provider depth at implementation time (re-verify endpoints, pricing, and regional availability from the fresh worktree before the first deploy).
• Groq's free tier rate limits for llama-3.3-70b-versatile are acceptable for a fallback-only role (it already serves the weekly regional scripts).
• chat-analyst and deduct-situation flipping to V4 Pro at U3 without their own shadow window is acceptable; rollback is an env revert.

Risks & Mitigations

Sources & Research

• Live OpenRouter pricing/endpoints pulled 2026-07-06 (/api/v1/models, /api/v1/models/:slug/endpoints): v4-flash 16 endpoints, v4-pro 15. x-ai/grok-4.1-fast's endpoints probe returned literally "endpoints": [] — the public model page still renders, so the page's existence is not evidence of routability; re-probe the endpoints API, not the page.
• Axiom wm_api_usage 7-day llm_call telemetry: groq 8B carries classify-event (134 calls) + summarize-article (94); brief-why-matters-analyst on gemini-3-flash-preview (57); forecast impact_expansion_single on gemini-2.5-flash (34).
• OpenRouter account state: monthly usage $44.93 (mostly the fixed fix(summarization): anon dashboards dispatch premium summarize-article RPCs — 401 flood + 3-provider fan-out since #4687 (client caller never swept) #4913 anon flood), credits 1242.7/1260 used (~$17 remaining).
• Benchmarks: Gemini 3 Flash beats GLM-4.7 on all five shared benchmarks (llm-stats/Artificial Analysis) — GLM rejected; DeepSeek V4 (April 2026) is the cheaper-and-competitive family (aicybr 2026 guide).
• Repo research (from origin/main): three config layers with no shared source of truth; tests/shared-llm.test.mts:68-69 pins the default groq model and chain order; no chat cache is model-aware; BRIEF_WHY_MATTERS_SHADOW is opt-in =='1' after chore(digest): BRIEF_WHY_MATTERS_SHADOW is default-on at 100% sampling — every cache-miss runs TWO gemini calls, one discarded #4893; llm-health.ts probes origin only; seed-forecasts has its own llm_call emitter, other seeders none.
• Institutional learnings: Vercel Edge region-403 on OpenRouter (pin iad1/lhr1/fra1/sfo1, log 403 bodies); 4xx-flood pattern demands a post-deploy anon /desktop console sweep; cache bumps must sweep every referencing site with an rg proof; api.worldmonitor.app browser-TTL rewrite means verification fetches need cache: 'no-cache'.

Implementation Units

U1. Core chain migration (Layer A)

• Goal: All shared-chain traffic runs OpenRouter DeepSeek V4 Flash primary with Groq 70B fallback; the dead tool-tier env is fixed; caches fed by the shared chain are bumped.
• Requirements: R1, R4, R6, R8 (utility half), R9 (summary + country-intel), R11 (env docs), R14.
• Dependencies: none (first code unit).
• Files: server/_shared/llm.ts, src/utils/summary-cache-key.ts, server/worldmonitor/intelligence/v1/get-country-intel-brief.ts, server/worldmonitor/market/v1/analyze-stock.ts, tests/shared-llm.test.mts, tests/summarize-reasoning.test.mjs, .env.example; Vercel prod env (LLM_TOOL_MODEL).
• Approach: In getProviderCredentials: openrouter default model → deepseek/deepseek-v4-flash with a reasoning-off extraBody (mirror the Ollama think:false idiom); groq default → llama-3.3-70b-versatile. Reorder PROVIDER_CHAIN to ['ollama','openrouter','groq','generic']. Add the opt-in path so the reasoning profile can enable reasoning (used by U3). Bump summary CACHE_VERSION v7→v8, ci-sebuf:v3→v4, and market:analyze-stock:v3→v4 (analyze-stock caches LLM output keyed only by symbol — analyze-stock.ts ~line 1227 on main). Fix tests/shared-llm.test.mts default-model/order pins in the same commit (a red unit on main blocks all PRs). Set Vercel LLM_TOOL_MODEL=deepseek/deepseek-v4-flash (LLM_TOOL_PROVIDER is already openrouter in prod — verified via vercel env pull — so the profile resolves openrouter+v4-flash with no provider/model mismatch). Document LLM_TOOL_* and fix the stale reasoning example in .env.example.
• Test scenarios:
  • Default chain with no Ollama configured resolves openrouter first and returns model=deepseek/deepseek-v4-flash (update of the existing tests/shared-llm.test.mts:68 pin).
  • OpenRouter HTTP 500 → falls through to groq and returns llama-3.3-70b-versatile; fallback_index increments in the emitted llm_call events.
  • Explicit providerOrder/modelOverrides still win over defaults (existing override tests keep passing).
  • The openrouter request body carries the reasoning-off field for plain callLlm and not (or enabled) for the reasoning profile.
  • Summary cache key emits v8 (tests/summarize-reasoning.test.mjs pin updated); no summary:v7 references remain (rg 'summary:v7' clean).
  • validate() rejection on openrouter falls through to groq without retrying openrouter.
• Verification: npx tsx --test tests/shared-llm.test.mts tests/summarize-reasoning.test.mjs tests/llm-usage-telemetry.test.mts green; one live smoke call per tier (classify + summarize) returns clean prose from v4-flash; post-deploy: anon /desktop console shows no 401/403 growth, Axiom llm_call by stage shows deepseek/deepseek-v4-flash on classify-event/summarize-article with ok=true.

U2. DeepSeek output-shape hardening

• Goal: DeepSeek V4 output shape is verified end-to-end — across the providers OpenRouter actually routes to — so reasoning remnants can't leak into served content and reasoning tokens can't be silently billed.
• Requirements: R7, R8.
• Dependencies: none (can land with or before U1).
• Files: server/_shared/llm.ts (shared stripThinkingTags), a test under tests/.
• Approach: The stripper-dedupe half of the original scope is already satisfied on main (summarize-article.ts imports the shared stripThinkingTags at line 18, uses it at line 238 — verified 2026-07-06); re-prove with rg that only one stripper implementation exists. Then capture real v4-flash/v4-pro responses via OpenRouter across the DeepSeek provider set it actually routes to (not a single capture — providers normalize the reasoning flag differently) and confirm whether reasoning arrives as a separate reasoning field (ignored by the parsers) or inline tags; extend the shared stripper's patterns only if a live shape demands it. Add per-provider tokens_completion to the rollout watch so a provider that ignores the reasoning-off body surfaces as cost inflation rather than staying invisible.
• Test scenarios:
  • DeepSeek-shaped fixture with a reasoning field and clean content → content passes through untouched.
  • Inline <think>…</think> and unterminated <think> fixtures → stripped (existing behavior, exercised from summarize-article's path).
  • Response that is entirely reasoning (empty content after strip) → falls through to the next provider (stripped_empty reason).
• Verification: targeted test file green; rg proof of a single stripper implementation; multi-provider capture results and the per-provider token watch documented in the PR.

U3. Reasoning-tier flip + brief-voice shadow window

• Goal: The reasoning tier runs DeepSeek V4 Pro while the served brief voice stays on the stable gemini path; shadow pairs accumulate for evaluation.
• Requirements: R2, R5 (window half), R8 (reasoning half).
• Dependencies: U1 (reasoning opt-in plumbing), U2.
• Files: Vercel prod env only (LLM_REASONING_MODEL=deepseek/deepseek-v4-pro, BRIEF_WHY_MATTERS_PRIMARY=gemini, BRIEF_WHY_MATTERS_SHADOW=1, optionally BRIEF_WHY_MATTERS_SHADOW_SAMPLE_PCT) + redeploy.
• Approach: Env-only deploy. chat-analyst and deduct-situation flip to V4 Pro immediately (rollback = env revert; the sampled pre/post read below is the trigger for pulling that lever). Run the window until ≥50 shadow pairs accumulate (~6–7 days at the observed ~8 analyst calls/day — the pair-count floor, not elapsed days, authorizes U4). Sample brief:llm:whymatters:shadow:v6:* pairs from Upstash against the quantitative bar: zero fabricated dates, 100% rankedStoryHashes validity, ≥95% signal word-count compliance, plus a subjective voice read by Elie. In the same window, run a sampled pre/post read of chat-analyst and deduct-situation (10–20 outputs each, scripted before/after comparison Elie signs off on) — these surfaces have no shadow protection, so this read is their only regression signal. Watch V4 Pro reasoning-token spend via Axiom duration_ms/tokens_completion; if reasoning inflates cost or latency, try effort-bounded reasoning or v4-flash-with-reasoning as the analyst model before abandoning.
• Test scenarios: none — env-only unit; the shadow mechanism's behavior is already covered by tests/brief-why-matters-analyst.test.mjs (shadow stays opt-in, PRIMARY parsing).
• Verification: Axiom shows stage=brief-why-matters-analyst, model=deepseek/deepseek-v4-pro and stage=brief-why-matters-gemini still serving; served brief output unchanged (spot-check /api/brief with cache:'no-cache'); shadow keys accumulating in Redis.

U4. Brief voice cutover + brief cache bumps

• Goal: DeepSeek becomes the served brief voice; every brief cache version is bumped; shadow is off.
• Requirements: R5 (cutover half), R9 (brief caches), R11 (docs/ai-intelligence.mdx).
• Dependencies: U3 evaluation passed (hard gate).
• Files: scripts/lib/brief-llm.mjs (whymatters v5→v6, description v3→v4, digest v8→v9, and the cross-read whymatters-v8 constant), api/internal/brief-why-matters.ts (whymatters v8→v9), tests/brief-why-matters-analyst.test.mjs + tests/brief-llm*.test.mjs (version pins / prompt-parity snapshot if touched), docs/ai-intelligence.mdx; Vercel env (BRIEF_WHY_MATTERS_PRIMARY back to analyst/unset, BRIEF_WHY_MATTERS_SHADOW unset).
• Approach: One PR carrying all brief cache-version bumps plus the env flip in the same deploy window. Sweep every referencing site (server handler, api/bootstrap.js, api/health.js, api/mcp.ts, seeders) and prove with rg 'whymatters:v(5|8)|digest:v8|description:v3' that no old-version reference survives. Update the AI-stack prose docs to the new tier map, running the R11 docs-wide model-name sweep — including docs/methodology/news-digest-and-briefing.mdx (brief chain) here, and docs/panels/forecast.mdx (seeder routes) with U6.
• Test scenarios:
  • Version-pin tests updated and green; prompt-parity snapshot regenerated if the gemini-path prompt moved.
  • Cache-key unit tests (where they exist) emit the new versions.
  • rg proof: zero old-version cache references repo-wide.
• Verification: post-deploy /api/brief serves DeepSeek-voiced prose (fresh keys); Axiom shows analyst path as producedBy primary; BRIEF_WHY_MATTERS_SHADOW confirmed unset (the chore(digest): BRIEF_WHY_MATTERS_SHADOW is default-on at 100% sampling — every cache-miss runs TWO gemini calls, one discarded #4893 double-spend guard); OpenRouter daily usage stays in envelope.

U5. Seeder telemetry parity

• Goal: seed-insights, the brief-llm injected chain, and regional-snapshot emit llm_call events so their DeepSeek spend and failures are visible before their models move.
• Requirements: R10.
• Dependencies: none (should merge before U6).
• Files: scripts/seed-insights.mjs, scripts/lib/brief-llm.mjs (or its injection site in the digest cron), scripts/regional-snapshot/narrative.mjs, scripts/regional-snapshot/weekly-brief.mjs, new test mirroring tests/forecast-llm-telemetry.test.mjs.
• Approach: Mirror the seed-forecasts emitter (AXIOM_WM_API_USAGE_INGEST_URL + USAGE_TELEMETRY gate, per-attempt events with stage, model, fallback_index) rather than importing server code into seeders — seeders deliberately don't import server/_shared. Pick distinct stage tags per surface (e.g. seed-insights, brief-whymatters-cron, regional-narrative, regional-weekly-brief).
• Test scenarios:
  • Success and provider-fallback runs emit one event per attempt with correct stage/model/fallback_index (mock ingest).
  • USAGE_TELEMETRY unset → zero events, zero behavioral change.
  • Telemetry delivery failure never fails the seed run (graceful, matches the seed exit-code conventions).
• Verification: targeted tests green; after one Railway cycle, Axiom shows the new stages.

U6. Railway seeder migration (Layer B)

• Goal: All seeder LLM chains run v4-flash primary / groq-70B fallback; forecast narrative moves before the epic(forecast): verified probabilistic forecasting — resolution engine, public Brier track record, learned calibration, LLM ensemble (MiroFish++ roadmap) #4930 resolver clock starts.
• Requirements: R3, R4, R9 (forecast caches), R13.
• Dependencies: U1 (conventions set), U5 (observability).
• Files: scripts/seed-forecasts.mjs (provider table), scripts/seed-insights.mjs, scripts/regional-snapshot/narrative.mjs, scripts/regional-snapshot/weekly-brief.mjs, plus the additional seeder chains to verify on the fresh worktree (scripts/ais-relay.cjs, scripts/lib/llm-chain.cjs — cited from memory, re-locate before editing); affected tests/forecast-*/tests/regional-*/tests/seed-insights-* files; Railway env (FORECAST_LLM_PROVIDER_ORDER + per-stage overrides as rollout lever).
• Approach: Sweep first: rg -n "llama-3\.1-8b|llama-3\.3-70b|gemini-2\.5-flash|providerOrder" scripts/ server/ api/ on the fresh worktree is the authoritative inventory; edit every hit to the target map. Reorder each provider table to openrouter-first with deepseek/deepseek-v4-flash (reasoning off — confirm these transports need the same body field as U1) and groq llama-3.3-70b-versatile fallback. Verify whether forecast LLM caches (forecast:llm-combined/llm-scenarios/llm-market-implications) gained prompt-hash keying in chore(llm): cache-identity churn batch 2 — summary key over non-deduped pairs + volatile regime floats, forecast narrative hash over non-prompt values, whyMatters v5 fallback blind to v8 #4914 — if the key still hashes only prediction inputs, bump those prefixes and add them to the Verification Contract sweep in the same PR; if prompt-hash keying already landed, record the no-bump confirmation in the PR. The keys live in scripts/seed-forecasts.mjs on main — forecast:llm-combined ~line 14954, forecast:llm-scenarios ~line 15101, forecast:llm-market-implications ~line 16258; verify whether their current hashes are model-sensitive before deciding to bump. Pre-cutover hard gate (R13): trace every consumer of the LLM-written forecast fields (impact-expansion drafts, scenarios, market-implications) and confirm none feeds ranking, probability, blend, or resolution — the impact-expansion prompt is fed rankingScore and its result is parsed into candidate structures, so inertness is not self-evident. The clean consumer trace, not the single-forecast spot-diff, is what authorizes the swap.
• Execution note: This is Railway-deployed code with graceful exit-code conventions (exit 75 = graceful); prefer smoke verification via one manual seeder run per service over unit coverage alone.
• Test scenarios:
  • Provider-order parsing tests updated: default order openrouter-first; FORECAST_LLM_*_PROVIDER_ORDER env still overrides per stage.
  • Forecast telemetry tests assert the new default model strings where they pin them.
  • Retry/budget behavior unchanged (existing retry tests re-run against the new tables).
  • Diff-scope check: no changes under detector/probability code paths (reviewable via the PR file list), plus the LLM-field consumer trace documented in the PR.
• Verification: manual run of each affected seeder in a Railway shell or local env exits green (75/0 conventions respected); Axiom shows seeder stages on deepseek/deepseek-v4-flash; forecast envelopes carry fresh narrative with unchanged probability fields (spot-diff one forecast before/after).

U7. Region pinning + 403 diagnosability

• Goal: LLM-calling edge functions can't be geo-routed into OpenRouter region 403s, and when a provider rejects a call the body is visible in logs.
• Requirements: R12.
• Dependencies: U1 (same file for the logging half).
• Files: server/_shared/llm.ts (non-stream error path: log a bounded slice of the error body, matching what the stream path already does), region config for every LLM edge route — known set: api/chat-analyst.ts, api/internal/brief-why-matters.ts, api/market/v1/[rpc].ts (analyze-stock reaches callLlm; its config is currently only { runtime: 'edge' }), plus any others the sweep finds.
• Approach: Sweep every Edge route whose handler reaches callLlm/getProviderCredentials and pin each to ['iad1','lhr1','fra1','sfo1'] per the documented region-403 gotcha — api/news/v1/[rpc].ts and api/intelligence/v1/[rpc].ts are already pinned on main and serve as the pattern to mirror. Keep the log slice bounded (≤300 chars) to avoid log bloat.
• Test scenarios:
  • Non-stream provider failure logs http_<status> plus a body excerpt (mock 403 with a JSON body).
  • Region config present on every function the sweep found reaching getProviderCredentials('openrouter') — including api/market/v1/[rpc].ts (checklist in PR description; no runtime test).
• Verification: targeted test green; post-deploy vercel inspect (or dashboard) shows the pinned regions.

Verification Contract

Verification fetches against api.worldmonitor.app must use cache: 'no-cache' (Cloudflare rewrites browser TTL to 1800s); curl does not see the browser-cache layer.

Definition of Done

• Every surface in the HTD tier table serves its target model, proven by the Axiom telemetry audit (including the newly instrumented seeder stages).
• LLM_TOOL_MODEL points at a routable model; no config references x-ai/grok-4.1-fast or llama-3.1-8b-instant anywhere (rg clean).
• All cache versions bumped with sweep proofs; no stale gemini-voiced output served past its TTL window.
• BRIEF_WHY_MATTERS_SHADOW is off after cutover (env checked, not assumed).
• Docs true: .env.example and docs/ai-intelligence.mdx match the deployed stack, and every hit from the R11 docs-wide model-name sweep is reviewed and intentional — the target fallback llama-3.3-70b legitimately remains — including docs/methodology/news-digest-and-briefing.mdx and docs/panels/forecast.mdx.
• Tests green on main after each merge (a red unit blocks all PRs); no abandoned experiment code left in any diff.
• Umbrella issue feat(llm): migrate LLM stack to OpenRouter DeepSeek tiers — core off Groq, upgraded fallback, config bug fixes #4944 and the standalone bug issues are closed or updated with outcomes.

Documentation & Operational Notes

• Ops pre-flight (before any cutover deploy): top up OpenRouter credits and enable auto-topup (~$17 of 1260 remaining on 2026-07-06); a mid-migration credit exhaustion takes down every OpenRouter surface with Groq as the only survivor.
• Issue tracking (standing rule): feat(llm): migrate LLM stack to OpenRouter DeepSeek tiers — core off Groq, upgraded fallback, config bug fixes #4944 is the migration umbrella (this plan is its body); file standalone issues for the dead LLM_TOOL_MODEL env, the telemetry-dark seeders, and the deferred llm-health model-validity gap before starting U1.
• Trust boundary / data handling: prompts sent to OpenRouter/Groq carry public news headlines and descriptions, market data, and (chat-analyst) user questions — no credentials are intentionally included, but chat-analyst query/history text is free-form and must be treated as potentially sensitive user-provided content (sanitizeForPrompt guards injection, not privacy). DeepSeek routing on OpenRouter may select non-US hosting providers; if provider policy or routing posture changes materially, the response is the same env-revert lever as any model rollback. OPENROUTER_API_KEY/GROQ_API_KEY rotation remains owner-managed (Elie) across Vercel and Railway env.
• Merge discipline: manual review on every PR; no auto-merge. Env changes are deploys: Vercel env binds at deploy time, Railway env at service restart.
• Rollback levers: U1/U6 code defaults roll back by revert; U3/U4 roll back by env flip (LLM_REASONING_MODEL, BRIEF_WHY_MATTERS_PRIMARY); cache bumps are forward-only (old versions simply expire).