feat(forecast): Bet 2 — resolution engine + Brier scorecard: score every forecast against ground truth (#4930)

🥇 PRIORITY (2026-07-09): the remaining deliverable here — judged resolution (dual-model LLM + citations + VOID-on-disagreement) — is the #1 do-now item for the whole epic.
The resolver core shipped (#5024) and is deployed (#5062); count-safety (#5063), coverage (#5072), and retention (#5073) all landed. But 62% of the live ledger is pending-judge with no resolver (scored: 0, pending-judge: 34/55 on 07-09), so most of every day's output can never score. Waiting for data does not fix this — only the judge does, and every day it's unbuilt is a day of forecasts accumulating at ~50% coverage instead of ~90%. See #4930 → Lane A #1. Everything below that isn't judged-resolution is either already shipped or a Lane-A/C follow-up.

Bet 2 of the verified-forecasting epic #4930. This is the verification loop — the piece that makes the whole initiative real. Bet 1 (#4976 → PR #5004, shipped 2026-07-07) made every published forecast scoreable by attaching a machine-checkable ResolutionSpec. Bet 2 actually resolves those specs against reality and computes a scored, auditable track record. Until Bet 2 ships, forecasts carry specs no one reads.

⏱️ Time-bounded — the clock is already running

Bet 1 started accumulating resolution specs into the 45-day rolling history on 2026-07-07. Bet 2's resolver can retroactively score everything still inside that window when it deploys, so the effective public-record start is max(Bet-1 deploy, Bet-2 deploy − 45d). If Bet 2 slips more than ~45 days past Bet 1, the head-start is lost and the record's real start date caps at ~45 days before Bet 2 ships. This bet should be prioritized accordingly (or Bet 1's specs durably persisted first — see Inherited deferrals).

What Bet 1 shipped (the contract Bet 2 consumes)

Every published forecast in forecast:predictions:v2 and the 45-day history (forecast:predictions:history:v1) carries a resolution object (proto ResolutionSpec, camelCase JSON):

• kind: "hard" | "judged"
• deadline: epoch ms — always non-null (both kinds)
• hard: metricKey (feed+path), operator (>=/<=/crosses), threshold, baselineValue (for crosses), window, sourceFeed (a real Redis feed key from RESOLUTION_FEED_KEYS)
• judged: question (resolution question + retrieval scope)

History persists resolution + projections (h24/d7/d30) + the stable forecast id (#4973) + probability/confidence/timeHorizon + run generatedAt.

What Bet 2 builds

1. Resolver seeder — scripts/seed-forecast-resolutions.mjs (daily cron, one runSeed() per process)

• Sweep the 45-day history for forecasts whose deadline has passed and aren't yet resolved.
• hard specs → read sourceFeed at/after the deadline, extract the metric per metricKey, apply operator/threshold/window (using baselineValue for crosses) → YES / NO.
• judged specs → LLM judge over the WM news archive scoped by the question; dual-model agreement required, mandatory citations; disagreement or no evidence → VOID (never a guess).
• Write per-forecast outcomes to forecast:resolutions:v1 + an append-only R2 audit ledger capturing the evidence snapshot at resolution time (the receipt).

2. Scorecard — forecast:scorecard:v1

• Rolling Brier + log score, overall and per-domain / per-detector.
• Per-domain calibration curves (predicted-vs-realized buckets).
• vs-market skill: on forecasts with a Polymarket/Kalshi anchor (calibration), did the internal probability beat the market close? (the headline selling-point metric).
• Sample counts + VOID rate per domain.

Resolution mechanics

• hard = free, deterministic feed comparison — no LLM cost. metricKey is a feed+path expression the resolver parses; sourceFeed names the exact Redis key to read.
• judged = dual-model LLM judge with citations, else VOID. Recurring per-forecast cost (this is why Bet 1's D3 deferred political/military/cyber to judged — promoting them to hard later is a cost reducer).

Audit-inherited constraints (from #4933 — non-negotiable)

• Score stored forecasts; never re-run detectors over historical inputs. Military inputs are wall-clock-gated (non-deterministic on replay); the stored history is the source of truth for what was predicted.
• Never trust stored trend / priorProbability. PRIOR_KEY ttl is 7200s, so any run gap >2h writes false 'stable'/self-referential priors into history. Recompute probability deltas from consecutive history snapshots.
• Key the outcome ledger on the fix(forecast): stable semantic-slot ids for military and state-derived forecasts (#4933) #4973 stable ids (fc-military-<theater hash>, fc-<domain>-<state-unit hash>, template-title hashes elsewhere) — the same identity Bet 1 stabilized so one evolving situation resolves as one outcome, not fragments.
• Deadline is authoritative (Bet 1 guarantees non-null); the resolver resolves at/after deadline, never earlier.

Anti-gaming invariants (the credibility guarantees)

• Every matured published forecast is resolved — no cherry-picking which to score.
• VOID rate is reported, not hidden — a high VOID rate is itself a signal the specs aren't resolvable.
• The scorer code path is locked + tested like the existing forecast benchmark suite (a scorer that silently miscounts is worse than none).
• The append-only R2 ledger is the auditable receipt: what we said, when, at what probability → what happened → resolving source.

Inherited deferrals from Bet 1 — decide at plan time (scope forks)

• Resolvability ranking preference (Bet 1's R7/D2, deferred here). If included: it may only be applied as a dedicated key at the final publish selection sort, downstream of filterPublishedForecasts — never on computeAnalysisPriority or scoreForecastReadiness.overall, both of which gate drops (priority < 0.08 weak-fallback, priorityGap ≥ 0.02 dedup, readiness < 0.34/0.4 floors). Must ship with a kept-set-identical test. Reasonable to defer to a Bet-2 follow-up rather than the resolver core.
• Durable spec persistence beyond the 45-day history — pull in only if the Bet-1→Bet-2 gap is going to exceed ~45 days (otherwise the rolling window suffices).

Deliverables

• scripts/seed-forecast-resolutions.mjs — daily resolver (hard feed-comparison + judged dual-model LLM judge + VOID), writes forecast:resolutions:v1 + append-only R2 ledger
• metricKey parser + per-feed metric extractors for the hard families (conflict/UCDP, market, prediction-market, supply_chain, GPS, infra)
• forecast:scorecard:v1 aggregator — rolling Brier/log-loss, per-domain calibration curves, vs-market skill, VOID rate, sample counts
• Delta recomputation from history snapshots (not stored trend/priorProbability)
• Railway cron wiring + seeder exit-convention compliance (graceful vs FATAL) + health/freshness surface
• Tests: hard-resolution per family (feed → outcome), judged dual-model agreement + VOID path, scorecard math (Brier/calibration), anti-gaming (all-matured-scored, VOID-rate-reported), locked-scorer regression
• Read-only surfacing hooks for the future track record (Bet 5) — get-forecast-scorecard / resolutions exposure (may be minimal here)

KPI

• 100% of matured published forecasts resolved within ~24h of deadline; VOID rate < 15%
• Rolling 90-day Brier computed and published to forecast:scorecard:v1; target < 0.20, aspirational < 0.15
• vs-market skill measured on anchored forecasts (beating the Polymarket close is the headline win if achieved)
• Zero silent scoring gaps — every published forecast either resolved or explicitly VOID with a logged reason

Blocks: Bet 3 (learned calibration — needs ~30d of resolutions), Bet 5 (public track record — needs ≥100 resolved). Depends on: ✅ Bet 1 (#4976 / PR #5004). Part of #4930. Constraints source: #4933.