You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
fix(forecast): resolvability coverage — 43% hard vs ≥80% KPI, and UCDP count specs can never resolve off the lagged feed (Bet-1/Bet-2 follow-up, #4930)
Now that the resolver is deployed (#5062) and the count-family false-NO is fixed (#5063), the binding constraint on the track record is coverage — what fraction of published forecasts can ever produce a scored YES/NO. Measured on prod 2026-07-09, it's far below the epic's ≥80%-hard KPI, and the largest "hard" family is effectively unresolvable with its current source.
Evidence (prod, 2026-07-09)
Live canonical forecast:predictions:v2 (14 forecasts): 6 hard / 8 judged = 43% hard (KPI target ≥80%).
Resolver dry-run over the history window (54 captured): 33 pending-judge (61%), 20 hard-pending, 1 VOID, 0 scored.
The count/conflict family — the flagship "hard" family — can't resolve off its feed.conflict:ucdp-events:v1 currently holds 2000 records dated 2025-11-05 … 2025-12-31 (2000-cap, ~7-month reporting lag). A Ukraine spec with an Aug-2026 deadline now correctly stays pending post-fix(forecast): avoid stale UCDP count resolutions #5063 (count_source_lags_deadline, maxTs < deadline) — but UCDP will not reach an Aug-2026 window for many months, and the 2000-cap means it may never retain it. Net: UCDP count specs go from false-NO → perpetual-pending. Safe, but they still never score.
So the effective trustworthy scored coverage at deploy is ~29%
(4 point-window hard families — price / yesPrice / hexCount / riskScore — minus judged, minus the perpetual-pending UCDP counts.) That's not enough for a credible public Brier (Bet 5 needs N≥100 resolved), and it's why the scorecard reads scored: 0 today.
Options (not mutually exclusive)
Repoint conflict counts to a timely, near-real-time feed (ACLED-style current events) whose maxTs actually advances past the deadline within the resolution window. UCDP is an academic lagged dataset — wrong tool for "events between now and 30d out." This is the single biggest hard-coverage win.
Raise the hard-ratio — audit which detectors emit judged when a hard metric_key was available (Bet 1 D3 deferred political/military/cyber to judged as a cost decision; some may be promotable to hard now). Each promotion removes an LLM-judge cost and adds a deterministic resolution.
Recompute the Bet-5 N≥100 timeline under realistic coverage — at ~29% trustworthy hard, minus VOID rate and settlement lags, time-to-100 is materially longer than the epic's "30–60 days."
KPI to restore
≥80% of published forecasts carry a resolvable spec (hard with a feed that covers the window, or judged with a working judge) — not just a well-formed one.
fix(forecast): resolvability coverage — 43% hard vs ≥80% KPI, and UCDP
countspecs can never resolve off the lagged feed (Bet-1/Bet-2 follow-up, #4930)Now that the resolver is deployed (#5062) and the count-family false-NO is fixed (#5063), the binding constraint on the track record is coverage — what fraction of published forecasts can ever produce a scored YES/NO. Measured on prod 2026-07-09, it's far below the epic's ≥80%-hard KPI, and the largest "hard" family is effectively unresolvable with its current source.
Evidence (prod, 2026-07-09)
forecast:predictions:v2(14 forecasts): 6 hard / 8 judged = 43% hard (KPI target ≥80%).pending-judge(61%), 20 hard-pending, 1 VOID, 0 scored.judgedspecs have no resolver — judged resolution was deferred in feat(forecast): add resolution ledger and scorecard #5024, so the 57% judged share sits inpending-judgeindefinitely. (Tracked as a deliverable in feat(forecast): Bet 2 — resolution engine + Brier scorecard: score every forecast against ground truth (#4930) #5007.)count/conflict family — the flagship "hard" family — can't resolve off its feed.conflict:ucdp-events:v1currently holds 2000 records dated 2025-11-05 … 2025-12-31 (2000-cap, ~7-month reporting lag). A Ukraine spec with an Aug-2026 deadline now correctly stays pending post-fix(forecast): avoid stale UCDP count resolutions #5063 (count_source_lags_deadline,maxTs < deadline) — but UCDP will not reach an Aug-2026 window for many months, and the 2000-cap means it may never retain it. Net: UCDP count specs go from false-NO → perpetual-pending. Safe, but they still never score.So the effective trustworthy scored coverage at deploy is ~29%
(4 point-window hard families — price / yesPrice / hexCount / riskScore — minus judged, minus the perpetual-pending UCDP counts.) That's not enough for a credible public Brier (Bet 5 needs N≥100 resolved), and it's why the scorecard reads
scored: 0today.Options (not mutually exclusive)
maxTsactually advances past the deadline within the resolution window. UCDP is an academic lagged dataset — wrong tool for "events between now and 30d out." This is the single biggest hard-coverage win.judgedwhen ahardmetric_key was available (Bet 1 D3 deferred political/military/cyber to judged as a cost decision; some may be promotable to hard now). Each promotion removes an LLM-judge cost and adds a deterministic resolution.KPI to restore
Related: #4930, #5007 (judged resolver), #5024, #5062 (deploy), #5063 (count-coverage guard), #4976 (Bet 1 emission).