Follow-up to #5066 (closed via #5072). Part of #4930 → Lane A #2.
#5072 moved the hard-resolvable ratio from 43% → 57% (UCDP→ACLED repoint, political-unrest + cyber-volume promoted to hard, publish-selection boost), but the epic's ≥80%-hard KPI is still not met. #5066 was closed on the fix landing, not on the KPI being reached.
Why this is the most time-sensitive Lane-A item
The resolver is live and accumulating (clock started 2026-07-09). Every forecast published at 57% coverage instead of ≥80% is permanently under-scored — you can't retro-fix the coverage of a forecast that already matured. Closing this gap now directly raises the quality of the data the whole track record is built on, during the wait.
Approach
- Measure the freshest-snapshot hard-ratio (not the full-window mix) as the KPI signal.
- The publish-selection boost is supply-limited: you can only publish hard specs if detectors emit them. So audit which detectors still emit
judged when a hard metric_key was available, and promote them to hard (Bet-1 D3 deferred political/military/cyber to judged as a cost decision — some are now promotable, each removing an LLM-judge cost).
- Re-run the coverage measurement over several
seed-forecasts cycles to confirm it holds ≥80%, not just spikes.
Done when
≥80% of freshly-published forecasts carry a resolvable hard spec (feed that covers the window), sustained across runs.
Related: #5066, #5072, #4976 (Bet 1 emission), #5007, #4930.
🔄 Reframed 2026-07-10 — the KPI is wrong, and "hard %" is hiding unresolvable specs
Investigating whether #5136/#5138 (conflict→judged) closed this surfaced two things: (a) the "≥80%-hard" target is the wrong metric, and (b) the live hard-ratio is inflated by hard specs that resolve against empty feeds.
Evidence (prod, 2026-07-10)
Live canonical (14 forecasts): 12 hard / 2 judged = 86% nominal hard — looks like the KPI is met. It is not. The count-resolution feeds the hard families point at:
conflict:acled-resolution:v1:all:0:0 → ❌ EMPTY (needs ACLED creds we don't have)
unrest:events-resolution:v1 → ❌ EMPTY — same root cause (see below)
cyber:threats-bootstrap:v2 → ✅ populated
supply_chain:chokepoints:v4, intelligence:gpsjam:v2, infra:outages:v1, prediction:markets-bootstrap:v1 → ✅ populated
So of the 12 "hard" specs, 2 conflict-hard + 1 political/unrest-hard resolve against empty feeds → real resolvable-hard ≈ 9/14 = 64%, well below 80%. Raw hard % overstates coverage.
The KPI reframe (do this)
Change the DoD from "≥80% hard" to:
≥80% of freshly-published forecasts carry a resolvable spec — a hard spec whose source feed is populated, OR a judged spec — sustained across runs.
A resolvable judged spec beats an unresolvable hard one. This also makes #5138 (conflict→judged) count toward the KPI instead of appearing to lower it.
Folded in: fix unrest (twin of conflict) — reclassify to judged
unrest:events-resolution:v1 is empty for the identical reason as conflict: seed-unrest-events.mjs:460 only writes it when its ACLED resolution fetch returns data (if (acledResolution?.events?.length)), and we have no ACLED credentials. So unrest hard-count specs resolve against nothing — the same bug #5138 fixed for conflict.
Remaining tasks for this issue
Status 2026-07-10: the code is done and deployed; re-measure shows 100% resolvable at emission. What's left is data-gated verification (~Jul 14), not code.
De-risk (startable now, not data-gated) → tracked in #5183: pre-empt the judged-resolver 48h-archive-window (deadline-centric window + decouple truncated from completeness) before the Jul-14 maturity, so the newly-judged conflict/unrest specs actually resolve instead of forced-VOID. This is the gate that makes the 100%-resolvable real.
Not a problem (audited clear)
cyber (cyber:threats-bootstrap:v2 ✅), supply_chain, gps, infra, prediction_market — populated resolution feeds.
Related: #5136/#5138 (conflict→judged, the template for the unrest fix), #5087 (judged resolver + its 48h-window caveat), #5134 (generator ACLED→GDELT), #4930.
Follow-up to #5066 (closed via #5072). Part of #4930 → Lane A #2.
#5072 moved the hard-resolvable ratio from 43% → 57% (UCDP→ACLED repoint, political-unrest + cyber-volume promoted to hard, publish-selection boost), but the epic's ≥80%-hard KPI is still not met. #5066 was closed on the fix landing, not on the KPI being reached.
Why this is the most time-sensitive Lane-A item
The resolver is live and accumulating (clock started 2026-07-09). Every forecast published at 57% coverage instead of ≥80% is permanently under-scored — you can't retro-fix the coverage of a forecast that already matured. Closing this gap now directly raises the quality of the data the whole track record is built on, during the wait.
Approach
judgedwhen a hardmetric_keywas available, and promote them to hard (Bet-1 D3 deferred political/military/cyber to judged as a cost decision — some are now promotable, each removing an LLM-judge cost).seed-forecastscycles to confirm it holds ≥80%, not just spikes.Done when
≥80% of freshly-published forecasts carry a resolvable hard spec (feed that covers the window), sustained across runs.
Related: #5066, #5072, #4976 (Bet 1 emission), #5007, #4930.
🔄 Reframed 2026-07-10 — the KPI is wrong, and "hard %" is hiding unresolvable specs
Investigating whether #5136/#5138 (conflict→judged) closed this surfaced two things: (a) the "≥80%-hard" target is the wrong metric, and (b) the live hard-ratio is inflated by hard specs that resolve against empty feeds.
Evidence (prod, 2026-07-10)
Live canonical (14 forecasts): 12 hard / 2 judged = 86% nominal hard — looks like the KPI is met. It is not. The count-resolution feeds the hard families point at:
conflict:acled-resolution:v1:all:0:0→ ❌ EMPTY (needs ACLED creds we don't have)unrest:events-resolution:v1→ ❌ EMPTY — same root cause (see below)cyber:threats-bootstrap:v2→ ✅ populatedsupply_chain:chokepoints:v4,intelligence:gpsjam:v2,infra:outages:v1,prediction:markets-bootstrap:v1→ ✅ populatedSo of the 12 "hard" specs, 2 conflict-hard + 1 political/unrest-hard resolve against empty feeds → real resolvable-hard ≈ 9/14 = 64%, well below 80%. Raw
hard %overstates coverage.The KPI reframe (do this)
Change the DoD from "≥80% hard" to:
A resolvable judged spec beats an unresolvable hard one. This also makes #5138 (conflict→judged) count toward the KPI instead of appearing to lower it.
Folded in: fix unrest (twin of conflict) — reclassify to judged
unrest:events-resolution:v1is empty for the identical reason as conflict:seed-unrest-events.mjs:460only writes it when its ACLED resolution fetch returns data (if (acledResolution?.events?.length)), and we have no ACLED credentials. So unrest hard-count specs resolve against nothing — the same bug #5138 fixed for conflict.unrestfamily to judged, exactly parallel to fix(forecast): reclassify conflict/ucdp_zone count specs to judged (#5136) #5138 — inderiveHardMetrics, gate theunrestcase behind an availability flag (mirrorCONFLICT_COUNT_FEED_AVAILABLE) →return null→buildJudgedSpec. Apply the family-aware escalation question to the political/unrest domain too.seed-unrest-eventsalready fetches GDELT, but article-volume ≠ event-count scale, so the horizon-scaled thresholds would mis-resolve (same reason it was rejected for conflict in fix(forecast): conflict/ucdp_zone count-resolution reads empty conflict:acled-resolution:v1 (no ACLED creds) — specs unresolvable (P2, #5097) #5136).#5138scoped-regression test ("unrest still emits hard") — it now asserts a hard-against-empty-feed spec; flip it to judged (or force the feed-available override, matching the conflict tests).Remaining tasks for this issue
Status 2026-07-10: the code is done and deployed; re-measure shows 100% resolvable at emission. What's left is data-gated verification (~Jul 14), not code.
seed-forecasts/seed-forecast-resolutions(auto-deploy was blocked by thecheckSuitesskip). Fresh canonical: conflict/unrest nowjudged, resolvable coverage = 14/14 = 100% (0 unresolvable-hard), hard% 86→79.pending-judge, the 100%-resolvable is hollow (unresolvable-hard → unresolvable-judged). Scorecard-watch M2 fires on the first judged resolution.De-risk (startable now, not data-gated) → tracked in #5183: pre-empt the judged-resolver 48h-archive-window (deadline-centric window + decouple
truncatedfrom completeness) before the Jul-14 maturity, so the newly-judged conflict/unrest specs actually resolve instead of forced-VOID. This is the gate that makes the 100%-resolvable real.Not a problem (audited clear)
cyber (
cyber:threats-bootstrap:v2✅), supply_chain, gps, infra, prediction_market — populated resolution feeds.Related: #5136/#5138 (conflict→judged, the template for the unrest fix), #5087 (judged resolver + its 48h-window caveat), #5134 (generator ACLED→GDELT), #4930.