You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#4930 built the verification loop (resolver + Brier scorecard) and it is now live and accumulating. But a full-code + production sweep on 2026-07-09 found the generator feeding it has real garbage-in: two entire signal layers are silently dead in prod, a market-calibration bug pushes probabilities the wrong direction, and a systemic silent-no-fire pattern hid all of it. Verification is not moot — a bad Brier will expose these — but the track record is being built on broken inputs right now, so fixing the generator is the top priority and is time-sensitive (every hour of forecasts accumulates on the broken inputs).
This epic tracks the generator-side fixes; #4930 tracks scoring them.
Sweep method
4-agent adversarial audit of scripts/seed-forecasts.mjs (signal wiring → writers · detector/blend correctness · missing-signals gap · MiroFish sim completeness+accuracy) + live prod grounding (feed freshness, key existence, stored-shape checks). Every finding below was cross-checked against code and prod.
Findings → sub-issues
🔴 P1 — silent dead signals & directional bugs (drop-everything):
MiroFish does NOT rewrite published probabilities — decorations are display + deep-path acceptance + next-run seeding only (the sole pred.probability= write is the market blend). Sim bugs can't directly corrupt Brier → lower urgency than the P1s.
Key-string wiring is clean — every input key has an exact-match writer; the dead signals are a shape mismatch (FRED) and an operational gap (ACLED), not typos.
iran-events is correctly flag-gated off (iranEventsEnabled() default false) — the 33-day-stale feed is not consumed.
KPI
Every declared signal-type actually fires (no silently-empty input family reaching a forecast).
Zero silent feed no-fire — each forecast input has presence logging + health coverage.
epic(forecast): generator signal integrity & correctness — dead signals, market-calibration polarity, silent no-fire
Why this is its own epic (upstream of #4930)
#4930 built the verification loop (resolver + Brier scorecard) and it is now live and accumulating. But a full-code + production sweep on 2026-07-09 found the generator feeding it has real garbage-in: two entire signal layers are silently dead in prod, a market-calibration bug pushes probabilities the wrong direction, and a systemic silent-no-fire pattern hid all of it. Verification is not moot — a bad Brier will expose these — but the track record is being built on broken inputs right now, so fixing the generator is the top priority and is time-sensitive (every hour of forecasts accumulates on the broken inputs).
This epic tracks the generator-side fixes; #4930 tracks scoring them.
Sweep method
4-agent adversarial audit of
scripts/seed-forecasts.mjs(signal wiring → writers · detector/blend correctness · missing-signals gap · MiroFish sim completeness+accuracy) + live prod grounding (feed freshness, key existence, stored-shape checks). Every finding below was cross-checked against code and prod.Findings → sub-issues
🔴 P1 — silent dead signals & directional bugs (drop-everything):
🟠 P2 — consistency / calibration / latent + observability:
temporal:anomalieskeep-warmDe-risking (verified during the sweep)
pred.probability=write is the market blend). Sim bugs can't directly corrupt Brier → lower urgency than the P1s.iranEventsEnabled()default false) — the 33-day-stale feed is not consumed.KPI
Related