Part of #4930 → Lane A #3 (Bet 4). Shadow now, promote later.
Today the LLM never sets a probability — the strongest known accuracy lever (retrieval-grounded LLM ensembles approach superforecaster parity on ForecastBench) is unused. Build the ensemble in shadow mode now so comparative Brier data accrues from day one; promotion is Lane C.
Scope (shadow)
- For the top K≈6 forecasts per run: 3 diverse reasoning passes over the already-built evidence table + news context + theater-sim paths — outside-view/base-rate, inside-view/signal-weighing, adversarial refuter. Trimmed-mean aggregate.
- Shadow-publish the ensemble probability alongside the detector probability (do NOT replace it). Bet 2's resolver already scores whatever's published — so it will score both and produce a head-to-head Brier.
- Stronger model for this stage only (~18 calls/run), prompt-hash cached like every other stage, respects the per-run LLM budget (cache-identity discipline).
Promotion (Lane C, not this issue)
Three-way blend (detector prior + LLM ensemble + market anchor, weights learned in Bet 3); promote only when the blend beats the detector on rolling Brier.
Done when
Every top-K forecast carries a shadow ensemble probability that Bet 2 scores separately, with a scorecard slice comparing detector-vs-ensemble Brier.
Related: #4930, #5007 (scores the shadow), Bet 3 (learned blend weights — Lane C).
Part of #4930 → Lane A #3 (Bet 4). Shadow now, promote later.
Today the LLM never sets a probability — the strongest known accuracy lever (retrieval-grounded LLM ensembles approach superforecaster parity on ForecastBench) is unused. Build the ensemble in shadow mode now so comparative Brier data accrues from day one; promotion is Lane C.
Scope (shadow)
Promotion (Lane C, not this issue)
Three-way blend (detector prior + LLM ensemble + market anchor, weights learned in Bet 3); promote only when the blend beats the detector on rolling Brier.
Done when
Every top-K forecast carries a shadow ensemble probability that Bet 2 scores separately, with a scorecard slice comparing detector-vs-ensemble Brier.
Related: #4930, #5007 (scores the shadow), Bet 3 (learned blend weights — Lane C).