Skip to content

fix(forecast): non-finite probability guards — NaN can no longer enter or anchor the probability spine (#4933)#4969

Merged
koala73 merged 3 commits into
mainfrom
fix/forecast-nonfinite-guards
Jul 6, 2026
Merged

fix(forecast): non-finite probability guards — NaN can no longer enter or anchor the probability spine (#4933)#4969
koala73 merged 3 commits into
mainfrom
fix/forecast-nonfinite-guards

Conversation

@koala73

@koala73 koala73 commented Jul 6, 2026

Copy link
Copy Markdown
Owner

What

PR-B of the #4933 hardening batch (MEDIUM audit finding): three sites let a malformed Redis market row inject NaN into the probability spine or anchor the market blend at a fabricated 50%.

Site Before After
makePrediction NaN/undefined probability survived the clamp chain as NaN → serialized null → forecast silently vanished at the publish filter coerced to 0 (finite, deterministic)
detectFromPredictionMarkets ('oops'||50)/100 = NaN passed the band gate (NaN comparisons are false) → null-probability forecast emitted non-finite yesPrice row skipped
calibrateWithMarkets matching market without a price anchored the 0.4/0.6 blend at 50% price-less markets excluded from candidates; second-best priced market can still match

Tests

Bug-fix protocol: 5 tests written first, 4 confirmed RED (NaN coercion ×2, NaN-emit skip, 50%-anchor skip), 1 positive control (finite in-band price still emits at 0.75).

  • forecast-detectors 219/219 · trace-export 310/310 · history/integrity/resilience/narrative/ticker/telemetry suites green · evaluate-forecast-benchmark 6/6

Part of #4933 (fix 2 of 3 P1/MEDIUM prerequisites for #4930). Branched off main AFTER #4939 merged — no stack.

https://claude.ai/code/session_01XhvRdRjkLqtkRBZi9ZYAu2

…r or anchor the probability spine (#4933)

Three guard sites from the adversarial audit (MEDIUM):

- makePrediction: NaN/undefined probability or confidence coerces to 0
  instead of surviving the clamp chain as NaN and serializing to null
  (previously the forecast silently vanished at the publish filter).
- detectFromPredictionMarkets: a market row with a non-numeric yesPrice
  is skipped; previously ('oops'||50)/100 = NaN passed the band gate
  (NaN comparisons are false) and emitted a null-probability forecast.
- calibrateWithMarkets: a matching market without a finite yesPrice is
  excluded from candidates instead of anchoring the blend at a
  fabricated 50%; a second-best priced market can still match.

5 new tests (4 RED pre-fix); detector suite 219/219, adjacent suites
green, benchmark 6/6.

Claude-Session: https://claude.ai/code/session_01XhvRdRjkLqtkRBZi9ZYAu2
@vercel

vercel Bot commented Jul 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
worldmonitor Ignored Ignored Preview Jul 6, 2026 11:33am

Request Review

@greptile-apps

greptile-apps Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR plugs three sites in the forecast pipeline where a malformed Redis market row could inject NaN into the probability spine or anchor a blend at a fabricated 50%, by adding Number.isFinite() guards at each entry point.

  • makePrediction: Number.isFinite() ternary coerces any non-finite probability or confidence to 0 before the Math.max/Math.min clamp, preventing NaN from propagating to serialization.
  • detectFromPredictionMarkets: non-finite yesPrice rows are now skipped via an early continue, removing the silent NaN-emit path introduced by the old 'oops' || 50 fallback.
  • calibrateWithMarkets: priceless market candidates are filtered out before the 0.4/0.6 blend step, so only markets with a real yesPrice can influence the calibrated probability.

Confidence Score: 4/5

The three guards are each correctly placed, the old || 50 fallback is fully removed, and five new tests cover both the failure paths and the positive control. Safe to merge.

All three bug sites are fixed correctly and tests confirm the expected behavior. One small edge case — Infinity input to makePrediction now becomes 0 instead of clamping to 1 — is unlikely to occur in practice but is an unannounced behavior change worth a second look.

The behavioral change for Infinity inputs in scripts/seed-forecasts.mjs lines 923-924 is the only thing worth double-checking before merge.

Important Files Changed

Filename Overview
scripts/seed-forecasts.mjs Three targeted guards added: NaN/undefined coerced to 0 in makePrediction, non-finite yesPrice rows skipped in detectFromPredictionMarkets, priceless markets excluded before calibrateWithMarkets blend. All fixes are correct; minor: Infinity now maps to 0 instead of clamping to 1 (no practical impact).
tests/forecast-detectors.test.mjs Five new tests added in a dedicated 'non-finite probability guards' describe block: two for makePrediction (NaN/undefined coercion), two for detectFromPredictionMarkets (skip non-numeric, positive control), one for calibrateWithMarkets (priceless anchor skip). Tests are well-structured and cover both the failure path and the happy path.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Redis market row] --> B{Number.isFinite yesPrice?}
    B -- No --> C[skip row / filter out]
    B -- Yes --> D[yesPrice / 100]
    D --> E[detectFromPredictionMarkets band gate 0.6-0.9]
    E -- in band --> F[makePrediction with yesPrice]
    E -- out of band --> C
    D --> G[calibrateWithMarkets candidate filter]
    G -- passes semantic match --> H[0.4 x marketProb + 0.6 x predProb]
    G -- filtered --> I[pred.calibration = null, probability unchanged]
    F --> J{Number.isFinite probability / confidence?}
    J -- Yes --> K[Math.max/min clamp stored]
    J -- No / NaN / undefined --> L[coerce to 0 clamp stored]
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[Redis market row] --> B{Number.isFinite yesPrice?}
    B -- No --> C[skip row / filter out]
    B -- Yes --> D[yesPrice / 100]
    D --> E[detectFromPredictionMarkets band gate 0.6-0.9]
    E -- in band --> F[makePrediction with yesPrice]
    E -- out of band --> C
    D --> G[calibrateWithMarkets candidate filter]
    G -- passes semantic match --> H[0.4 x marketProb + 0.6 x predProb]
    G -- filtered --> I[pred.calibration = null, probability unchanged]
    F --> J{Number.isFinite probability / confidence?}
    J -- Yes --> K[Math.max/min clamp stored]
    J -- No / NaN / undefined --> L[coerce to 0 clamp stored]
Loading

Reviews (1): Last reviewed commit: "fix(forecast): non-finite probability gu..." | Re-trigger Greptile

Comment on lines +923 to +924
probability: Math.round(Math.max(0, Math.min(1, Number.isFinite(probability) ? probability : 0)) * 1000) / 1000,
confidence: Math.round(Math.max(0, Math.min(1, Number.isFinite(confidence) ? confidence : 0)) * 1000) / 1000,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Infinity now coerces to 0 instead of clamping to 1

Number.isFinite(Infinity) is false, so a caller passing Infinity (or any non-finite value) is mapped to 0 before the clamp. The original code let Math.max(0, Math.min(1, Infinity)) correctly produce 1. The change is intentional for NaN/undefined, but a +Infinity probability would now silently become 0 instead of 1. In practice, probabilities fed from Redis should never reach Infinity, but if the guard is meant to express "non-numeric inputs get a safe default," a fallback of 0 is defensible — it's just a different semantic than the previous behavior and is worth confirming intentionally.

…0 for unreadable Kalshi prices

The downstream finite-only guard was defeated upstream: the Kalshi
normalizer converted a non-finite last_price_dollars into a finite 50,
which then matched and calibrated forecasts against invented data.
New parseKalshiYesPrice mirrors the Polymarket parseYesPrice contract
(null for missing/non-numeric/out-of-range → market skipped); a genuine
0 price is kept and filtered downstream, not fabricated.

6 new tests (RED via missing export); prediction-scoring 61/61,
forecast-detectors 219/219.

Claude-Session: https://claude.ai/code/session_01XhvRdRjkLqtkRBZi9ZYAu2
@koala73

koala73 commented Jul 6, 2026

Copy link
Copy Markdown
Owner Author

Review finding addressed in cb4e426 — confirmed at source: seed-prediction-markets.mjs Kalshi path converted a non-finite last_price_dollars into a finite fabricated 50 (Number.isFinite(rawPrice) ? … : 50), which sailed through the downstream finite-only guard and could calibrate a matching forecast against invented data.

Fix is upstream where it belongs: new parseKalshiYesPrice in _prediction-scoring.mjs mirrors the existing Polymarket parseYesPrice contract — null for missing/non-numeric/out-of-range (market skipped entirely), genuine 0 kept and filtered downstream by the [10,90] band rather than fabricated. The Polymarket path already worked this way; Kalshi was the only laundering site (repo grep for other : 50 fabrications came back clean).

6 new tests RED-first (missing-export); prediction-scoring 61/61, forecast-detectors 219/219.

…hole-string price validation

P2: Kalshi selected the top-volume market BEFORE checking price, so an
unpriced top market sank the whole event even when a priced sibling
existed. New selectPricedKalshiMarket picks the highest-volume market
among PRICED ones; the 5000 volume gate applies to the selected market.

P3: parseKalshiYesPrice used parseFloat, which accepts malformed
prefixes ('0.62oops' → 0.62, '0,62' → 0). Now whole-string Number()
validation with an empty/whitespace guard.

8 new tests (RED via missing export); prediction-scoring 66/66,
forecast-detectors 219/219.

Claude-Session: https://claude.ai/code/session_01XhvRdRjkLqtkRBZi9ZYAu2
@koala73

koala73 commented Jul 6, 2026

Copy link
Copy Markdown
Owner Author

Round-2 findings addressed in 466dba1:

P2 priced-sibling fallback — fixed. New selectPricedKalshiMarket picks the highest-volume market among priced ones instead of price-checking only the top-volume pick — your exact repro is now a test (unpriced HIGH @10000 + priced VALID @6000/0.64 → selects VALID at 64). The ≤5000 volume gate applies to the selected market's own volume. Returns null only when NO sibling has a readable price.

P3 whole-string validation — fixed. parseKalshiYesPrice now uses Number() over the trimmed whole string with an empty-string guard: '0.62oops' → null, '0,62' → null; plain numbers and whitespace-padded numerics still parse. Noted: the sibling Polymarket parseYesPrice has the same parseFloat laxity pre-existing — left untouched per the same pre-existing demotion logic as the out-of-range item, can batch with PR-D.

prediction-scoring 66/66 (8 new tests RED-first), forecast-detectors 219/219.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant