fix(forecast): present() observes 0 (not NaN) on absence — VOID-rate regression by koala73 · Pull Request #5327 · koala73/worldmonitor

koala73 · 2026-07-15T00:21:32Z

Why

Triaging the 68% resolution VOID rate surfaced a regression I introduced in the sampling refactor (#5243 P1).

extractMetricObservation returned NaN for a missing matched record. But present() is a boolean existence check — an absent subject means the event did not occur → 0, not NaN. So samplePendingEntries stored an error sample instead of a valid 0, and a legitimately-NO within-horizon present spec (e.g. infra:outages:v1|present(country==X) for a quiet country) stranded with an empty timeline and VOIDed (no_establishable_metric) instead of resolving NO. The old extractMetricValue did record ? 1 : 0; the refactored helper must match it.

What

extractMetricObservation: present() now observes record ? 1 : 0 (0 on absence), asOf from the record when present.

Evidence

Tests: present-absent → 0 (parity with extractMetricValue); a within-horizon present spec resolves NO on a sampled absence.
908/908 forecast+bet suites green; api tsc exit 0.

VOID-rate triage context (separate from this fix)

Historical VOIDs are all legacy-detector hard specs, zero bet_engine, two causes, both aging out:

unsupported_window: "sustained-48h" (gpsjam/chokepoint) — an old generator window the resolver never implemented; the current generator no longer emits it (0 in the pending pool).
present within-horizon 24h on a daily resolver — sparse sampling + this NaN bug (fixed here). The cadence + vague-forecast issues are what the bet-template re-engine supersedes.

Refs #5233, #4930

… VOID-rate regression While triaging the 68% resolution VOID rate, found a regression I introduced in the sampling refactor (#5243 P1): extractMetricObservation returned NaN for a missing matched record, but present() is a boolean existence check — an absent subject means the event did NOT occur → 0, not NaN. samplePendingEntries then stored an error sample instead of a valid 0, so a legitimately-NO within-horizon present spec (e.g. infra:outages present(country==X) for a quiet country) stranded with an empty timeline and VOIDed as no_establishable_metric instead of resolving NO. Now mirrors extractMetricValue's `record ? 1 : 0`. Tests: extractMetricObservation present-absent → 0 (matches extractMetricValue); present-within-horizon resolves NO on a sampled absence. 908/908 forecast+bet green; api tsc exit 0. Refs #5233, #4930 Claude-Session: https://claude.ai/code/session_01StNurp4TGC3JLHbTtKJhbp

vercel · 2026-07-15T00:21:38Z

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
worldmonitor	Ignored	Preview	Jul 15, 2026 12:40am

greptile-apps · 2026-07-15T00:23:46Z

Greptile Summary

This PR fixes a regression introduced in the #5243 sampling refactor where extractMetricObservation returned { value: NaN } for an absent present() subject. Because downstream resolution treats NaN as no_establishable_metric (a VOID), specs for legitimately quiet subjects were incorrectly voided instead of resolving NO. The fix mirrors the pre-existing logic in extractMetricValue (record ? 1 : 0), and the accompanying tests directly encode parity between the two helpers.

extractMetricObservation now handles present() before the generic if (!record) return NaN guard, returning { value: 0, asOf: null } on absence and { value: 1, asOf: <parsed> } on presence.
Three new test cases are added: absent → 0 (with parity assertion against extractMetricValue), present → 1, and an end-to-end test that a within-horizon present spec resolves NO (not VOID) when a sampled absence exists.

Confidence Score: 5/5

Safe to merge — the change is a minimal, surgical fix to one helper function with direct test coverage of the regression path.

The fix correctly replicates the record ? 1 : 0 pattern from extractMetricValue, the early-return guard prevents the absent-record NaN path for all present() calls, and the new test suite directly encodes parity between the two helpers plus an end-to-end NO-resolution check. No other call sites or code paths are touched.

No files require special attention.

Important Files Changed

Filename	Overview
scripts/_forecast-resolution-eval.mjs	Adds an early-return present() branch in extractMetricObservation that returns value: 0 (not NaN) when no matching record exists, mirroring extractMetricValue and preventing false VOIDs on absent subjects.
tests/forecast-resolution-eval.test.mjs	Adds a focused describe block with three tests covering the fixed semantics: absent → 0, present → 1, and end-to-end NO resolution from a sampled absence, including a parity assertion against extractMetricValue.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[extractMetricObservation called] --> B[findMatchingRecord]
    B --> C{parsed.fn === 'present'?}
    C -- YES --> D{record found?}
    D -- YES --> E["asOf = parseAsOfMs(record.asOf ?? record.date)"]
    E --> F["return { value: 1, asOf: isFinite(asOf) ? asOf : null }"]
    D -- NO --> G["return { value: 0, asOf: null }"]
    C -- NO --> H{record found?}
    H -- NO --> I["return { value: NaN, asOf: null }"]
    H -- YES --> J["asOf = parseAsOfMs(record.asOf ?? record.date)"]
    J --> K["return { value: valueFromRecord(fn, record), asOf: ... }"]

    style G fill:#d4edda,stroke:#28a745
    style I fill:#f8d7da,stroke:#dc3545
    style F fill:#d4edda,stroke:#28a745

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[extractMetricObservation called] --> B[findMatchingRecord]
    B --> C{parsed.fn === 'present'?}
    C -- YES --> D{record found?}
    D -- YES --> E["asOf = parseAsOfMs(record.asOf ?? record.date)"]
    E --> F["return { value: 1, asOf: isFinite(asOf) ? asOf : null }"]
    D -- NO --> G["return { value: 0, asOf: null }"]
    C -- NO --> H{record found?}
    H -- NO --> I["return { value: NaN, asOf: null }"]
    H -- YES --> J["asOf = parseAsOfMs(record.asOf ?? record.date)"]
    J --> K["return { value: valueFromRecord(fn, record), asOf: ... }"]

    style G fill:#d4edda,stroke:#28a745
    style I fill:#f8d7da,stroke:#dc3545
    style F fill:#d4edda,stroke:#28a745

_{Reviews (1): Last reviewed commit: "fix(forecast): present() observation mus..." | Re-trigger Greptile}

vercel Bot deployed to Preview July 15, 2026 00:24 View deployment

fix(review): assert present observation timestamp

76b925e

koala73 merged commit e8743e9 into main Jul 15, 2026
26 checks passed

koala73 deleted the fix/forecast-present-observation-void branch July 15, 2026 00:44

koala73 mentioned this pull request Jul 15, 2026

forecast: retire/repoint the legacy infra 'cascade risk' present() forecasts (VOID-prone, mis-resolved, trivially-NO) — #5233 fan-out #5330

Closed

koala73 mentioned this pull request Jul 23, 2026

feat(forecast): Phase 2 re-engine — LLM ensemble + calibration on prediction-markets + FRED (#4930 Bet 4) #5525

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(forecast): present() observes 0 (not NaN) on absence — VOID-rate regression#5327

fix(forecast): present() observes 0 (not NaN) on absence — VOID-rate regression#5327
koala73 merged 2 commits into
mainfrom
fix/forecast-present-observation-void

koala73 commented Jul 15, 2026

Uh oh!

vercel Bot commented Jul 15, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Jul 15, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

koala73 commented Jul 15, 2026

Why

What

Evidence

VOID-rate triage context (separate from this fix)

Uh oh!

vercel Bot commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps Bot commented Jul 15, 2026

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Flowchart

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented Jul 15, 2026 •

edited

Loading