fix(forecast): resolve judged entries from deadline evidence#5196
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Greptile SummaryThis PR allows long-horizon judged forecasts to resolve against evidence collected around their deadline (a configurable 7-day window anchored on the deadline, bounded by a 14-day Redis ceiling) instead of remaining pending until the retry safety net forces a VOID. Hash-cap truncation is deliberately reclassified as bounded recency sampling so a deadline-window-covered archive can terminate on agreement, disagreement, or no-evidence.
Confidence Score: 4/5Safe to merge; the new resolution path is fail-closed for every genuine-incompleteness case and the test suite directly exercises deadline windowing, budget expiry, and chunk alignment. The new resolution path is carefully fail-closed for every genuine-incompleteness case. The two areas worth a second look: the silent drop of The central logic in Important Files Changed
Reviews (1): Last reviewed commit: "fix(forecast): resolve judged entries fr..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
- make chunk-read failure logs caller-aware - document cap sampling versus incomplete coverage - clamp evidence lookback to the archive read ceiling
- derive coverage from retained Redis scores - preserve partial-response alignment handling - degrade isolated row errors to incomplete evidence
1 participant
Summary
Long-horizon judged forecasts can now resolve from evidence around their deadline instead of remaining pending until the retry safety net forces a VOID. The resolver uses a configurable seven-day deadline window, reads up to a hard configurable 14-day Redis ceiling, and filters the shared archive back to each forecast's own window before judging.
Hash-cap truncation is now treated as bounded recency sampling, so a covered window can honestly terminate on agreement, disagreement, or no evidence. Genuine incompleteness remains fail-closed: unavailable archives, explicit incomplete coverage, missing Redis rows, invalid pipeline rows, and exhausted read budgets keep entries pending.
The larger 15,000-hash read is split into 500-command pipelines under a shared 25-second budget. The new knobs are:
FORECAST_RESOLUTION_JUDGE_EVIDENCE_LOOKBACK_MS(default 7 days)FORECAST_RESOLUTION_JUDGE_EVIDENCE_MAX_LOOKBACK_MS(default 14 days, enforced as a hard ceiling)FORECAST_RESOLUTION_JUDGE_ARCHIVE_HASH_LIMIT(default 15,000)Validation
npm run test:data: 14,552 passed, 6 skipped, 0 failednpm run typecheck:all: passednpm run lint: passed; only existing repository warnings/info were reportedVITE_VARIANT=full vite build: passed, including dashboard critical-CSS checksPost-deploy monitoring and validation
Owner: forecast service on-call / repository maintainer. Observe the first manual seed-service deployment through the first judged deadlines around 2026-07-14.
Healthy signals:
forecast:scorecard:v1.pendingJudgefalls as due judged entries terminate, whilescoredandvoidrise.FORECAST_RESOLUTION_JUDGE_RUN_BUDGET_MSwithout repeatedjudged archive unavailableor archive-budget errors.judge_retry_exhaustedoutcomes.Failure signals and response:
voidRaterises above roughly 40%, preserve the receipts and investigate judge agreement/evidence quality before considering a tiebreaker follow-up.The
seed-forecast-resolutionsservice still requires a manual Railway deployment after merge because its automatic deploy is blocked by the existingcheckSuitesskip. Use the establishedserviceInstanceDeployV2followed bydeploymentInstanceExecutionCreateflow documented in #5062; this PR does not deploy or merge the service.Fixes #5183