Skip to content

fix(forecast): harden simulation keyword matching#5128

Merged
koala73 merged 2 commits into
mainfrom
codex/issue-5102-sim-keyword-matching
Jul 9, 2026
Merged

fix(forecast): harden simulation keyword matching#5128
koala73 merged 2 commits into
mainfrom
codex/issue-5102-sim-keyword-matching

Conversation

@koala73

@koala73 koala73 commented Jul 9, 2026

Copy link
Copy Markdown
Owner

Summary

Simulation bucket and channel matching now avoids broad substring hits inside unrelated words, so labels like "Corporate debt briefing" no longer satisfy the rate keyword and "Paramilitary procurement" no longer satisfies security escalation matching.

The security-escalation channel also drops the bare attack and military keywords, keeping the channel tied to more specific escalation signals such as military action, conflict, war, strike, escalat, and geopolit.

Validation

  • Red proof: node --test --test-name-pattern "matching helpers" tests/forecast-trace-export.test.mjs failed before the production change on the new substring-matching regressions.
  • node --test --test-name-pattern "matching helpers" tests/forecast-trace-export.test.mjs — 21 tests passed.
  • node --test tests/forecast-trace-export.test.mjs — 333 tests passed.
  • npm run typecheck
  • ./node_modules/.bin/biome lint scripts/seed-forecasts.mjs tests/forecast-trace-export.test.mjs — exit 0; reported only pre-existing informational notes outside this diff.
  • git push -u origin HEAD pre-push hook passed: frontend typecheck, API typecheck, Convex typecheck, CJS syntax, Unicode safety, architectural boundary check, safe HTML guard, rate-limit policy lint, premium-fetch parity, edge bundle check, changed test file run, and version sync.

Related

Fixes #5102
Related: #5108, #5112


Compound Engineering
GPT-5 Codex

@vercel

vercel Bot commented Jul 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
worldmonitor Ignored Ignored Preview Jul 9, 2026 5:08pm

Request Review

@greptile-apps

greptile-apps Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR hardens simulation bucket/channel keyword matching by replacing plain .includes() with a word-boundary-anchored regex (^|\s)keyword, and removes the bare attack and military keywords from the security_escalation channel to prevent over-broad hits (e.g. "Paramilitary procurement" matching military, "Corporate debt briefing" matching rate).

Confidence Score: 3/5

Safe to merge for the reported false-positive fixes, but silently drops coverage for compound military terms like "airstrike" via the strike keyword — worth resolving or explicitly documenting before shipping.

The left-only word-boundary anchor correctly eliminates the infix false positives the PR targets. However, one-word compound forms where the keyword is not at the token start — most notably "airstrike" against the kept strike keyword — no longer match security_escalation. A simulation labeled "Airstrike on Hormuz oil infrastructure" with an empty summary would now go unclassified. This regression is unintentional and is not covered by any test.

The matching logic in scripts/seed-forecasts.mjs around simulationTextMatchesKeyword and the security_escalation keyword list; tests/forecast-trace-export.test.mjs needs a case for one-word compound forms such as "airstrike".

Important Files Changed

Filename Overview
scripts/seed-forecasts.mjs Introduces word-boundary-at-left matching for bucket/channel keyword lookup; drops attack and military from security_escalation. The regex correctly fixes infix false positives but silently removes coverage for compound words like "airstrike" where the kept keyword strike is only matched when it starts a token.
tests/forecast-trace-export.test.mjs Adds two new test cases covering the reported false-positive regressions. Tests are well-targeted; no test covers compound one-word forms like "airstrike" that the new regex now silently excludes.

Reviews (1): Last reviewed commit: "fix(forecast): harden simulation keyword..." | Re-trigger Greptile

Comment thread scripts/seed-forecasts.mjs
Comment thread scripts/seed-forecasts.mjs Outdated
@koala73
koala73 merged commit f48f4af into main Jul 9, 2026
23 checks passed
@koala73
koala73 deleted the codex/issue-5102-sim-keyword-matching branch July 9, 2026 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(forecast): MiroFish sim — demotion-gate threshold + completeness gaps (geo-dedup, archetypes, round-2 bias, timingMarkers) (P2, #5097)

1 participant