Phase 0: Regional Intelligence snapshot writer foundation#2940
Merged
Conversation
…napshot types
Adds shared/ foundation for the Regional Intelligence Model:
- shared/regions.types.d.ts: type contract for RegionalSnapshot, BalanceVector
(7 axes, pressures vs buffers split), ScenarioSet (normalized per horizon),
TriggerThreshold (structured operator+baseline), ActorState/LeverageEdge,
TransmissionPath (typed magnitude/latency/asset class with template
provenance), MobilityState, NarrativeSection, SnapshotDiff, SnapshotMeta
(snapshot_id + freshness tracking).
- shared/iso2-to-region.json: 218 ISO2 codes mapped to 8 regions, generated
from World Bank /v2/country with strategic overrides documented inline:
AF/PK -> south-asia (WB has them in MEA)
TR -> mena (WB has it in ECS)
MX -> north-america (WB has it in LCN, USMCA frame)
TW -> east-asia (WB does not list Taiwan)
- shared/geography.js: 8 regions, 19 theaters, 11 corridors with chokepoint
links. Country and corridor criticality weights matching the scoring
appendix. Helper functions: getRegion, regionForCountry, getRegionCorridors,
countryCriticality.
- scripts/shared/iso2-to-region.json: mirror per the existing convention
enforced by tests/edge-functions.test.mjs.
Spec: docs/internal/pro-regional-intelligence-upgrade.md
…te pipeline
Adds the Phase 0 Regional Intelligence snapshot writer. No LLM narrative call;
narrative arrives in Phase 1. All scoring is deterministic.
Compute pipeline (canonical order, mirrors spec):
1. read raw inputs (parallel Redis pipeline)
2. build pre_meta (versions, freshness, missing/stale, valid_until)
3. compute 7-axis balance vector with weighted-tail domestic fragility
4. score actors and leverage edges from forecast case files
5. evaluate structured trigger thresholds (BEFORE scenarios)
6. build scenario sets normalized to 1.0 per horizon
7. resolve transmission templates against active triggers
8. mobility (empty in v1, see appendix)
9. gather and attribute evidence chain
10. SKIPPED: narrative (Phase 1)
11. assign UUID v7 snapshot_id
12. read previous snapshot, run diffRegionalSnapshot
13. build final_meta with diff-derived trigger_reason
14. persist atomically (SET NX EX dedup, ZADD index, ZREMRANGEBYSCORE prune,
DEL optional :live)
15. (alerts emitted from diff in Phase 2)
Modules under scripts/regional-snapshot/:
_helpers, freshness (registry of source max-age),
balance-vector (per-axis formulas from scoring appendix, weighted-tail
fragility, deterministic energy split into vulnerability vs leverage),
regime-derivation (rule table), actor-scoring,
trigger-evaluator + triggers.config (delta operators dormant in Phase 0),
scenario-builder (per-horizon normalization), transmission-templates
(~7 templates with template_id + template_version),
evidence-collector, snapshot-meta (pre/final split),
diff-snapshot (single source of truth for state changes),
persist-snapshot (atomic SET NX EX for idempotency, prune on every persist).
Entry point scripts/seed-regional-snapshots.mjs:
- isMain guard so the bundle runner can spawn it as a child process
- Reads all inputs once, then iterates 8 regions
- Writes seed-meta:intelligence:regional-snapshots after the loop
- Exits non-zero only if every region failed
Tests in tests/regional-snapshot.test.mjs (50 tests, all passing):
geography invariants, helpers, balance vector axes and decomposition,
weighted-tail amplification, regime rules, trigger gating, scenario
normalization sums, transmission template matching, snapshot meta
pre/final merge, diff engine for every alert type, end-to-end across all
8 regions.
Spec: docs/internal/pro-regional-intelligence-upgrade.md
…y 6h
- scripts/seed-bundle-derived-signals.mjs: add Regional-Snapshots section
(6h interval, 180s timeout). Bundle cron fires every 5min; the freshness
gate ensures the snapshot script only runs when seed-meta is >4.8h old,
so the effective cadence is 6h while sharing the existing service slot.
- api/health.js SEED_META: add regionalSnapshots entry with maxStaleMin 720
(12h = 2x cron interval). Health check goes red if no snapshot has been
persisted in the last 12 hours.
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Contributor
Greptile SummaryThis PR ships the Phase 0 foundation for the Regional Intelligence Model: geography taxonomy, deterministic compute pipeline (balance vector → regime → actors → triggers → scenarios → transmissions → evidence → diff), Redis persistence with SET NX idempotency, and health-check wiring. The architecture is clean and the pipeline ordering is sound, but one input key version mismatch in the freshness registry will silently degrade every snapshot's
Confidence Score: 4/5Safe to merge after fixing the advisories key version; all other findings are P2 style/future-proofing. One P1 defect in freshness.mjs references a Redis key (advisories-bootstrap:v2) that is never written, permanently understating snapshot_confidence. The fix is a one-character change. All remaining comments are P2. scripts/regional-snapshot/freshness.mjs (wrong key version at line 30) Important Files Changed
Reviews (1): Last reviewed commit: "chore(seed-bundle): wire regional-snapsh..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| { key: 'economic:eu-gas-storage:v1', maxAgeMin: 2880, feedsAxes: ['energy_vulnerability'] }, | ||
| { key: 'economic:spr:v1', maxAgeMin: 10080, feedsAxes: ['energy_buffer'] }, | ||
| { key: 'energy:chokepoint-flows:v1', maxAgeMin: 240, feedsAxes: ['transmission'] }, | ||
| { key: 'intelligence:advisories-bootstrap:v2', maxAgeMin: 240, feedsAxes: ['mobility', 'evidence'] }, |
Contributor
There was a problem hiding this comment.
The key here is intelligence:advisories-bootstrap:v2, but every other reference in the codebase uses v1 — the seed script (seed-security-advisories.mjs), health check (api/health.js), api/bootstrap.js, server/_shared/cache-keys.ts, and seed-cross-source-signals.mjs all write and read v1. Because v2 is never written to Redis, classifyInputs will always classify this entry as missing, permanently reducing snapshot_confidence by roughly 6–7 % and polluting missing_inputs on every snapshot with this key.
Change v2 to v1 to match the actual key written by the advisories seeder.
Comment on lines
+120
to
+126
| function isCloseToThreshold(value, threshold) { | ||
| // 80% of the threshold counts as "watching" | ||
| const target = threshold.value; | ||
| if (target === 0) return false; | ||
| const ratio = value / target; | ||
| return ratio > 0.8 && ratio < 1.0; | ||
| } |
Contributor
There was a problem hiding this comment.
isCloseToThreshold never fires for lt / lte operators
When evaluateThreshold returns false for a lt threshold, the caller already guarantees value >= target, meaning ratio = value / target >= 1.0. The condition ratio > 0.8 && ratio < 1.0 can never be true, so the watching state is unreachable for any lt/lte trigger — they jump directly from dormant to active. No current triggers use these operators, so there's no runtime impact today, but future phases may add them.
Comment on lines
+74
to
+77
| if (targetHorizon === '24h') return /h24|24h|day|24h/.test(forecastHorizon); | ||
| if (targetHorizon === '7d') return /d7|7d|week|d7/.test(forecastHorizon); | ||
| if (targetHorizon === '30d') return /d30|30d|month|d30/.test(forecastHorizon); | ||
| return false; |
Contributor
There was a problem hiding this comment.
matchesHorizon
Each pattern contains a duplicated alternative — 24h appears twice in the first regex, d7 twice in the second, d30 twice in the third. The duplicates are harmless but suggest four distinct patterns when there are only three.
| trigger_reason: triggerReason, | ||
| }); | ||
|
|
||
| return { ...tentativeSnapshot, meta: finalMeta, diff }; |
Contributor
There was a problem hiding this comment.
diff persisted in snapshot JSON but absent from the RegionalSnapshot type
The spread here attaches diff: SnapshotDiff to the object before it is serialised to Redis, but RegionalSnapshot in regions.types.d.ts has no diff field. Consumers that deserialise via readLatestSnapshot will hold an undeclared field that may confuse Phase 1 codegen. Either add diff?: SnapshotDiff to the interface, or strip diff before persisting.
4 tasks
…riter Resolves all 5 critical findings from the multi-agent code review of PR #2940. End-to-end smoke test passes for all 8 regions with confidence=1.0. P1 #166: health-seed-meta-not-in-keys-loops - Add regionalSnapshots to STANDALONE_KEYS in api/health.js - Without this, the SEED_META entry was dead wiring and the 12h staleness budget was unobservable. Same failure mode as the 'empty data ok keys bootstrap blind spot' the team has hit before. P1 #167: oref-trigger-key-not-in-freshness-registry - Add relay:oref:history:v1 to FRESHNESS_REGISTRY (the canonical OREF key written by ais-relay, not the wrong intelligence:oref-alerts:v1 that the code was originally reading). - Update trigger-evaluator to read activeAlertCount/historyCount24h from the actual relay:oref:history:v1 payload shape. - oref_cluster trigger now fires when activeAlertCount > 10 (verified end-to-end against mock data). P1 #168: zombie-freshness-registry-keys - Remove 4 freshness registry entries that no compute module reads: supply_chain:shipping_stress, energy:chokepoint-flows, intelligence:advisories-bootstrap, market:commodities-bootstrap. - These were dragging snapshot_confidence below 1.0 and wasting Redis pipeline reads. Add a header comment forbidding speculative entries. P1 #169: diff-field-leaks-into-persisted-snapshot - Change computeSnapshot to return { snapshot, diff } separately so the diff is consumed locally for inferTriggerReason and never serialized into the persisted snapshot. The diff field was not part of the RegionalSnapshot type and would have broken Phase 1 proto codegen. - Update main() to destructure { snapshot } before persistSnapshot. P1 #170: jsdoc-types-not-enforced-jsconfig-missing - Add scripts/seed-regional-snapshots.mjs and scripts/regional-snapshot/** to scripts/jsconfig.json's include array so tsc --checkJs validates the JSDoc @type annotations. - Add // @ts-check directive to all 14 .mjs files in the regional-snapshot pipeline. - Fix 4 type-safety bugs surfaced by enabling type-checking: actor-scoring: explicit ActorState[] type on local actors array; cast leverage_domains to ActorLeverageDomain[]; typed ActorRole return on inferRole. evidence-collector: explicit EvidenceItem[] type on local out array. scenario-builder: typed HORIZONS as ScenarioHorizon[] and LANE_NAMES as ScenarioName[]. transmission-templates: cast tpl.affectedRegions to RegionId[] when building TransmissionPath objects. - Pre-existing seed-forecasts.mjs and _r2-storage.mjs errors (46 total) are not part of this PR's scope and remain untouched. Verification - npx tsc --noEmit -p scripts/jsconfig.json: 0 errors in regional-snapshot files - npm run typecheck:all: clean - npm run test:data: 3946/3946 pass - tests/regional-snapshot.test.mjs: 50/50 pass - End-to-end smoke test: all 8 regions compute clean with confidence=1.0 - Verified: oref_cluster trigger fires when activeAlertCount > 10 - Verified: persisted snapshot has no diff field
… review Records 19 P2 and 3 P3 todos from the kieran-typescript, security-sentinel, performance-oracle, architecture-strategist, code-simplicity, agent-native, and learnings researcher reviews of PR #2940 and PR #2942. P2 (19): isCloseToThreshold inverted for lt; sequential readLatestSnapshot 1600ms; sequential persist 1600ms; seeder bypasses runSeed; region taxonomy 3 sources of truth; Redis keys coupled to compute modules; stored XSS risk; AbortController missing; getForecasts no cachedFetchJson; REGIONS.find duplicated; undated inputs treated as fresh; inconsistent unknown-region error handling; writeExtraKeyWithMeta positional args; pipeline non-atomic persist; trigger watching runs on delta operators; regime.transition_driver always empty; orphan scripts/shared mirror; dangling docs/internal refs; refetchForRegion silent error swallowing. P3 (3): redundant JSON.stringify(caseFile) hot loops; helper/config cleanups (round dedup, generateSnapshotId entropy, matchesHorizon regex, coercive_pressure dead 0.30 weight); perf micro-cleanups (buildPreMeta x8, signal indexing, prototype pollution guards).
Owner
Author
P1 review findings addressed (5/5)All 5 P1 findings from the multi-agent code review have been fixed in commits 3f449f3 and d322b25. The 22 remaining P2/P3 findings are tracked as pending todos in What changed
Verification
Pre-existing tech debt (NOT addressed in this PR)
Open todos (22 P2/P3, tracked in
|
…tor and seed exit
Fixes wrong payload shapes in computeCapitalStress/computeEnergyVulnerability
and silent partial-failure handling in the seed entry point.
P1: computeCapitalStress wired to wrong production shapes and units
- economic:macro-signals:v1 emits verdict 'BUY' | 'CASH' | 'UNKNOWN', not
'BEARISH' | 'NEUTRAL' | 'BULLISH' (seed-economy.mjs:497). Map CASH=1,
BUY=0, UNKNOWN=0.5.
- economic:national-debt:v1 shape is { entries: [{ iso3, debtToGdp, ... }] },
not { countries: [{ country, debtToGdp }] } (seed-national-debt.mjs:101).
Filter via iso3 -> iso2 lookup (shared/iso3-to-iso2.json) to match
region countries.
- debtToGdp is a PERCENTAGE (e.g., 123 for 123%), not a 0-1 fraction.
Old code computed (avgDebt - 0.6) / 0.8 assuming fraction; that
saturated every non-zero value to 1.0. New code uses
(avgDebtPct - 60) / 80 so 60% is neutral and 140% saturates.
- economic:stress-index:v1 is a SINGLE GLOBAL object with
{ compositeScore: 0-100, label, components } from seed-economy.mjs
computeStressIndex(), NOT a per-country table. Apply as a global
overlay that scales every region's capital_stress equally (the FRED
composite primarily reflects US financial conditions, which do affect
every region).
P2: computeEnergyVulnerability read wrong field and wrong scale
- energy:mix:v1:_all stores importShare (OWID percentage 0-100), not
imported (seed-owid-energy-mix.mjs:193). Old code read m.imported
which is undefined, producing avgImport = 0 always.
- Divide by 100 to normalize percentage to 0-1 fraction before clipping.
- Skip countries with null importShare (exclude from average, don't
treat as zero). Report the valid-count in the driver description so
the evidence chain reflects actual coverage.
P1: partial region failures silently treated as successful run
- Old exit condition (failed > 0 && persisted === 0) meant 7 successful
+ 1 failed region wrote fresh seed-meta and returned exit 0. Health
check stayed OK and the bundle runner never retried.
- New behavior: only write seed-meta when ALL regions succeed. On any
failure, log the failedRegions array with error messages, skip the
seed-meta write, and exit 1. The 12h maxStaleMin means /api/health
will flip to STALE after two consecutive degraded runs, and the
bundle runner freshness gate will retry on the next 5-min cycle.
- Track failedRegions for operator observability in the log tail.
Verification
- tests/regional-snapshot.test.mjs: 50/50 pass
- npm run test:data: 3950/3950 pass
- npm run typecheck:all: clean
- npx tsc --noEmit -p scripts/jsconfig.json: 0 errors in regional-snapshot
- biome lint: clean on modified files
- End-to-end smoke test with production payload shapes verifies:
- CASH verdict drives cMacro=1.0 (MENA capital_stress=0.454 matches formula)
- Debt 123%/107% drives cDebt=0.688 (N.America capital_stress=0.592)
- Stress compositeScore=68 drives cStress=0.68 (global overlay)
- importShare 85% normalizes to 0.85 (MENA energy_vulnerability=0.197)
- importShare null excluded from average, not zeroed
Owner
Author
Review round 2 findings addressed (2 P1 + 1 P2)Fixes pushed in commit c92eb12 address all 3 findings from the round-2 review. What changed
VerificationEnd-to-end smoke test with actual production payload shapes confirms the math:
All assertions pass with predicted numerical matches. Test coverage
Follow-up considerationsThe global stress-index overlay is a simplification. The FRED composite primarily reflects US/global financial conditions. Phase 1 could introduce regional overlays (e.g., EU financial stress, EM sovereign spreads) and weight them per region. For Phase 0, the global overlay is acceptable since every region is affected by US conditions. |
…skipped Addresses 1 P1 finding on PR #2940: the seed script was overwriting a good summary with an empty {regions: [], recordCount: 0} payload whenever a cron invocation landed inside the 15-minute idempotency bucket and every region returned duplicate-bucket. api/health.js classifies the empty array as EMPTY_DATA, flipping /api/health to red on a completely harmless retry. New health policy in the seed main(): 1. persisted > 0 && failed === 0: write fresh summary + seed-meta 2. persisted === 0 && failed === 0: all regions dedup-skipped. Do NOT write (preserve the prior good summary). Return cleanly. 3. failed > 0: skip meta write and exit 1. The 12h maxStaleMin budget flips /api/health to STALE on sustained degradation. The bundle runner's freshness gate (intervalMs * 0.8) means the next full run lands roughly every 6h regardless. In steady state, the 15min dedup bucket only catches bundle cron retries after a prior success within the same bucket, so path 2 is rare but real: the first cron tick after a pod restart inside the dedup window is the failure mode this fixes. Verification - tests/regional-snapshot.test.mjs: 50/50 pass - npm run typecheck:all: clean - tsc -p scripts/jsconfig.json: 0 errors in regional-snapshot - biome lint: clean on touched file - Static analysis confirms exactly 1 writeExtraKeyWithMeta call (only in branch 1) so branches 2 and 3 cannot accidentally overwrite the summary
Owner
Author
Review round 3 — 1 P1 addressedFix in commit bef3225. Problem
FixNew three-branch health policy in
In steady state, the 15-minute dedup bucket rarely traps a full run because the bundle runner's freshness gate ( Verification
|
5 tasks
koala73
added a commit
that referenced
this pull request
Apr 11, 2026
Two review findings on PR #2940 caused MENA/SSA snapshots to silently drop broad cross-source signals and leak foreign chokepoint events into every region's evidence chain. P1 - theater label matching seed-cross-source-signals.mjs normalizes raw values to broad display labels like "Middle East" and "Sub-Saharan Africa". The consumer side compared these against region.theaters kebab IDs (levant, persian-gulf, horn-of-africa). "middle east" does not substring-match any of those, so every MENA signal emitted with the broad label was silently dropped from coercive_pressure and the evidence chain. Same for SSA. Added signalAliases per region and a shared isSignalInRegion helper in shared/geography.js. Both balance-vector.mjs and evidence-collector.mjs now route signals through the helper, which normalizes both sides and matches against theater IDs or region aliases. P2 - chokepoint region leak evidence-collector.mjs:62 iterated every chokepoint in the payload without filtering by regionId, so Taiwan Strait, Baltic, and Panama threat events surfaced in MENA and SSA evidence chains. Now derives the allowed chokepoint ID set from getRegionCorridors(regionId) and skips anything not owned by the region. Added 15 unit tests covering: broad-label matching, kebab/spaced input, cross-region rejection, and the chokepoint filter for MENA/East Asia/ Europe/SSA.
5 tasks
koala73
added a commit
that referenced
this pull request
Apr 11, 2026
Two review findings on PR #2940 caused MENA/SSA snapshots to silently drop broad cross-source signals and leak foreign chokepoint events into every region's evidence chain. P1 - theater label matching seed-cross-source-signals.mjs normalizes raw values to broad display labels like "Middle East" and "Sub-Saharan Africa". The consumer side compared these against region.theaters kebab IDs (levant, persian-gulf, horn-of-africa). "middle east" does not substring-match any of those, so every MENA signal emitted with the broad label was silently dropped from coercive_pressure and the evidence chain. Same for SSA. Added signalAliases per region and a shared isSignalInRegion helper in shared/geography.js. Both balance-vector.mjs and evidence-collector.mjs now route signals through the helper, which normalizes both sides and matches against theater IDs or region aliases. P2 - chokepoint region leak evidence-collector.mjs:62 iterated every chokepoint in the payload without filtering by regionId, so Taiwan Strait, Baltic, and Panama threat events surfaced in MENA and SSA evidence chains. Now derives the allowed chokepoint ID set from getRegionCorridors(regionId) and skips anything not owned by the region. Added 15 unit tests covering: broad-label matching, kebab/spaced input, cross-region rejection, and the chokepoint filter for MENA/East Asia/ Europe/SSA.
koala73
added a commit
that referenced
this pull request
Apr 11, 2026
* fix(intelligence): region-scope signals and chokepoint evidence Two review findings on PR #2940 caused MENA/SSA snapshots to silently drop broad cross-source signals and leak foreign chokepoint events into every region's evidence chain. P1 - theater label matching seed-cross-source-signals.mjs normalizes raw values to broad display labels like "Middle East" and "Sub-Saharan Africa". The consumer side compared these against region.theaters kebab IDs (levant, persian-gulf, horn-of-africa). "middle east" does not substring-match any of those, so every MENA signal emitted with the broad label was silently dropped from coercive_pressure and the evidence chain. Same for SSA. Added signalAliases per region and a shared isSignalInRegion helper in shared/geography.js. Both balance-vector.mjs and evidence-collector.mjs now route signals through the helper, which normalizes both sides and matches against theater IDs or region aliases. P2 - chokepoint region leak evidence-collector.mjs:62 iterated every chokepoint in the payload without filtering by regionId, so Taiwan Strait, Baltic, and Panama threat events surfaced in MENA and SSA evidence chains. Now derives the allowed chokepoint ID set from getRegionCorridors(regionId) and skips anything not owned by the region. Added 15 unit tests covering: broad-label matching, kebab/spaced input, cross-region rejection, and the chokepoint filter for MENA/East Asia/ Europe/SSA. * fix(intelligence): address Greptile P2 review findings on #2952 Two P2 findings from Greptile on the region-scoping PR. 1) Drop 'eu' short alias from europe.signalAliases `isSignalInRegion` uses substring matching, and bare 'eu' would match any theater label containing those two letters ('fuel', 'neutral zone', 'feudal'). Replaced with 'european union' which is long enough to be unambiguous. No seed currently emits a bare 'eu' label, so this is pure hardening. 2) Make THEATERS.corridorIds live data via getRegionCorridors union horn-of-africa declared corridorIds: ['babelm'] and caribbean declared corridorIds: ['panama'], but `getRegionCorridors` only consulted CORRIDORS.theaterId — so those entries were dead data. After yesterday's region-scoped chokepoint filter, Bab el-Mandeb threat events landed ONLY in MENA evidence (via the primary red-sea theater), never in SSA, even though the corridor physically borders Djibouti/Eritrea. Same for Panama missing from LatAm. `getRegionCorridors` now unions direct theater membership (via CORRIDORS.theaterId) with indirect claims (via THEATERS.corridorIds), de-duplicated by corridor id. This reflects geopolitical reality: - MENA + SSA both see babelm threat events - NA + LatAm both see panama threat events Scoring impact: SSA maritime_access now weighs babelm (weight 0.9), LatAm maritime_access now weighs panama (weight 0.6). These were missing buffers under the pre-fix model. Added regression tests for both new paths. The existing "SSA evidence has no chokepoints" test was inverted to assert SSA now DOES include babelm (and excludes MENA/East Asia corridors).
This was referenced Apr 11, 2026
10 tasks
koala73
added a commit
that referenced
this pull request
May 18, 2026
…ated inputs as stale (#3728) Two independent defects in the regional snapshot meta builder were both fabricating snapshot_confidence and valid_until: 1. classifyInputs() in freshness.mjs defaulted present-but-undated payloads to "fresh" — a stalled seeder that left an old payload without a timestamp would silently inflate snapshot_confidence. Flipped the default to "stale": you cannot prove freshness without a timestamp. 2. buildPreMeta() in snapshot-meta.mjs hardcoded valid_until = now + 6h, ignoring the per-input maxAgeMin in FRESHNESS_REGISTRY. Replaced with min(ts + maxAgeMin*60_000) across fresh inputs, clamped to [now, now + 6h]. When no inputs are fresh, valid_until collapses to now. Updates tests/regional-snapshot.test.mjs (5 new cases) and the pre-existing mobility test that asserted the old undated-as-fresh behavior. Resolves TODO #181 (pending since PR #2940). Closes #3728
koala73
added a commit
that referenced
this pull request
May 18, 2026
…d_until (#3728) (#3781) * fix(regional-snapshot): derive valid_until from input TTLs; treat undated inputs as stale (#3728) Two independent defects in the regional snapshot meta builder were both fabricating snapshot_confidence and valid_until: 1. classifyInputs() in freshness.mjs defaulted present-but-undated payloads to "fresh" — a stalled seeder that left an old payload without a timestamp would silently inflate snapshot_confidence. Flipped the default to "stale": you cannot prove freshness without a timestamp. 2. buildPreMeta() in snapshot-meta.mjs hardcoded valid_until = now + 6h, ignoring the per-input maxAgeMin in FRESHNESS_REGISTRY. Replaced with min(ts + maxAgeMin*60_000) across fresh inputs, clamped to [now, now + 6h]. When no inputs are fresh, valid_until collapses to now. Updates tests/regional-snapshot.test.mjs (5 new cases) and the pre-existing mobility test that asserted the old undated-as-fresh behavior. Resolves TODO #181 (pending since PR #2940). Closes #3728 * fix(regional-snapshot): wire 5 upstream feeds to seed-meta to prevent on-deploy false STALE (#3781 follow-on) Static analysis identified 5 Redis keys whose payloads carry no timestamp `extractTimestamp` recognises and would therefore classify as STALE on first deploy under #3728's new logic. All five already have a `seed-meta:*` companion written by existing seeders (and tracked by `api/health.js`); we just point the freshness registry at them. - risk:scores:sebuf:stale:v1 → seed-meta:intelligence:risk-scores - intelligence:cross-source-signals:v1 → seed-meta:intelligence:cross-source-signals - energy:mix:v1:_all → seed-meta:economic:owid-energy-mix - supply_chain:transit-summaries:v1 → seed-meta:supply_chain:transit-summaries - relay:oref:history:v1 → seed-meta:relay:oref:history (new write added next to existing envelopeWrite in ais-relay.cjs) Tests lock in the wiring so a later removal trips a unit test, not production. * fix(regional-snapshot): address review #3781 — gate seed-meta on envelope ok, tick-consistent valid_until, fix wrong-assertion test, parameterize metaKey resolution Review round-2 fixes for PR #3781: - ais-relay.cjs: gate the seed-meta:relay:oref:history write on envelope write success. Previously the meta write could land alone if the envelope write failed (Upstash 5xx / blip), letting the freshness classifier report FRESH for data that does not exist in Redis. Added a note about the pre-existing asymmetry at the transit-summaries write (~line 7370), which is intentionally out of scope here. - snapshot-meta.mjs: snapshot `Date.now()` once in buildPreMeta and pass it as a 4th arg to deriveValidUntil so the two-step compute is tick-consistent. Removes the need for timing tolerance in tests that assert exact valid_until values. - regional-snapshot.test.mjs: split the misnamed 6h-cap test into (a) "tightest input wins" and (b) "cap binds when all inputs out-TTL the cap". The previous test claimed to exercise the cap but the 15-min input always won. New (b) restricts the sources map to registry entries with maxAgeMin > 6h so the cap is actually binding. - regional-snapshot.test.mjs: parameterize the metaKey-resolution test across all 5 newly-wired keys (risk:scores, cross-source-signals, energy:mix, transit-summaries, relay:oref:history) so removing any single metaKey trips a test.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 0 of the Regional Intelligence Model. Foundation only - no UI changes, no PRO panel, no LLM calls. This PR ships the canonical data contract, the deterministic compute pipeline, and the persistence/idempotency layer that every subsequent phase reads from.
shared/geography.jsdefines 8 display regions, 19 theaters, 11 corridors with chokepoint links, country and corridor criticality weights matching the scoring appendix.shared/regions.types.d.tsis the type contract forRegionalSnapshotand every nested message (BalanceVector7-axis split,ScenarioSetper horizon,TriggerThresholdstructured operator+baseline,ActorState/LeverageEdge, typedTransmissionPathwith template provenance,MobilityState,NarrativeSection,SnapshotDiff). Mirrors the proto shape that Phase 1 codegens.shared/iso2-to-region.json(218 codes from World Bank API) with strategic overrides documented inline: AF/PK to south-asia, TR to mena, MX to north-america (USMCA frame), TW to east-asia.scripts/regional-snapshot/modules implement the canonical compute pipeline: pre-meta -> balance -> actors -> triggers (BEFORE scenarios) -> normalized scenarios -> transmissions -> evidence -> snapshot_id -> diff -> final_meta. No LLM in Phase 0.scripts/seed-regional-snapshots.mjsentry point withisMainguard. Persists atomically viaSET NX EXfor idempotency, prunes the index on every persist (ZREMRANGEBYSCORE), busts the optional:livecache.seed-bundle-derived-signalsat 6h interval. The bundle cron fires every 5min; the freshness gate ensures the snapshot only runs when seed-meta is >4.8h old, sharing the existing service slot.tests/regional-snapshot.test.mjswith 50 unit tests covering geography invariants, balance vector formulas, weighted-tail amplification, regime rules, trigger gating, scenario normalization sums, transmission template matching, snapshot meta merging, diff engine for every alert type, and end-to-end across all 8 regions.Spec
Architecture:
docs/internal/pro-regional-intelligence-upgrade.mdEngineering appendix:
docs/internal/pro-regional-intelligence-appendix-engineering.mdScoring appendix:
docs/internal/pro-regional-intelligence-appendix-scoring.mdTesting
npm run test:data- 3950 tests pass (the new 50 + the existing 3900)npm run typecheck:all- cleannpx biome lint- 0 warnings on the new filesWhat this PR does NOT include
RegionalIntelligenceBoardpanel and proto/RPC handlers (Phase 1)get-regional-snapshotRPC (Phase 1)Post-Deploy Monitoring & Validation
seed-bundle-derived-signalslogs for[Regional-Snapshots]lines. Expect one persist entry per region (8 lines) every 6 hours./api/healthendpoint should showregionalSnapshotsas OK once the first run lands (within 6h of merge).curl https://worldmonitor.app/api/health | jq '.checks.regionalSnapshots'should return ok status with a recent fetchedAt timestamp.GET intelligence:snapshot:v1:mena:latestreturns a snapshot_id, thenGET intelligence:snapshot-by-id:v1:{snapshotId}returns the full snapshot JSON.ZCARD intelligence:snapshot-index:v1:menashould be >= 1 after the first run.snapshot_confidencebetween 0.5 and 1.0 (lower if many input keys are missing or stale)regimedefaults tocalmfor low-activity regions, more interesting labels for active regionsregionalSnapshotsred (>720min stale): check Railway logs for failed runs, then revert this PR if the bundle service crashesduplicate-bucket: dedup logic inverted, revertsnapshot_confidence0 across all regions: input pipeline broken, investigate Redis credentialsTest plan
[Regional-Snapshots] Skipped(because no prior seed-meta exists yet, the first run should actually fire)seed-meta:intelligence:regional-snapshotsexists in Redislatestpointer per region/api/healthendpoint includesregionalSnapshotsin OK status