Phase 3 PR1: Regime drift history (writer + RPC)#2981
Conversation
Phase 3 PR1 of the Regional Intelligence Model. Adds an append-only
regime transition log per region plus a premium-gated RPC to read it.
## What landed
### New writer module: scripts/regional-snapshot/regime-history.mjs
Single public entry point:
recordRegimeTransition(region, snapshot, diff, opts?)
-> { recorded, entry, pushed, trimmed }
Pure builder + Redis-ops orchestrator + dependency-injected publisher.
Flow:
1. buildTransitionEntry() returns null when diff has no regime_changed
(steady-state snapshots produce no entry — pure transition stream)
2. publishTransitionWithOps() LPUSHes onto
intelligence:regime-history:v1:{region}, then LTRIMs to keep the
most recent REGIME_HISTORY_MAX (100) entries
3. defaultPublisher binds real Upstash REST calls; tests inject an
in-memory ops object for offline coverage
LTRIM failure is non-fatal — entry already landed, next cycle will
re-trim. LPUSH failure short-circuits and reports pushed=false. The
recorder NEVER throws and is wrapped in its own try/catch in the seed
loop so snapshot persist is never blocked.
### seed-regional-snapshots.mjs hook
Added a regime-history call alongside the existing alert-emitter call,
right after persistSnapshot success. Same best-effort contract:
unconditional try/catch, log warn on throw, continue main loop.
### Proto + RPC: GetRegimeHistory
proto/worldmonitor/intelligence/v1/get_regime_history.proto
- GetRegimeHistoryRequest { region_id, limit (0..100) }
- GetRegimeHistoryResponse { transitions: RegimeTransition[] }
- RegimeTransition { region_id, label, previous_label,
transitioned_at, transition_driver, snapshot_id }
region_id validated as strict kebab-case (same regex as
get-regional-snapshot). limit capped server-side at MAX_LIMIT=100,
defaulting to 50 when omitted.
Added to IntelligenceService in service.proto. Generated openapi
JSON/YAML committed via `make generate`.
### Server handler: server/worldmonitor/intelligence/v1/get-regime-history.ts
LRANGE-based read (newest-first because the writer LPUSHes). adapter
is a dedicated exported function adaptTransition(raw) for testability.
LRANGE helper is inlined here because server/_shared/redis.ts has no
list helpers yet — this is the first list-reading handler in the
intelligence service. If a second list reader lands, the helper can
be promoted to a shared util.
Empty list / Redis miss / failed JSON parse all return
{ transitions: [] } so the client can distinguish "never changed" from
"upstream broken" via the HTTP status code, not the body.
Registered in handler.ts.
### Premium gating + cache tier
src/shared/premium-paths.ts: added /api/intelligence/v1/get-regime-history
server/gateway.ts RPC_CACHE_TIER: same path with 'slow' tier (matches
route-parity contract enforced by
tests/route-cache-tier.test.mjs)
## Tests — 44 new unit tests
tests/regional-snapshot-regime-history.test.mjs (22 tests):
buildTransitionEntry (7):
- null on missing diff/region/snapshot
- returns entry on regime change
- first-ever transition (empty previous_label)
- falls back to generated_at when transitioned_at is missing
- preserves snapshot_id
publishTransitionWithOps (8):
- happy path (LPUSH + LTRIM both succeed)
- canonical key prefix
- LTRIM uses REGIME_HISTORY_MAX-1 stop
- LPUSH failure → not pushed, LTRIM not called
- LTRIM failure → pushed=true, trimmed=false (non-fatal)
- LPUSH/LTRIM throwing caught and reported
- null/empty entry → no-op
recordRegimeTransition (5):
- no-op on no regime change
- records on regime change
- publisher returning false → recorded=false
- publisher exceptions swallowed
- critical escalation labels preserved
module constants (2): key prefix + max are valid
tests/get-regime-history.test.mts (22 tests):
adaptTransition (4):
- all fields snake → camel
- missing fields → empty/zero defaults
- first-ever transition shape preserved
- non-numeric transitioned_at → 0
handler structural checks (7): canonical key prefix, LRANGE usage,
adapter export, handler export signature, MAX_LIMIT cap matches
writer, missing-region short-circuit, malformed-entry filter
intelligence handler registration (2): import + registration
security wiring (2): premium path + cache-tier entry
proto definition (7): RPC method declared, import wired, request
shape, kebab regex, limit bounds, RegimeTransition fields,
response shape
## Verification
- node --test tests/regional-snapshot-regime-history.test.mjs: 22/22 pass
- npx tsx --test tests/get-regime-history.test.mts: 22/22 pass
- npm run test:data: 4621/4621 pass
- npm run typecheck: clean
- npm run typecheck:api: clean
- biome lint on touched files: clean
## Deferred to future iterations
- Phase 3 PR2: weekly regional briefs LLM seeder (consumes regime history
to highlight drift events in the weekly summary)
- Phase 3 PR3: UI block in RegionalIntelligenceBoard for regime drift
timeline (can ride alongside or after PR2)
- Drift analytics: % of last N days spent in each regime, transition
frequency rolling window, regime cycle detection
- Alert triggers on drift cycles (e.g., "thrashed between regimes 3 times
in 7 days")
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryAdds an append-only per-region regime transition log: a writer module ( Confidence Score: 5/5Safe to merge — only a one-line comment inaccuracy found; all logic, security wiring, and error-handling paths are correct. The sole finding is a P2 comment that says "log once per region" but fires per malformed entry. All functional paths — LPUSH/LTRIM orchestration, handler limit capping, premium gating, proto validation, and 44 unit tests — are correct. No files require special attention. Important Files Changed
Sequence DiagramsequenceDiagram
participant Cron as seed-regional-snapshots.mjs
participant RH as regime-history.mjs
participant Upstash as Upstash Redis
Cron->>Cron: persistSnapshot(snapshot) → {persisted:true}
Cron->>RH: recordRegimeTransition(region, snapshot, diff)
RH->>RH: buildTransitionEntry() → entry or null
alt diff.regime_changed present
RH->>Upstash: LPUSH intelligence:regime-history:v1:{region} entry
Upstash-->>RH: list length (number)
RH->>Upstash: LTRIM intelligence:regime-history:v1:{region} 0 99
Upstash-->>RH: OK (failure non-fatal)
RH-->>Cron: {recorded:true, pushed:true, trimmed:true/false}
else no regime change
RH-->>Cron: {recorded:false, entry:null}
end
Note over Cron: try/catch — never blocks persist
participant Client as Premium Client
participant GW as Vercel Edge (gateway)
Client->>GW: GET /api/intelligence/v1/get-regime-history?region_id=mena&limit=10
GW->>GW: validateApiKey / bearer Pro check
GW->>GW: Cache-Control: slow-browser (max-age=300)
GW->>GW: getRegimeHistory handler
GW->>Upstash: LRANGE intelligence:regime-history:v1:mena 0 9
Upstash-->>GW: array of JSON strings
GW->>GW: JSON.parse + adaptTransition (snake to camel)
GW-->>Client: {transitions: [RegimeTransition, ...]}
Reviews (1): Last reviewed commit: "feat(intelligence): regime drift history..." | Re-trigger Greptile |
P2 #1 — transition_driver always empty in the live path buildRegimeState(balance, previousLabel, '') at Step 11 passed an empty driver because the diff hasn't been computed yet. The regime-history recorder reads snapshot.regime.transition_driver which was therefore always '' in production, despite tests exercising synthetic fixtures with a populated driver. Fix: after Step 15 derives triggerReason via inferTriggerReason(diff), backfill regime.transition_driver = triggerReason when a genuine regime change occurred. This ensures both the persisted snapshot's regime block AND the regime-history entry carry the real driver (e.g., 'regime_shift', 'trigger_activation', 'corridor_break'). Added 2 regression tests: populated driver flows through, and pre-fix empty-driver snapshots remain back-compatible. P2 #2 — Redis failure returns cached false-empty history get-regime-history.ts returned 200 {transitions:[]} on LRANGE failure. The gateway caches 200 GET responses at the slow tier, so a transient Upstash outage would be pinned as a false-empty history until the cache TTL expired. Fix: when redisLrange returns null (Redis unavailable or credentials missing), the response now includes upstreamUnavailable: true in the body. The gateway already checks for this flag in the response body (line 434) and sets Cache-Control: no-store, so transient failures are not cached. Added 1 structural test asserting the upstreamUnavailable flag is set. Verification: - 24/24 writer tests, 23/23 handler tests, 4624/4624 full suite pass - npm run typecheck: clean - biome lint on touched files: clean
Phase 3 PR2 of the Regional Intelligence Model. Adds LLM-powered weekly intelligence briefs per region, completing the core feature set. ## New seeder: scripts/seed-regional-briefs.mjs Standalone weekly cron script (not part of the 6h derived-signals bundle). For each non-global region: 1. Read the latest snapshot via two-hop Redis read 2. Read recent regime transitions from the history log (#2981) 3. Call the LLM once per region with regime trajectory + balance + triggers + narrative context 4. Write structured brief to intelligence:regional-briefs:v1:weekly:{region} with 8-day TTL (survives one missed weekly run) Reuses the same injectable-callLlm + parse-validation + provider-chain pattern from narrative.mjs and weekly-brief.mjs. ## New module: scripts/regional-snapshot/weekly-brief.mjs generateWeeklyBrief(region, snapshot, transitions, opts?) -> { region_id, generated_at, period_start, period_end, situation_recap, regime_trajectory, key_developments[], risk_outlook, provider, model } buildBriefPrompt() — pure prompt builder parseBriefJson() — JSON parser with prose-extraction fallback emptyBrief() — canonical empty shape Global region is skipped. Provider chain: Groq -> OpenRouter. Validate callback ensures only parseable responses pass (narrative.mjs PR #2960 review fix pattern). ## Proto + RPC: GetRegionalBrief proto/worldmonitor/intelligence/v1/get_regional_brief.proto - GetRegionalBriefRequest { region_id } - GetRegionalBriefResponse { brief: RegionalBrief } - RegionalBrief { region_id, generated_at, period_start, period_end, situation_recap, regime_trajectory, key_developments[], risk_outlook, provider, model } ## Server handler server/worldmonitor/intelligence/v1/get-regional-brief.ts Simple getCachedJson read + adaptBrief snake->camel adapter. Returns upstreamUnavailable: true on Redis failure so the gateway skips caching (matching the get-regime-history pattern from #2981). ## Premium gating + cache tier src/shared/premium-paths.ts + server/gateway.ts RPC_CACHE_TIER ## Tests — 27 new unit tests buildBriefPrompt (5): region/balance/transitions/narrative rendered, empty transitions handled, missing fields tolerated parseBriefJson (5): valid JSON, garbage, all-empty, cap at 5, prose extraction generateWeeklyBrief (6): success, global skip, LLM fail, garbage, exception, period_start/end delta emptyBrief (2): region_id + empty fields handler (4): key prefix, adapter export, upstreamUnavailable, registration security (2): premium path + cache tier proto (3): RPC declared, import wired, RegionalBrief fields ## Verification - npm run test:data: 4651/4651 pass - npm run typecheck + typecheck:api: clean - biome lint: clean
* feat(intelligence): weekly regional briefs (Phase 3 PR2) Phase 3 PR2 of the Regional Intelligence Model. Adds LLM-powered weekly intelligence briefs per region, completing the core feature set. ## New seeder: scripts/seed-regional-briefs.mjs Standalone weekly cron script (not part of the 6h derived-signals bundle). For each non-global region: 1. Read the latest snapshot via two-hop Redis read 2. Read recent regime transitions from the history log (#2981) 3. Call the LLM once per region with regime trajectory + balance + triggers + narrative context 4. Write structured brief to intelligence:regional-briefs:v1:weekly:{region} with 8-day TTL (survives one missed weekly run) Reuses the same injectable-callLlm + parse-validation + provider-chain pattern from narrative.mjs and weekly-brief.mjs. ## New module: scripts/regional-snapshot/weekly-brief.mjs generateWeeklyBrief(region, snapshot, transitions, opts?) -> { region_id, generated_at, period_start, period_end, situation_recap, regime_trajectory, key_developments[], risk_outlook, provider, model } buildBriefPrompt() — pure prompt builder parseBriefJson() — JSON parser with prose-extraction fallback emptyBrief() — canonical empty shape Global region is skipped. Provider chain: Groq -> OpenRouter. Validate callback ensures only parseable responses pass (narrative.mjs PR #2960 review fix pattern). ## Proto + RPC: GetRegionalBrief proto/worldmonitor/intelligence/v1/get_regional_brief.proto - GetRegionalBriefRequest { region_id } - GetRegionalBriefResponse { brief: RegionalBrief } - RegionalBrief { region_id, generated_at, period_start, period_end, situation_recap, regime_trajectory, key_developments[], risk_outlook, provider, model } ## Server handler server/worldmonitor/intelligence/v1/get-regional-brief.ts Simple getCachedJson read + adaptBrief snake->camel adapter. Returns upstreamUnavailable: true on Redis failure so the gateway skips caching (matching the get-regime-history pattern from #2981). ## Premium gating + cache tier src/shared/premium-paths.ts + server/gateway.ts RPC_CACHE_TIER ## Tests — 27 new unit tests buildBriefPrompt (5): region/balance/transitions/narrative rendered, empty transitions handled, missing fields tolerated parseBriefJson (5): valid JSON, garbage, all-empty, cap at 5, prose extraction generateWeeklyBrief (6): success, global skip, LLM fail, garbage, exception, period_start/end delta emptyBrief (2): region_id + empty fields handler (4): key prefix, adapter export, upstreamUnavailable, registration security (2): premium path + cache tier proto (3): RPC declared, import wired, RegionalBrief fields ## Verification - npm run test:data: 4651/4651 pass - npm run typecheck + typecheck:api: clean - biome lint: clean * fix(intelligence): address 3 review findings on #2989 P2 #1 — no consumer surface for GetRegionalBrief Acknowledged. The consumer is the RegionalIntelligenceBoard panel, which will call GetRegionalBrief and render a weekly brief block. This wiring is Phase 3 PR3 (UI) scope — the RPC + Redis key are the delivery mechanism, not the end surface. No code change in this commit; the RPC is ready for the panel to consume. P2 #2 — readRecentTransitions collapses failure to [] readRecentTransitions returned [] on Redis/network failure, which is indistinguishable from a genuinely quiet week. The LLM then generates a brief claiming "no regime transitions" when in reality the upstream is down — fabricating false input. Fix: return null on failure. The seeder skips the region with a clear log message when transitions is null, so the brief is never written with unreliable input. Empty array [] now only means genuinely no transitions in the 7-day window. P2 #3 — parseBriefJson accepts briefs the seeder rejects parseBriefJson treated non-empty key_developments as valid even if situation_recap was empty. The seeder gate only writes when brief.situation_recap is truthy. That mismatch means the validator pass + provider-fallback logic could accept a response that the seeder then silently drops. Fix: require situation_recap in parseBriefJson for valid=true, matching the seeder gate. Now both checks agree on what constitutes a usable brief, and the provider-fallback chain correctly falls through when a provider returns a brief with developments but no recap. * fix(intelligence): TTL path-segment fix + seed-meta always-write (Greptile P1+P2 on #2989) P1 — TTL silently not applied (briefs never expire) Upstash REST ignores query-string SET options (?EX=N). The correct form is path-segment: /set/{key}/{value}/EX/{seconds}. Without this fix every brief persists indefinitely and Redis storage grows unboundedly across weekly runs. P2 — seed-meta not written when all regions skipped writeExtraKeyWithMeta was gated on generated > 0. If every region was skipped (no snapshot yet, or LLM failed), seed-meta was never written, making the seeder indistinguishable from "never ran" in health tooling. Now writes seed-meta whenever failed === 0, carrying regionsSkipped count. P2 #3 (validate gate) — already fixed in previous commit (parseBriefJson now requires situation_recap for valid=true). * fix(intelligence): register regional-briefs in health.js SEED_META + STANDALONE_KEYS (review P2 on #2989) * fix(intelligence): register regional-briefs in api/seed-health.js (review P2 on #2989) * fix(intelligence): raise brief TTL to 15 days to cover missed weekly cycle (review P2 on #2989) * fix(intelligence): distinguish missing-key from Redis-error + coverage-gated health (review P2s on #2989) P2 #1 — false upstreamUnavailable before first seed getCachedJson returns null for both "key missing" and "Redis failed", so the handler was advertising an outage for every region before the first weekly seed ran. Switched to getRawJson (throws on Redis errors) so null = genuinely missing key → clean empty 200, and thrown error = upstream failure → upstreamUnavailable: true for gateway no-store. P2 #2 — partial run hides coverage loss in health The seed-meta was written with generated count even if only 1 of 7 regions produced a brief. /api/health treats any positive recordCount as healthy, so broad regional failure was invisible to operators. Fix: recordCount is set to 0 when generated < ceil(expectedRegions/2). This makes /api/health report EMPTY_DATA for severely partial runs while still writing seed-meta (so the seeder is confirmed to have run). coverageOk flag in the summary payload lets operators drill into the exact coverage state. * fix(intelligence): tighten coverage gate to expectedRegions-1 (review P2 on #2989)
Summary
Phase 3 PR1 of the Regional Intelligence Model. Adds an append-only regime transition log per region plus a premium-gated RPC to read it. Server-only; no UI changes.
What landed
New writer module: `scripts/regional-snapshot/regime-history.mjs`
Single public entry point:
```js
recordRegimeTransition(region, snapshot, diff, opts?)
-> { recorded, entry, pushed, trimmed }
```
Pure builder + Redis-ops orchestrator + dependency-injected publisher.
Flow:
LTRIM failure is non-fatal — entry already landed, next cycle will re-trim. LPUSH failure short-circuits and reports `pushed=false`. The recorder NEVER throws and is wrapped in its own try/catch in the seed loop so snapshot persist is never blocked.
`seed-regional-snapshots.mjs` hook
Added a regime-history call alongside the existing alert-emitter call, right after `persistSnapshot` success. Same best-effort contract: unconditional try/catch, log warn on throw, continue main loop.
Proto + RPC: `GetRegimeHistory`
New `proto/worldmonitor/intelligence/v1/get_regime_history.proto`:
`region_id` validated as strict kebab-case (same regex as `get-regional-snapshot`). `limit` capped server-side at `MAX_LIMIT=100`, defaulting to 50 when omitted. Generated openapi JSON/YAML committed via `make generate`.
Server handler
`server/worldmonitor/intelligence/v1/get-regime-history.ts` — LRANGE-based read (newest-first because the writer LPUSHes). `adaptTransition(raw)` is exported for testability. LRANGE helper is inlined here because `server/_shared/redis.ts` has no list helpers yet — first list-reading handler in the intelligence service. Empty list / Redis miss / failed JSON parse all return `{ transitions: [] }` so the client distinguishes "never changed" from "upstream broken" via HTTP status, not body. Registered in `handler.ts`.
Premium gating + cache tier
Testing
44 new unit tests across 2 files:
`tests/regional-snapshot-regime-history.test.mjs` (22 tests):
`tests/get-regime-history.test.mts` (22 tests):
Verification:
Post-Deploy Monitoring & Validation
`curl -H "X-WorldMonitor-Key: $KEY" "https://worldmonitor.app/api/intelligence/v1/get-regime-history?region_id=mena&limit=10" | jq .`
Deferred to future iterations
PR sequence