feat(energy): Energy Atlas end-to-end — pipelines + storage + shortages + disruptions + country drill-down#3294
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Day 6 of the Energy Atlas Release 1 plan (Week 2). First curated asset
registry for the atlas — the real gap vs GEF.
## Curated data (critical assets only, not global completeness)
scripts/data/pipelines-gas.json — 12 critical gas lines:
Nord Stream 1/2 (offline; Swedish EEZ sabotage 2022; EU sanctions refs),
TurkStream, Yamal–Europe (offline; Polish counter-sanctions),
Brotherhood/Soyuz (offline; Ukraine transit expired 2024-12-31),
Power of Siberia, Dolphin, Medgaz, TAP, TANAP,
Central Asia–China, Langeled.
scripts/data/pipelines-oil.json — 12 critical oil lines:
Druzhba North/South (N offline per EU 2022/879; S under landlocked
derogation), CPC, ESPO (+ price-cap sanction ref), BTC, TAPS,
Habshan–Fujairah (Hormuz bypass), Keystone, Kirkuk–Ceyhan (offline
since 2023 ICC ruling), Baku–Supsa, Trans-Mountain (TMX expansion
May 2024), ESPO spur to Daqing.
Scope note: 75+ each is Week 2b work via GEM bulk import. Today's cut
is curated from first-hand operator disclosures + regulator filings so
I can stand behind every evidence field.
## Evidence-based schema (not conclusion labels)
Per docs/methodology/pipelines.mdx: no bare `sanctions_blocked` field.
Every pipeline carries an evidence bundle with `physicalState`,
`physicalStateSource`, `operatorStatement`, `commercialState`,
`sanctionRefs[]`, `lastEvidenceUpdate`, `classifierVersion`,
`classifierConfidence`. The public badge (`flowing|reduced|offline|
disputed`) is derived server-side from this bundle at read time.
## Seeder
scripts/seed-pipelines.mjs — single process publishes BOTH keys
(energy:pipelines:{gas,oil}:v1) via two runSeed() calls. Tiny datasets
(<20KB each) so co-location is cheap and guarantees classifierVersion
consistency.
Conventions followed (worldmonitor-bootstrap-registration skill):
- TTL 21d = 3× weekly cadence (gold-standard per
feedback_seeder_gold_standard.md)
- maxStaleMin 20_160 = 2× cadence (health-maxstalemin-write-cadence skill)
- sourceVersion + schemaVersion + recordCount + declareRecords wired
(seed-contract-foundation)
- Zero-case explicitly NOT allowed — MIN_PIPELINES_PER_REGISTRY=8 floor
## Health registration (dual, per feedback_two_health_endpoints_must_match)
- api/health.js: BOOTSTRAP_KEYS adds pipelinesGas + pipelinesOil;
SEED_META adds both with maxStaleMin=20_160.
- api/seed-health.js: mirror entries with intervalMin=10_080 (maxStaleMin/2).
## Bundle registration
scripts/seed-bundle-energy-sources.mjs adds a single Pipelines entry
(not two) because seed-pipelines.mjs publishes both keys in one run —
listing oil separately would double-execute. Monitoring of the oil key
staleness happens in api/health.js instead.
## Tests (tests/pipelines-registry.test.mts)
17 passing node:test assertions covering:
- Schema validation (both registries pass validateRegistry)
- Identity resolution (no id collisions, id matches object key)
- Country ISO2 normalization (from/to/transit all match /^[A-Z]{2}$/)
- Endpoint geometry within Earth bounds
- Evidence rigor: non-flowing badges require at least one supporting
evidence source (operator statement / sanctionRefs / ais-relay /
satellite / press)
- ClassifierConfidence in 0..1
- Commodity/capacity pairing (gas uses capacityBcmYr, oil uses
capacityMbd — mixing = test fail)
- validateRegistry rejects: empty object, null, no-evidence fixtures,
below-floor counts
Typecheck clean (both tsconfig.json and tsconfig.api.json).
Next: Day 7 will add list-pipelines / get-pipeline-detail RPCs in
supply-chain/v1. Day 8 ships PipelineStatusPanel with DeckGL PathLayer
consuming the registry.
… hard-exits
High finding from PR review. scripts/seed-pipelines.mjs called runSeed()
twice in one process and awaited Promise.all. But runSeed() in
scripts/_seed-utils.mjs hard-exits via process.exit on ~9 terminal paths
(lines 816, 820, 839, 888, 917, 989, plus fetch-retry 946, fatal 859,
skipped-lock 81). The first runSeed to reach any terminal path exits the
entire node process, so the second runSeed's resolve never fires — only
one of energy:pipelines:{gas,oil}:v1 would ever be written.
Since the bundle scheduled seed-pipelines.mjs exactly once, and both
api/health.js and api/seed-health.js expect both keys populated, the
other registry would stay permanently EMPTY/STALE after deploy.
Fix: split into two entry-point scripts around a shared utility.
- scripts/_pipeline-registry.mjs (NEW, was seed-pipelines.mjs) — shared
helpers ONLY. Exports GAS_CANONICAL_KEY, OIL_CANONICAL_KEY,
PIPELINES_TTL_SECONDS, MAX_STALE_MIN, buildGasPayload, buildOilPayload,
validateRegistry, recordCount, declareRecords. Underscore prefix marks
it as non-entry-point (matches _seed-utils.mjs / _seed-envelope-source.mjs
convention).
- scripts/seed-pipelines-gas.mjs (NEW) — imports from the shared module,
single runSeed('energy','pipelines-gas',…) call.
- scripts/seed-pipelines-oil.mjs (NEW) — same shape, oil.
- scripts/seed-bundle-energy-sources.mjs — register BOTH seeders (not one).
- scripts/seed-pipelines.mjs — deleted.
- tests/pipelines-registry.test.mts — update import path to the shared
module. All 17 tests still pass.
Typecheck clean (both configs). Tests pass. No other consumers import
from the deleted script.
…klist High finding from PR review. My earlier PR description claimed worldmonitor-bootstrap-registration was complete, but I only touched two of the four registries (api/health.js + api/seed-health.js). The bootstrap hydration payload itself (api/bootstrap.js) and the shared cache-keys registry (server/_shared/cache-keys.ts) still had no entry for either pipeline key, so any consumer that reads bootstrap data would see pipelinesGas/pipelinesOil as missing on first load. Files updated this commit: - api/bootstrap.js — KEYS map + SLOW_KEYS set both gain pipelinesGas + pipelinesOil. Placed next to sprPolicies (same curated-registry cadence and tier). Slow tier is correct: weekly cron, not needed on first paint. - server/_shared/cache-keys.ts — PIPELINES_GAS_KEY + PIPELINES_OIL_KEY exported constants (matches SPR_POLICIES_KEY pattern), BOOTSTRAP_KEYS map entries, and BOOTSTRAP_TIERS entries (both 'slow'). Not touched (intentional): - server/gateway.ts — pipeline data is free-tier per the Energy Atlas plan; no PREMIUM_RPC_PATHS entry required. Energy Atlas monetization hooks (scenario runner, MCP tools, subscriptions) are Release 2. Full 4-file checklist now complete: ✅ server/_shared/cache-keys.ts (this commit) ✅ api/bootstrap.js (this commit) ✅ api/health.js (earlier in PR) ✅ api/seed-health.js (earlier in PR — dual-registry rule) Typecheck clean (both configs).
def96a7 to
6b01fa5
Compare
…rived badges
Day 7 of the Energy Atlas Release 1 plan (Week 2). Exposes the pipeline
registries (shipped in Day 6) via two supply-chain RPCs and ships the
evidence-to-badge derivation server-side.
## Proto
proto/worldmonitor/supply_chain/v1/list_pipelines.proto — new:
- ListPipelinesRequest { commodity_type?: 'gas' | 'oil' }
- ListPipelinesResponse { pipelines[], fetched_at, classifier_version, upstream_unavailable }
- GetPipelineDetailRequest { pipeline_id (required, query-param) }
- GetPipelineDetailResponse { pipeline?, revisions[], fetched_at, unavailable }
- PipelineEntry — wire shape mirroring scripts/data/pipelines-{gas,oil}.json
+ a server-derived public_badge field
- PipelineEvidence, OperatorStatement, SanctionRef, LatLon, PipelineRevisionEntry
service.proto adds both rpc methods with HTTP_METHOD_GET + path bindings:
/api/supply-chain/v1/list-pipelines
/api/supply-chain/v1/get-pipeline-detail
`make generate` regenerated src/generated/{client,server}/… + docs/api/
OpenAPI json/yaml.
## Evidence-derivation
server/worldmonitor/supply-chain/v1/_pipeline-evidence.ts — new.
derivePublicBadge(evidence) → 'flowing' | 'reduced' | 'offline' | 'disputed'
is deterministic + versioned (DERIVER_VERSION='badge-deriver-v1').
Rules (first match wins):
1. offline + sanctionRef OR expired/suspended commercial → offline
2. offline + operator statement → offline
3. offline + only press/ais/satellite → disputed (single-source negative claim)
4. reduced → reduced
5. flowing → flowing
6. unknown / malformed → disputed
Staleness guard: non-flowing badges on >14d-old evidence demote to
disputed. Flowing is the optimistic default — stale "still flowing" is
safer than stale "offline". Matches seed-pipelines-{gas,oil}.mjs maxStaleMin.
Tests (tests/pipeline-evidence-derivation.test.mts) — 15 passing cases
covering happy paths, disputed fallbacks, staleness guard, versioning.
## Handlers
server/worldmonitor/supply-chain/v1/list-pipelines.ts
- Reads energy:pipelines:{gas,oil}:v1 via getCachedJson.
- projectPipeline() narrows the Upstash `unknown` into PipelineEntry
shape + calls derivePublicBadge.
- Honors commodity_type filter (skip the opposite registry's Redis read
when the client pre-filters).
- Returns upstream_unavailable=true when BOTH registries miss.
server/worldmonitor/supply-chain/v1/get-pipeline-detail.ts
- Scans both registries by id (ids are globally unique per
tests/pipelines-registry.test.mts).
- Empty revisions[] for now; auto-revision log wires up in Week 3.
handler.ts registers both into supplyChainHandler.
## Gateway
server/gateway.ts adds 'static' cache-tier for both new RPC paths
(registry is slow-moving; 'static' matches the other read-mostly
supply-chain endpoints).
## Consumer wiring
Not in this commit — PipelineStatusPanel (Day 8) is what will call
listPipelines/getPipelineDetail via the generated client. pipelinesGas
+ pipelinesOil stay in PENDING_CONSUMERS until Day 8.
Typecheck clean (both configs). 15 new tests + 17 registry tests all pass.
…awer
Day 8 of the Energy Atlas Release 1 plan. First consumer of the Day 6–7
registries + RPCs.
## What this PR adds
- src/components/PipelineStatusPanel.ts — new panel (id=pipeline-status).
* Bootstrap-hydrates from pipelinesGas + pipelinesOil for instant first
paint; falls through to listPipelines() RPC if bootstrap misses.
Background re-fetch runs on every render so a classifier-version bump
between bootstrap stamp and first view produces a visible update.
* Table rows sorted non-flowing-first (offline / reduced / disputed
before flowing) — what an atlas reader cares about.
* Click-to-expand drawer calls getPipelineDetail() lazily — operator
statements, sanction refs (with clickable source URLs), commercial
state, classifier version + confidence %, capacity + route metadata.
* publicBadge color-chip palette matches the methodology doc.
* Attribution footer with GEM (CC-BY 4.0) credit + classifier version.
- src/components/index.ts — barrel export.
- src/app/panel-layout.ts — import + createPanel('pipeline-status', …).
- src/config/panels.ts — ENERGY_PANELS adds 'pipeline-status' at priority 1.
## PENDING_CONSUMERS cleanup
tests/bootstrap.test.mjs — removes 'pipelinesGas' + 'pipelinesOil' from
the allowlist. The invariant "every bootstrap key has a getHydratedData
consumer" now enforces real wiring for these keys: the panel literally
calls getHydratedData('pipelinesGas') and getHydratedData('pipelinesOil').
Future regressions that remove the consumer will fail pre-push.
## Consumer contract verified
- 67 tests pass including bootstrap.test.mjs consumer coverage check.
- Typecheck clean.
- No DeckGL PathLayer in this commit — existing 'pipelines-layer' has a
separate data source, so modifying DeckGLMap.ts to overlay evidence-
derived badges on the map is a follow-up commit to avoid clobbering.
## Out of scope for Day 8 (next steps on same PR)
- DeckGL PathLayer integration (color pipelines on the main map by
publicBadge, click-to-open this drawer) — Day 8b commit.
- Storage facility registry + StorageFacilityMapPanel — Days 9-10.
…erivation
High finding from PR review. The Day-8 panel crashed on first paint
whenever bootstrap hydration succeeded, because:
- Bootstrap hydrates raw scripts/data/pipelines-{gas,oil}.json verbatim.
- That JSON does NOT include publicBadge — that field is only added by
the server handler's projectPipeline() in list-pipelines.ts.
- PipelineStatusPanel passed raw entries into badgeChip(), which called
badgeLabel(undefined).charAt(0) → TypeError.
The background RPC refresh that would have repaired the data never ran
because the panel threw before reaching it. So the exact bootstrap path
newly wired in commit 6b01fa5 was broken for the new panel.
Fix: move the evidence→badge deriver to src/shared/pipeline-evidence.ts
so the client panel and the server handler run the identical function on
identical inputs. Panel projects raw bootstrap JSON through the shared
deriver client-side, producing the same publicBadge the RPC would have
returned. No UI flicker on hydration because pre- and post-RPC badges
match exactly (same function, same input).
## Changes
- src/shared/pipeline-evidence.ts (NEW) — pure deriver with duck-typed
PipelineEvidenceInput (no generated-type dependency, so both client
and server assign their proto-typed evidence bundles by structural
subtyping). Exports derivePipelinePublicBadge + version + type.
- server/worldmonitor/supply-chain/v1/_pipeline-evidence.ts — now a thin
re-export of the shared module under its older name so in-handler
imports keep working without a sweep.
- src/components/PipelineStatusPanel.ts:
* Imports derivePipelinePublicBadge from @/shared/pipeline-evidence.
* NEW projectRawPipeline() defensively coerces every field from
unknown → PipelineEntry shape, mirroring the server projection.
* buildBootstrapResponse now routes every raw entry through the
projection before returning, so the wire-format PipelineEntry[] the
renderer receives always has publicBadge populated.
* badgeChip() gained a null-guard fallback to 'disputed' — belt +
braces so even if a future caller passes an undefined, the UI
renders safely instead of throwing.
* BootstrapRegistry renamed RawBootstrapRegistry with a comment
explaining why the seeder ships raw JSON (not wire format).
## Regression tests
tests/pipeline-panel-bootstrap.test.mts (NEW) — 6 tests that exercise
the bootstrap-first-paint path end-to-end:
- Every gas + oil curated entry produces a valid badge.
- Raw entries never ship with pre-computed publicBadge (contract guard
on the seed data format).
- Deriver never throws on undefined/null/{} evidence (was the crash).
- Nord Stream 1 regression check (offline + paperwork → offline).
- Druzhba-South staleness behavior (reduced when fresh, disputed after
60 days without update).
38/38 tests now pass (17 registry + 15 deriver + 6 new bootstrap-path).
Typecheck clean on both configs.
## Invariant preserved
The server handler and the panel render identical badges because:
1. Same pure function (imported from the same module).
2. Same deterministic rules, same staleness window.
3. Same bootstrap data read by both paths (Redis → either bootstrap
payload or RPC response).
No UI flicker on hydration.
## P2-1 — sanitizeUrl on external evidence links (XSS hardening) Sanction-ref URLs and operator-statement URLs were interpolated with escapeHtml only. HTML-escaping blocks tag injection but NOT javascript: or data: URL schemes, so a bad URL in the seeded registry would execute in-app when a reader clicked the evidence link. Every other panel in the codebase (NewsPanel, GdeltIntelPanel, GeoHubsPanel, AirlineIntelPanel, MonitorPanel) uses sanitizeUrl for this exact reason. Fix: import sanitizeUrl from @/utils/sanitize and route both hrefs through it. sanitizeUrl() drops non-http(s) schemes + returns '' on invalid URLs. The renderer now suppresses the <a> entirely when sanitize rejects — the date label still renders as plain text instead of becoming an executable link. ## P2-2 — loadDetail catch path missing stale-response guard The success path at loadDetail() checked `this.selectedId !== pipelineId` to suppress stale responses when the user has clicked another pipeline mid-flight. The catch path at line 219 had no such guard: if the user clicked A, then B, and A's request failed before B resolved, A's error handler cleared detailLoading and detail, showing "Pipeline detail unavailable" for B's drawer even though B was still loading. Fix: mirror the same `if (this.selectedId !== pipelineId) return` guard in the catch path. The newer request now owns the drawer state regardless of which path (success OR failure) the older one took. ## P2-3 — always-gas-preference aggregator for classifierVersion + fetchedAt Three call sites (list-pipelines.ts handler, get-pipeline-detail.ts handler, PipelineStatusPanel bootstrap projection) computed aggregate classifier version and fetchedAt by `gas?.x || oil?.x || fallback`. That was defensible when a single seed-pipelines.mjs wrote both keys atomically (fix commit 29b4ac7 split this into two separate Railway cron entry points). Now gas + oil cron independently, so mixed-version (gas=v1, oil=v2 during classifier rollout) and mixed-timestamp (oil refreshed 6h after gas) windows are the EXPECTED state, not the exceptional one. The comment in list-pipelines.ts even said "pick the newest classifier version" but the code didn't actually compare. Fix: add two shared helpers in src/shared/pipeline-evidence.ts — - pickNewerClassifierVersion(a,b) — parses /^v(\\d+)$/ and returns the higher-numbered version; falls back to lexicographic for non-v- prefixed values; handles single-missing inputs. - pickNewerIsoTimestamp(a,b) — Date.parse()-compares and returns the later ISO; handles missing / malformed inputs gracefully. Both server RPCs and the panel bootstrap projection now call these helpers identically, so clients are told the truth about version + freshness during partial rollouts. ## Tests Extended tests/pipeline-evidence-derivation.test.mts with 8 new assertions covering both pickers: - Higher v-number wins regardless of order (v1 vs v2 → v2 both ways) - Single-missing falls back to the one present - Missing + missing → default 'v1' for version / '' for ts - Non-v-numbered values fall back to lexicographic - Explicit regression: "gas=v1 + oil=v2 during rollout" returns v2 - Explicit regression: "oil fresher than gas" returns the oil timestamp 38 → 46 tests. All pass. Typecheck clean on both configs.
…p↔panel link Day 8b of the Energy Atlas plan. Pipelines now render on the main DeckGL map of the energy variant colored by their derived publicBadge, and clicking a pipeline on the map opens the same evidence drawer the panel row-click opens. ## Why this commit Day 8 shipped the PipelineStatusPanel as a table + drawer view. Reviewer flag notwithstanding (fixed in 149d33e + db52965), a table-only pipeline view is a weak product compared to the map-centric atlas it's meant to rival. The map-layer differentiation is the whole point of the feature. ## What this adds src/components/DeckGLMap.ts: - New createEnergyPipelinesLayer() — reads hydrated pipeline registries via getHydratedData, projects raw JSON through the shared deriver (src/shared/pipeline-evidence.ts), renders a DeckGL PathLayer colored by publicBadge: flowing → green (46,204,113) reduced → amber (243,156,18) offline → red (231,76,60) disputed → purple (155,89,182) Offline + disputed get thicker strokes (3px vs 2px) for at-a-glance surfacing of disrupted assets. Geometry comes from raw startPoint + waypoints[] + endPoint per asset (straight line when no waypoints). - Branching at line ~1498: SITE_VARIANT === 'energy' routes to the new method; other variants keep the static PIPELINES config (colored by oil/gas type). Existing commodity/finance/full map layers are untouched — no cross-variant leakage. - onClick handler emits `energy:open-pipeline-detail` as a window CustomEvent with { pipelineId }. Loose coupling: the map doesn't import the panel, the panel doesn't import the map. - Fallback: if bootstrap hasn't hydrated yet, createEnergyPipelinesLayer falls back to the static createPipelinesLayer() so the pipelines toggle always shows *something*. src/components/PipelineStatusPanel.ts: - Constructor registers a window event listener for 'energy:open-pipeline-detail' → calls this.loadDetail(pipelineId) → drawer opens on the clicked asset. Map click and row click converge on the same drawer, same evidence view. - destroy() removes the listener to prevent ghost handlers after panel unmount. ## Guarantees - Bootstrap parity: the DeckGL layer calls the SAME derivePipelinePublicBadge as the panel and the server handler, so the map color, the table row chip, and the RPC response all agree on the badge. No flicker, no drift, no confused user. - Variant isolation: only SITE_VARIANT === 'energy' triggers the new path. Commodity / finance / full map layers untouched. - No cross-component import: the panel doesn't reference the map class and vice versa. The event contract is the only coupling — testable, swappable, tauri-safe (guarded with `typeof window !== 'undefined'`). Typecheck clean. PR #3294 now has 8 commits. Follow-up backlog: - Add waypoints[] to the curated pipelines-{gas,oil}.json so the map draws real routes instead of straight lines (cosmetic; does not affect correctness). - Tooltip case in the picking tooltip registry (line ~3748) so hover shows "Nord Stream 1 · OFFLINE" before click.
## P1 — getHydratedData single-use race between map + panel
src/services/bootstrap.ts:34 — `if (val !== undefined) hydrationCache.delete(key);`
The helper drains its slot on first read. Day 8 (PipelineStatusPanel) and
Day 8b (createEnergyPipelinesLayer) BOTH call getHydratedData('pipelinesGas')
and getHydratedData('pipelinesOil') — whoever renders first drains the cache
and forces the loser onto its fallback path (panel → RPC, map → static
PIPELINES layer). The commit's "shared bootstrap-backed data" guarantee
did not actually hold.
Fix: new src/shared/pipeline-registry-store.ts that reads once and memoizes.
Both consumers read through getCachedPipelineRegistries() — same data, same
reference, unlimited re-reads. When the panel's background RPC fetch lands,
it calls setCachedPipelineRegistries() to back-propagate fresh data into
the store so the map's next re-render sees the newer classifierVersion +
fetchedAt too (no map/panel drift during classifier rollouts).
Test-only injection hook (__setBootstrapReaderForTests) makes the drain-once
semantics observable without a real bootstrap payload.
## P2 — pipelines-layer tooltip regresses to blank label on energy variant
src/components/DeckGLMap.ts:3748 (pipelines-layer tooltip case) still assumed
the static-config shape (obj.type). The new energy layer emits objects with
commodityType + badge fields, so the tooltip's type-ternary fell through to
the generic fallback — hover rendered " pipeline" (empty leading commodity)
instead of "Nord Stream 1 · OFFLINE".
Fix: differentiate by presence of obj.badge (only the energy layer sets it).
On the energy variant, tooltip now reads name + commodity + badge. Static-
config variants (commodity / finance / full) keep their existing format
unchanged.
## P2 — createEnergyPipelinesLayer dropped highlightedAssets behavior
The static createPipelinesLayer() reads this.highlightedAssets.pipeline and
threads it into getColor / getWidth with an updateTrigger on the signature.
Any caller using flashAssets('pipeline', [...]) or highlightAssets([...])
gets a visible red-outline flash on the matching paths. My Day 8b energy
layer ignored the set entirely — those APIs silently no-op'd on the energy
variant.
Fix: createEnergyPipelinesLayer() now reads the same highlight set, applies
HIGHLIGHT_COLOR + wider stroke to matching IDs, and wires
updateTriggers: { getColor: sig, getWidth: sig } so DeckGL actually
recomputes when the set changes.
Also removed the unnecessary layerCache.set() in the energy path: the
store can update via RPC back-propagation, and a cache keyed only on
highlight-signature would serve stale data. With ~25 critical-asset
pipelines, rebuild per render is trivial.
## Tests
tests/pipeline-registry-store.test.mts (NEW) — 5 tests covering the
drain-once read-many invariant: multiple consumers get cached data
without re-draining, RPC back-propagation updates the source, partial
updates preserve the other commodity, and pure RPC-first (no bootstrap)
works without invoking the reader.
All 51 PR tests pass. Typecheck clean on both configs.
…ude hubs) Ships 21 critical strategic storage facilities as a curated registry, same evidence-bundle pattern as the pipeline registries in Day 7/8: - scripts/data/storage-facilities.json — 4 UGS + 4 SPR + 6 LNG export + 3 LNG import + 4 crude tank farms. Each carries physicalState + sanctionRefs + classifierVersion/Confidence + fillDisclosed/fillSource. - scripts/_storage-facility-registry.mjs — shared helpers (validator, builder, canonical key, MAX_STALE_MIN). Validator enforces facility-type × capacity-unit pairing (ugs→TWh, spr/tank-farm→Mb, LNG→Mtpa) and the non-operational badge ⇒ evidence invariant. - scripts/seed-storage-facilities.mjs — single runSeed entry (only one key, so no split-seeder dance needed). - Registered in the 4-file bootstrap checklist: cache-keys.ts (STORAGE_FACILITIES_KEY + BOOTSTRAP_CACHE_KEYS + BOOTSTRAP_TIERS), api/bootstrap.js (KEYS + SLOW_KEYS), api/health.js (BOOTSTRAP_KEYS + SEED_META, 14d threshold = 2× weekly cron), api/seed-health.js (mirror). - tests/bootstrap.test.mjs PENDING_CONSUMERS adds storageFacilities — Day 10 StorageFacilityMapPanel will remove it. - tests/storage-facilities-registry.test.mts — 20 tests covering schema, identity, geometry, type×capacity pairing, evidence contract, and negative-input validator rejection. Registry fields are slow-moving; badge derivation happens at read-time server-side once the RPC handler lands in Day 10 (panel + deckGL ScatterplotLayer). Seeded data is live in Redis from this commit so the Day 10 PR only adds display surfaces. Tests: 56 pass (36 prior + 20 new). Typecheck + typecheck:api clean.
…kGL ScatterplotLayer + panel) End-to-end wiring for the strategic storage registry seeded in Day 9. Same pattern as the pipeline shipping path (Days 7+8+8b): proto → handler → shared evidence deriver → panel → DeckGL map layer, with a shared read-once store keeping map + panel aligned. Proto + generated code: - list_storage_facilities.proto: ListStorageFacilities + GetStorageFacilityDetail messages with StorageFacilityEntry, StorageEvidence, StorageSanctionRef, StorageOperatorStatement, StorageLatLon, StorageFacilityRevisionEntry. - service.proto wires both RPCs under /api/supply-chain/v1. - make generate → regenerated client + server stubs + OpenAPI. Server handlers: - src/shared/storage-evidence.ts: shared pure deriver. Duck-typed input interface avoids generated-type deps; identical rules to the pipeline deriver (sanction/commercial paperwork vs external-signal-only offline, 14d staleness window, version pin). - _storage-evidence.ts: thin re-export for server handler import ergonomics. - list-storage-facilities.ts: reads STORAGE_FACILITIES_KEY from Upstash, projects raw → wire format, attaches derived publicBadge, filters by optional facilityType query arg. - get-storage-facility-detail.ts: single-asset lookup for drawer. - handler.ts registers both new methods. - gateway.ts: both routes → 'static' cache tier (registry is near-static). Panel + map: - src/shared/storage-facility-registry-store.ts: drain-once memo mirroring pipeline-registry-store. Both panel and DeckGL layer read through this so the single-use getHydratedData drain doesn't race between consumers. RPC back-propagation via setCachedStorageFacilityRegistry() keeps map ↔ panel on the same classifierVersion during rollouts. - StorageFacilityMapPanel.ts: table + evidence drawer. Bootstrap hot path projects raw registry through same deriver as server so first-paint badge matches post-RPC badge (no flicker). sanitizeUrl + stale-response guards (success + catch paths) carried over from PipelineStatusPanel. - DeckGLMap.ts createEnergyStorageLayer(): ScatterplotLayer keyed on badge color; log-scale radius (6km–26km) keeps Rehden visible next to Ras Laffan. Click dispatches 'energy:open-storage-facility-detail' — panel listens and opens its drawer (loose coupling, no direct refs). - Tooltip branch on storage-facilities-layer shows facility type, country, capacity unit, and badge. - Added 'storageFacilities' optional field to MapLayers type (optional so existing variant literals across commodity/finance/tech/happy/full/etc. don't need touching). Wired into LAYER_REGISTRY + VARIANT_LAYER_ORDER.energy + ENERGY_MAP_LAYERS + ENERGY_MOBILE_MAP_LAYERS. Panel entry added to ENERGY_PANELS + panel-layout createPanel. PENDING_CONSUMERS entry from Day 9 removed — panel + map layer are now real consumers. Tests: - storage-evidence-derivation.test.mts (17 tests): covers every curated facility yields a valid badge, null/malformed input never throws, offline sanction/commercial/operator rules, external-signal-only offline → disputed, staleness demotion. - storage-facility-registry-store.test.mts (4 tests): drain-once, no-data drain, RPC update, pure-RPC-first path. All 6,426 unit tests pass. Typecheck + typecheck:api clean. Pre-existing src-tauri/sidecar/ test failure is unrelated (no diff touches src-tauri/).
…assifier post-launch) Ships v1 of the global fuel-shortage alert registry. Severity is the CLASSIFIER OUTPUT (confirmed/watch), not a client derivation — we ship the evidence alongside so readers can audit the grounds. v1 is seeded from curated JSON; post-launch the proactive-intelligence classifier (Day 12 work) extends the same key directly. Data: - scripts/data/fuel-shortages.json — 15 known active shortages (PK, LK, NG×2, CU, VE, LB, ZW, AR, IR, BO, KE, PA, EG, BY) spanning petrol/diesel/jet across confirmed + watch tiers. Each entry carries evidenceSources[] (regulator/operator/press), firstSeen, lastConfirmed, resolvedAt, impactTypes[], causeChain[], classifier version + confidence. Confirmed severity enforces authoritative evidence at schema level. Seeder: - scripts/_fuel-shortage-registry.mjs — shared validator (enforces iso2 country, enum products/severities/impacts/causes, authoritative evidence for confirmed). MIN_SHORTAGES=10. - scripts/seed-fuel-shortages.mjs — single runSeed entry. - Registered in seed-bundle-energy-sources.mjs at DAY cadence (shortages move faster than registry assets). Bootstrap 4-file registration: - cache-keys.ts: FUEL_SHORTAGES_KEY + BOOTSTRAP_CACHE_KEYS + BOOTSTRAP_TIERS. - api/bootstrap.js: KEYS + SLOW_KEYS. - api/health.js: BOOTSTRAP_KEYS + SEED_META (2880min = 2× daily cron). - api/seed-health.js: mirrors intervalMin=1440. Proto + RPC: - list_fuel_shortages.proto: ListFuelShortages (country/product/severity query facets) + GetFuelShortageDetail messages with FuelShortageEntry, FuelShortageEvidence, FuelShortageEvidenceSource. - service.proto wires both new RPCs under /api/supply-chain/v1. - list-fuel-shortages.ts handler projects raw → wire format, supports server-side country/product/severity filtering. - get-fuel-shortage-detail.ts single-shortage lookup. - handler.ts registers both. gateway.ts: 'medium' cache-tier (daily classifier updates warrant moderate freshness). Shared evidence helper: - src/shared/shortage-evidence.ts: deriveShortageEvidenceQuality maps (confidence + authoritative-source count + freshness) → 'strong' | 'moderate' | 'thin' for client-side sort/trust indicators. Does NOT change severity — classifier owns that decision. - countEvidenceSources buckets sources for the drawer's "n regulator / m press" line. Tests: - tests/fuel-shortages-registry.test.mts (19 tests): schema, identity, enum coverage, evidence contract (confirmed → authoritative source), validateRegistry negative cases. - tests/shortage-evidence.test.mts (10 tests): quality deriver edge cases, source bucketing. - tests/bootstrap.test.mjs PENDING_CONSUMERS adds fuelShortages — FuelShortagePanel arrives Day 12 which will remove the entry. Typecheck + typecheck:api clean. 64 tests pass.
End-to-end wiring of the fuel-shortage registry shipped in Day 11: panel on the Energy variant page, ScatterplotLayer pins on the DeckGL map, both reading through a shared single-drain store so they don't race on the bootstrap cache. Panel: - src/components/FuelShortagePanel.ts — table sorted by severity (confirmed first) then evidence quality (strong → thin) then most-recent lastConfirmed. Drawer shows short description, first-seen / last-confirmed / resolved, impact types, cause chain, classifier version/confidence, and a typed evidence-source list with regulator/operator/press chips. sanitizeUrl on every href so classifier-ingested URLs can't render as javascript:. Same stale-response guards on success + catch paths as the other detail drawers. - Consumes deriveShortageEvidenceQuality for client-side trust indicator (three-dot ●●● / ●●○ / ●○○), NOT for severity — severity is classifier output. - Registered in ENERGY_PANELS + panel-layout.ts + components barrel. Shared store: - src/shared/fuel-shortage-registry-store.ts — same drain-once memoize pattern as pipeline- and storage-facility-registry-store. Both the panel and the DeckGL shortage-pins layer read through it. DeckGL layer: - DeckGLMap.createEnergyShortagePinsLayer: ScatterplotLayer placing one pin per active shortage at the country centroid (via getCountryCentroid from services/country-geometry). Stacking offset (~0.8° lon) when multiple shortages share a country so Nigeria's petrol + diesel don't render as a single dot. Confirmed pins 55km radius; watch 38km. Click dispatches 'energy:open-fuel-shortage-detail' — panel listens. - Tooltip branch on fuel-shortages-layer: country · product · short description · severity. - Layer registered in LAYER_REGISTRY, VARIANT_LAYER_ORDER.energy, ENERGY_MAP_LAYERS, ENERGY_MOBILE_MAP_LAYERS. MapLayers.fuelShortages is optional on the type so other variants' literals remain valid. Tests: - tests/fuel-shortage-registry-store.test.mts (4 tests): drain-once, no-data, RPC back-prop, pure-RPC-first path. - tests/bootstrap.test.mjs — fuelShortages removed from PENDING_CONSUMERS. Typecheck + typecheck:api clean. 39 tests pass (plus full suite in pre-push).
…rawer
Ships the energy:disruptions:v1 registry that threads together pipelines
and storage facilities: state transitions (sabotage, sanction, maintenance,
mechanical, weather, commercial, war) keyed by assetId so any asset's
drawer can render its history without a second registry lookup.
Data + seeder:
- scripts/data/energy-disruptions.json — 12 curated events spanning
Nord Stream 1/2 sabotage, Druzhba sanctions, CPC force majeure,
TurkStream maintenance, Yamal halt, Rehden trusteeship, Arctic LNG 2
sanction, ESPO drone strikes, BTC fire (historical), Sabine Pass
Hurricane Beryl, Power of Siberia ramp. Each event links back to a
seeded asset.
- scripts/_energy-disruption-registry.mjs — validator enforces valid
assetType/eventType/cause enums, http(s) sources, startAt ≤ endAt,
MIN_EVENTS=8.
- scripts/seed-energy-disruptions.mjs — runSeed entry (weekly cron).
- Bundle entry at 7×DAY cadence.
Bootstrap 4-file registration (cache-keys.ts + bootstrap.js + health.js +
seed-health.js) — energyDisruptions in PENDING_CONSUMERS because panel
drawers fetch lazily via RPC on drawer-open rather than hydrating from
bootstrap directly.
Proto + handler:
- list_energy_disruptions.proto: ListEnergyDisruptions with
assetId / assetType / ongoingOnly query facets. Returns events sorted
newest-first.
- list-energy-disruptions.ts projects raw → wire format, supports all
three query facets.
- Registered in handler.ts. gateway.ts: 'medium' cache tier.
Shared timeline helper:
- src/shared/disruption-timeline.ts — pure formatters (formatEventWindow,
formatCapacityOffline, statusForEvent). No generated-type deps so
PipelineStatusPanel + StorageFacilityMapPanel import the same helpers
and render the timeline identically.
Panel integration:
- PipelineStatusPanel.loadDetail now fetches getPipelineDetail +
listEnergyDisruptions({assetId, assetType:'pipeline'}) in parallel.
Drawer gains "Disruption timeline (N)" section with event type, date
window, capacity offline, cause chain, and short description per entry.
- StorageFacilityMapPanel gets identical treatment with assetType='storage'.
- Both reset detailEvents on closeDetail and on fresh click (stale-response
safety).
Tests:
- tests/energy-disruptions-registry.test.mts (17 tests): schema, identity,
enum coverage, evidence, negative inputs.
- tests/bootstrap.test.mjs — energyDisruptions added to PENDING_CONSUMERS.
Typecheck + typecheck:api clean. 51 tests pass locally (plus full suite
in pre-push).
Extends CountryDeepDivePanel's existing "Energy Profile" card with a mini Atlas-exposure section that surfaces per-country exposure to the new registries we shipped in Days 7-13. For each country: - Pipelines touching this country (from, to, or transit) — clickable rows that dispatch 'energy:open-pipeline-detail' so the PipelineStatusPanel drawer opens on the energy variant; no-op on other variants. - Storage facilities in this country — same loose-coupling pattern with 'energy:open-storage-facility-detail'. - Active fuel shortages in this country — severity breakdown line (N confirmed · M watch) plus clickable rows emitting 'energy:open-fuel-shortage-detail'. Silent absence: sections render only when the country has matching assets/events, so countries with no pipeline, storage, or shortage touchpoints see the existing energy-profile card unchanged. Lazy stores: reads go through the same shared drain-once stores (getCachedPipelineRegistries, getCachedStorageFacilityRegistry, getCachedFuelShortageRegistry) so CountryDeepDivePanel does NOT race with Atlas panels over the single-drain bootstrap cache. Dynamic import() keeps the three stores out of the panel's static import graph so non-energy variants can tree-shake them. Typecheck clean. No schema changes; purely additive UI read from already-shipped registries.
Fills the /docs/methodology/disruptions URL referenced by list_energy_disruptions.proto, scripts/_energy-disruption-registry.mjs, and the panel attribution footers. Explains scope (state transitions not daily noise), data shape, what counts as a disruption, classifier evolution path, RPC contract, and ties into the sibling pipeline + storage + shortage methodology pages. No code change; pure docs completion for Week 4 launch polish.
Two handlers (list-storage-facilities + list-pipelines) conflated "empty filter result on a healthy registry" with "upstream unavailable". A caller who queried one facilityType/commodityType and legitimately got zero matches was told the upstream was down — which may push clients to error-state rendering or suppress caching instead of showing a valid empty list. list-storage-facilities.ts — upstreamUnavailable now only fires when `raw` is null (Redis miss). Zero filtered rows on a healthy registry returns upstreamUnavailable: false + empty array. Matches the sibling list-fuel-shortages handler and the wire contract in list_storage_facilities.proto. list-pipelines.ts — same bug, subtler shape. Now checks "requested at least one side AND received nothing" rather than "zero rows after collection". A filter that legitimately matches no gas/oil pipelines on a healthy registry now returns upstreamUnavailable: false. list-energy-disruptions.ts and list-fuel-shortages.ts already had the correct shape (only flag unavailable when raw is missing) — left as-is. Typecheck + typecheck:api clean. No tests added: the existing registry schema tests cover the projection/filter helpers, and the handler-level gating change is documented in code comments for future audits.
Greptile SummaryThis PR delivers the full Energy Atlas feature end-to-end: five Redis registries (gas/oil pipelines, storage facilities, fuel shortages, disruptions), nine RPCs, three DeckGL map layers, three UI panels, and a country drill-down exposure section, all gated behind the energy variant. Architecture is well-considered — the drain-once registry stores solve the single-use bootstrap race, the shared deriver pattern guarantees badge parity between first-paint and post-RPC renders, and the
Confidence Score: 4/5Safe to merge after fixing the resolved-shortage display bug; the upstreamUnavailable inconsistency is low-risk but worth correcting for contract clarity. Two P1 findings (resolved shortages shown as active on map and in country drill-down) mean users could see misleading crisis data. Both are one-liner filter additions. The upstreamUnavailable inconsistency is P2. All other 66 files are clean. src/components/DeckGLMap.ts and src/components/CountryDeepDivePanel.ts (resolved shortage filter), server/worldmonitor/supply-chain/v1/list-energy-disruptions.ts (upstreamUnavailable gate) Important Files Changed
Sequence DiagramsequenceDiagram
participant Seed as Railway Cron Seeder
participant Redis as Upstash Redis
participant BS as api/bootstrap.js
participant Store as Registry Store (memo)
participant Panel as UI Panel
participant Map as DeckGLMap Layer
participant RPC as RPC Handler
Seed->>Redis: SET energy:pipelines:gas:v1 + seed-meta
Seed->>Redis: SET energy:fuel-shortages:v1 + seed-meta
BS->>Redis: GET all SLOW_KEYS (incl. new energy keys)
BS-->>Panel: bootstrap payload (pipelinesGas, fuelShortages ...)
Panel->>Store: getCachedPipelineRegistries() [drain-once]
Map->>Store: getCachedPipelineRegistries() [returns memo - no race]
Store-->>Panel: {gas, oil} registries
Store-->>Map: same {gas, oil} registries
Panel->>Panel: derivePipelinePublicBadge(evidence) -> badge
Map->>Map: derivePipelinePublicBadge(evidence) -> identical badge
Panel->>RPC: listPipelines() [background refresh]
RPC->>Redis: getCachedJson(PIPELINES_GAS_KEY) + getCachedJson(PIPELINES_OIL_KEY)
Redis-->>RPC: raw registries
RPC->>RPC: projectPipeline + derivePublicBadge
RPC-->>Panel: ListPipelinesResponse
Panel->>Store: setCachedPipelineRegistries(fresh data)
Map->>Map: re-render with updated store on next cycle
Reviews (1): Last reviewed commit: "fix(energy): upstreamUnavailable only fi..." | Re-trigger Greptile |
| window.dispatchEvent(new CustomEvent('energy:open-pipeline-detail', { | ||
| detail: { pipelineId: obj.id }, | ||
| })); | ||
| } catch { |
There was a problem hiding this comment.
Resolved shortages rendered as active pins
createEnergyShortagePinsLayer builds pins from every entry in registry?.shortages, with no check for resolvedAt. A shortage marked resolved (non-empty resolvedAt string) will still appear as a coloured pin on the map, which can mislead users into thinking an already-resolved fuel crisis is ongoing. The same issue exists in CountryDeepDivePanel.renderAtlasExposure where the confirmed/watch severity count also includes resolved shortages.
Add a filter before the .map() step:
| } catch { | |
| const rawEntries: RawEntry[] = Object.values(registry?.shortages ?? {}) | |
| .filter(raw => !raw.resolvedAt); |
| const confirmedCount = shortages.filter(s => s.severity === 'confirmed').length; | ||
| const severityLine = confirmedCount > 0 | ||
| ? `${confirmedCount} confirmed · ${shortages.length - confirmedCount} watch` | ||
| : `${shortages.length} watch`; |
There was a problem hiding this comment.
Resolved shortages inflate country drill-down count
shortages here includes entries with a non-empty resolvedAt, so a country might show "2 confirmed · 0 watch" even when all shortages are resolved. The confirmation count and the rows dispatched on click all mix active and historical records with no visual distinction.
Filter to active-only shortages:
| : `${shortages.length} watch`; | |
| const shortages = Object.values(registry?.shortages ?? {}) | |
| .filter(s => s.country === iso2 && !s.resolvedAt); |
| return { | ||
| events: [], | ||
| fetchedAt: new Date().toISOString(), | ||
| classifierVersion: '', | ||
| upstreamUnavailable: true, | ||
| }; | ||
| } |
There was a problem hiding this comment.
upstreamUnavailable fires on partial-write, not only on null Redis return
The PR description and the sibling handlers (list-storage-facilities.ts checks if (!raw)) establish the contract: upstreamUnavailable = Redis returned nothing. Here, if (!raw?.events) will also fire when Redis returns a valid registry object that simply has no events key — e.g. a race during the seed write, or a future schema migration that renames the field. That's a partial write, not a Redis outage, and callers would be sent into error-state rendering unnecessarily.
For consistency with the stated contract, consider splitting the check:
if (!raw) {
return { events: [], fetchedAt: new Date().toISOString(), classifierVersion: '', upstreamUnavailable: true };
}
// raw.events absent means unexpected seed format — return empty but healthy
const events = Object.values(raw.events ?? {});Two P1 filter bugs (resolved shortages rendered as active) and one P2 contract inconsistency on the disruptions handler. P1: DeckGLMap createEnergyShortagePinsLayer rendered every shortage in the registry as an active crisis pin — including entries where the classifier has written resolvedAt to mark the crisis over. Added a filter so only entries with a null/empty resolvedAt become map pins. Curated v1 data has resolvedAt=null everywhere so no visible change today, but the moment the classifier starts writing resolutions post-launch, resolved shortages would have appeared as ongoing. P1: CountryDeepDivePanel renderAtlasExposure had the same bug in the country drill-down — "N confirmed · M watch" counts included resolved entries, inflating the active-crisis line per country. Same one-line filter fix. P2: list-energy-disruptions.ts gated upstreamUnavailable on `!raw?.events` — a partial write (top-level object present but `events` property missing) fired the "upstream down" flag, inconsistent with the sibling handlers (list-pipelines, list-storage-facilities, list-fuel-shortages) that only fire on `!raw`. Rewrote to match: `!raw` → upstreamUnavailable, empty events → normal empty list. This also aligns with the contract documented on the upstream-unavailable- vs-empty-filter skill extracted from the earlier P2 review. Typecheck + typecheck:api clean. All three fixes are one-liner filter or gate changes; no test additions needed (registry tests still pass with v1 data since resolvedAt is null throughout).
…t stdout (#3320) * chore(bundle-runner): emit reliable per-section summary line on parent stdout Fixes observability asymmetry in Railway bundle service logs where some seeders appeared to skip lines like \`Run ID\`, \`Mode\`, or the structured \`seed_complete\` JSON event. Root cause is Railway's log ingestion dropping child-stdout lines when multiple seeders emit at similar timestamps — observed in the PR #3294 launch run where Pipelines-Gas was missing its \`=== Seed ===\` banner, Pipelines-Oil had \`Key:\` emitted BEFORE the banner, Storage-Facilities and Energy- Disruptions were missing Run ID + Mode + seed_complete entirely, despite identical code paths. All child processes emit the same lines; Railway just loses some. Fix is to piggy-back on the observation that bundle-level lines (\`[Bundle:X] Starting\`, \`[Bundle:X] Finished\`) ARE reliably captured — they come from the parent process's single stdout stream. Changes in scripts/_bundle-runner.mjs: - spawnSeed now captures the child's \`{"event":"seed_complete",...}\` JSON line while streaming stdout, parses it, and attaches to the settle result. - Main loop emits one bundle-level summary line per section after child exit: [Bundle:X] section=NAME status=OK durationMs=1237 records=15 state=OK (or \`status=FAILED elapsed=...s reason=...\` for failures). - Summary line survives Railway's log ingestion even when per-section child lines drop, giving monitors a reliable event to key off. Observability consumers (log-based alerts, seed telemetry scrapers) should now key off the bundle-level summary rather than per-section child lines which remain best-effort. The per-section child lines stay as-is for interactive debugging. Verification: parse logic sanity-checked against the exact seed_complete line format. Node syntax check clean. No schema changes. * fix(bundle-runner): emit FAILED summary line to stderr, not stdout The prior commit introduced a bundle-level structured summary line per section. On success that correctly goes to stdout; on FAILED it was also going to stdout — but that broke tests/bundle-runner.test.mjs test 140 ("timeout emits terminal reason BEFORE SIGTERM/SIGKILL grace"). The test concatenates stdout+stderr and asserts that `SIGKILL` appears AFTER `Failed after` in the combined string (verifying the kill-decision log line is emitted BEFORE the 10s SIGTERM→SIGKILL grace window, so it survives container termination). My new FAILED summary line — which includes the reason string `timeout after 1s (signal SIGKILL)` — landed on stdout, which comes first in the concatenation, and its `SIGKILL` substring matched before the stderr-side `Did not exit on SIGTERM...SIGKILL` line. Ordering assertion failed. Fix: route the FAILED summary line through console.error (same stream as the pre-kill `Failed after ... sending SIGTERM` and the grace-window `Did not exit...SIGKILL` lines). Chronological ordering in combined output is preserved; test passes. OK summary lines stay on stdout — they're observability data, not error diagnostics, and belong on the normal stream alongside the bundle Starting/Finished lines. Local: `node --test tests/bundle-runner.test.mjs` — 4/4 pass including the previously-failing ordering test.
All five methodology pages reference /corrections (the auto-revision-log
URL promised in the Global Energy Flow parity plan §20) but the page
didn't exist — clicks 404'd. This lands the page.
Content:
- Explains the revision-log shape: `{date, assetOrEventId, fieldChanged,
previousValue, newValue, trigger, sourcesUsed, classifierVersion}`.
- Defines the trigger vocabulary (classifier / source / decay / override)
so readers know what kind of change they're seeing.
- States the v1-launch truth honestly: the log is empty at launch and
fills as the post-launch classifier pass (in proactive-intelligence.mjs)
runs on its normal schedule. No fake entries, no placeholder rows.
- Documents the correction-submission path (operators / regulators /
researchers with public source URLs) and the contract that
corrections write `override`-trigger entries citing the submitted
source — not anonymous overrides.
- Cross-links all five methodology pages.
- Explains WHY we publish this: evidence-first classification only
works if the audit trail is public; otherwise "the classifier said
so" has no more authority than any other opaque pipeline.
Also fixes a navigation gap: docs/docs.json was missing both
methodology/disruptions (landed in PR #3294 but never registered in
nav) and the new corrections page. Both now appear in the "Intelligence
& Analysis" group alongside the other methodology pages.
No code changes. MDX lint + docs.json JSON validation pass.
…nt) (#3323) * docs(energy): /corrections revision-log page (Week 4 launch requirement) All five methodology pages reference /corrections (the auto-revision-log URL promised in the Global Energy Flow parity plan §20) but the page didn't exist — clicks 404'd. This lands the page. Content: - Explains the revision-log shape: `{date, assetOrEventId, fieldChanged, previousValue, newValue, trigger, sourcesUsed, classifierVersion}`. - Defines the trigger vocabulary (classifier / source / decay / override) so readers know what kind of change they're seeing. - States the v1-launch truth honestly: the log is empty at launch and fills as the post-launch classifier pass (in proactive-intelligence.mjs) runs on its normal schedule. No fake entries, no placeholder rows. - Documents the correction-submission path (operators / regulators / researchers with public source URLs) and the contract that corrections write `override`-trigger entries citing the submitted source — not anonymous overrides. - Cross-links all five methodology pages. - Explains WHY we publish this: evidence-first classification only works if the audit trail is public; otherwise "the classifier said so" has no more authority than any other opaque pipeline. Also fixes a navigation gap: docs/docs.json was missing both methodology/disruptions (landed in PR #3294 but never registered in nav) and the new corrections page. Both now appear in the "Intelligence & Analysis" group alongside the other methodology pages. No code changes. MDX lint + docs.json JSON validation pass. * docs(energy): reframe /corrections as planned-surface spec (P1 review fix) Greptile P1: the prior /corrections page made live-product claims ("writes an append-only entry here", "expect the first entries within days", "email [email protected]") that the code doesn't back. The revision-log writer ships with the post-launch classifier; the correction-intake pipeline does not yet exist; and the related detail handlers still return empty `revisions` arrays with code comments explicitly marking the surface as future work. Fix: rewrite the page as a planned-surface specification with a prominent Status callout. Changes: - Page title: "Revision Log" → "Revision Log (Planned)" - Prominent <Note> callout at the top states v1 launch truth: log is not yet live, RPC `revisions` arrays are empty by design, corrections are handled manually today. - "Current state (v1 launch)" section removed; replaced with two explicit sections: "What IS live today" (evidence bundles, methodology, versioned classifier output) and "What is NOT live today" (log entries, automated correction intake, override-writer). - "Within days" timeline language removed — no false operational SLA. - Email submission path removed (no automated intake exists). Points readers to GitHub issues for manual review today. - Preserves the planned data shape, trigger vocabulary, policy commitment, and "why we publish this" framing — those are spec, not claims. Also softens /corrections references in the four methodology pages (pipelines, storage, shortages, disruptions) so none of them claim the revision log is live. Each now says "planned revision-log shape and submission policy" and points manual corrections at GitHub issues. MDX lint 122/122 clean. docs.json JSON validation clean. No code changes; pure reframing to match reality. * docs(shortages): fix P1 overclaim + wrong RPC name (round-2 review) Two findings on the same file: P1 — `energy_asset_overrides` table documented as existing. It doesn't. The PR's corrections.mdx explicitly lists the override-writer as NOT live in v1; this section contradicted that. Rewrote as "Break-glass overrides (planned)" with a clear Status callout matching the pattern established in docs/corrections.mdx and the other methodology pages. Points readers at GitHub issues for manual corrections today. P2 — Wrong RPC name: `listActiveFuelShortages` doesn't exist. The shipped RPC (in proto/worldmonitor/supply_chain/v1/ list_fuel_shortages.proto + server/worldmonitor/supply-chain/v1/ list-fuel-shortages.ts) is `ListFuelShortages`. Replaced the name + reframed the sentence to describe what the actual RPC already exposes (every FuelShortageEntry includes evidence.evidenceSources[]) rather than projecting a future surface. Also swept the other methodology pages for the same class of bug: - grep for _overrides: only the one line above - grep for listActive/ getActive RPC names: none found - verified all RPC mentions in docs/methodology + docs/corrections.mdx match names actually in proto (ListPipelines, ListStorageFacilities, ListFuelShortages, ListEnergyDisruptions, GetPipelineDetail, GetStorageFacilityDetail, GetFuelShortageDetail) MDX lint clean. No code changes. * docs(methodology): round-3 sibling sweep for revision-log overclaims Reviewer (Greptile) caught a third round of the same overclaim pattern I've been trying to stamp out: docs/methodology/shortages.mdx line 46 said "Stale shortages never persist silently. Every demotion writes to the public revision log." — contradicting the same PR's /corrections page which explicitly frames the revision log as not-yet-live. Fixed that one AND did the mechanical sibling sweep the review pattern clearly called for. Changes: - `docs/methodology/shortages.mdx:46` — rewrote the auto-decay footer to future tense: "When the post-launch classifier ships, stale shortages will never persist silently — every demotion will write an entry to the planned public revision log." Points readers at /corrections for the designed shape. Notes that today the demotion thresholds ARE the contract; the structured audit trail is what lands with the classifier. - `docs/methodology/chokepoints.mdx:64` — sibling sweep caught the same bug class ("Every badge transition writes to the public revision log"). Reworded to future tense and pointed manual corrections at GitHub issues, matching the pattern already applied to pipelines / storage / shortages in prior commits on this PR. Final audit of remaining revision-log mentions across all 5 methodology pages + corrections.mdx — every one uses hedged tense now ("planned", "will", "when live", "designed", "not yet live", "once the classifier ships"). The one remaining present-tense "emit" in shortages.mdx:77 is inside the "(planned)" break-glass section with its own Status callout, so it's correctly scoped. Following the plan-doc-as-docs-source-overclaim skill's step-4 (sibling sweep) explicitly this time — which also retroactively validates the skill extraction: three review rounds was the cost of not running the sweep on round 1. MDX lint clean. No code changes. * docs(corrections): drop hardcoded launch date (Greptile P2) Greptile inline P2 at docs/corrections.mdx:60: the phrase "v1 launch (2026-04-23)" pins a specific calendar date that will read inaccurately to visitors months later once entries start appearing. Dropped the parenthetical date. "Status — v1 launch:" keeps the scoping clear without tying it to a specific day. When live entries start appearing on this page (or when the page is rewritten to show real rows), a "last updated" marker will replace the status callout entirely — no migration churn needed. MDX lint 122/122 clean.
* feat(energy-atlas): expose Atlas panels on FULL variant + CMD+K Three Atlas panels (PipelineStatusPanel, StorageFacilityMapPanel, FuelShortagePanel) shipped in PR #3294 but were registered only in ENERGY_PANELS — invisible on worldmonitor.app because the energy variant subdomain is not yet wired. Additionally, no CMD+K entries existed for them, so command-palette search for "pipeline" or "storage" returned nothing. Changes: - src/config/commands.ts: add panel:pipeline-status, panel:storage-facility-map, panel:fuel-shortages with relevant keywords (oil/gas/nord stream/druzhba/spr/lng/ugs/rationing/…). - src/config/panels.ts: add the 3 panel keys to FULL_PANELS with priority 2 so they appear in the main worldmonitor.app drawer under the existing energy-crisis block. ENERGY_PANELS keeps its own priority-1 copies so the future energy.worldmonitor.app subdomain still surfaces them top of list. Unblocks the plan's announcement gate item "UI: at least one Atlas panel renders registry data on worldmonitor.app in a browser." Part of docs/internal/energy-atlas-registry-expansion.md follow-up. * fix(cmd-k): resolve spr/lng keyword collisions so Atlas panel wins Reviewer found that the original PR wiring let CMD+K "spr" and "lng" route to the wrong panel because matchCommands() (SearchModal.ts:273) ranks by exact/prefix/substring then keeps array-insertion order on ties. Storage-atlas was declared AFTER the colliding entries. Collisions: - "spr": panel:oil-inventories (line 105) had exact 'spr' → tied with the new storage-facility-map (line 108) → insertion order kept oil-inventories winning. - "lng": panel:hormuz-tracker (line 135) has exact 'lng' → storage-facility-map only had substring 'lng terminals' (score 1) → hormuz won outright. Fix: - Remove 'spr' from oil-inventories keywords. The SPR as a *site list* semantically belongs to Strategic Storage Atlas. Stock-level queries still route to oil-inventories via 'strategic petroleum' (the word 'spr' is not a substring of 'strategic petroleum', so no fallback score leaks). - Add exact 'lng' to storage-facility-map. Both it and hormuz-tracker now score 3 on 'lng'; stable sort preserves declaration order, so storage (line 108) outranks hormuz (line 135). Hormuz still matches via 'hormuz', 'strait of hormuz', 'tanker', 'shipping'.
Summary
Full Energy Atlas feature for the
energy.worldmonitor.appvariant, implementing Release 1 scope of the Global Energy Flow parity plan (4-week plan, internal doc). 17 commits, 69 files, +9,663 lines. Ready for merge.Five registries + nine RPCs + three panels + three DeckGL map layers + drill-down + methodology, all behind the existing energy variant scaffold shipped in #3291.
What's in here
Registries (curated v1, classifier-extended post-launch)
energy:pipelines:gas:v1energy:pipelines:oil:v1energy:storage-facilities:v1energy:fuel-shortages:v1energy:disruptions:v1Every entry carries a versioned evidence bundle (
physicalState,commercialState,sanctionRefs[],operatorStatement,classifierVersion,classifierConfidence,lastEvidenceUpdate). Public badges (flowing/reduced/offline/disputedfor assets,confirmed/watchfor shortages) are either derived server-side from evidence or emitted directly by the classifier — never hand-labeled.All five registries registered across the 4-file bootstrap checklist (
cache-keys.ts,api/bootstrap.js,api/health.js,api/seed-health.js) with correctmaxStaleMinandintervalMinmirroring cron cadence.RPCs (all under
/api/supply-chain/v1, gateway cache-tier set)ListPipelines+GetPipelineDetailListStorageFacilities+GetStorageFacilityDetailListFuelShortages+GetFuelShortageDetailListEnergyDisruptionsEvery handler projects raw Redis → wire format through a pure projection function that attaches the derived
public_badge.upstreamUnavailableis correctly gated on raw-fetch-null, NOT on post-filter row count (fixed in20341c324after P2 review — seeupstream-unavailable-vs-empty-filterskill).Shared deriver modules (
src/shared/)Duck-typed, dependency-free so the same pure function powers both the RPC handler projection AND the panel's bootstrap-first-paint projection. Identical output in both paths → no hydration flicker.
pipeline-evidence.ts—derivePipelinePublicBadge+ classifier-version picker + ISO timestamp pickerstorage-evidence.ts—deriveStoragePublicBadgeshortage-evidence.ts—deriveShortageEvidenceQuality(trust hint only, NOT severity — that's classifier output)disruption-timeline.ts— formatters for the shared timeline UIpipeline-registry-store.ts/storage-facility-registry-store.ts/fuel-shortage-registry-store.ts— drain-once memo stores so panel + DeckGL layer don't race on the single-usegetHydratedDatacachePanels (energy variant only)
window.dispatchEvent('energy:open-*-detail')(loose coupling — no-ops when the target panel isn't mounted)DeckGL map layers (energy variant only)
Country drill-down
CountryDeepDivePanel.renderEnergyProfilegains an "Atlas exposure" sub-section per country showing:Rows dispatch the same panel-open events — click-through works on the energy variant, silently no-ops elsewhere. Dynamic imports keep the three stores out of non-energy variant bundles.
Methodology pages
Public docs at
/docs/methodology/{pipelines,storage,shortages,chokepoints,disruptions}. Each explains scope, evidence schema, what the classifier does vs. what humans do, public API, versioning, and the revision-log audit path.Tests
Commit log (stacked)
```
20341c3 fix(energy): upstreamUnavailable only fires when Redis returned nothing
3ff0151 docs(energy): methodology page for energy disruption event log
6999138 feat(energy): Day 14 — country drill-down Atlas exposure section
ccce482 feat(energy): Day 13 — energy disruption event log + asset timeline drawer
44e6d78 feat(energy): Day 12 — FuelShortagePanel + DeckGL shortage pins
0454a9b feat(energy): Day 11 — fuel-shortage registry schema + seed + RPC
d78bab4 feat(energy): Day 10 — storage atlas (ListStorageFacilities RPC + DeckGL layer + panel)
c75a2ac feat(energy): Day 9 — storage facility registry (UGS + SPR + LNG + crude hubs)
57f4341 fix(energy): three PR-review findings on Day 8b DeckGL integration
2e70d05 feat(energy): DeckGL PathLayer colored by evidence-derived badge
db52965 fix(energy): three PR-review P2s on PipelineStatusPanel + aggregators
149d33e fix(energy): PipelineStatusPanel bootstrap path — client-side badge derivation
792eb1a feat(energy): PipelineStatusPanel — evidence-backed status table + drawer
0f60611 feat(energy): ListPipelines + GetPipelineDetail RPCs with evidence-derived badges
6b01fa5 fix(energy): complete pipeline bootstrap registration per 4-file checklist
fd0adcd fix(energy): split seed-pipelines.mjs into two entry points
c49ab39 feat(energy): pipeline registries (gas + oil) — evidence-based schema
```
Design principles enforced
bootstrap-hydration-derived-field-parityskill).runSeed()hard-exits. Two runSeed calls in one process = second key silently unwritten. Pipelines split into two entries accordingly (seerunseed-then-after-process-exitskill).Known follow-ups (intentionally deferred to post-launch per plan)
Test plan
Related