* feat(intelligence): weekly regional briefs (Phase 3 PR2)

Phase 3 PR2 of the Regional Intelligence Model. Adds LLM-powered
weekly intelligence briefs per region, completing the core feature set.

## New seeder: scripts/seed-regional-briefs.mjs

Standalone weekly cron script (not part of the 6h derived-signals bundle).
For each non-global region:
  1. Read the latest snapshot via two-hop Redis read
  2. Read recent regime transitions from the history log (koala73#2981)
  3. Call the LLM once per region with regime trajectory + balance +
     triggers + narrative context
  4. Write structured brief to intelligence:regional-briefs:v1:weekly:{region}
     with 8-day TTL (survives one missed weekly run)

Reuses the same injectable-callLlm + parse-validation + provider-chain
pattern from narrative.mjs and weekly-brief.mjs.

## New module: scripts/regional-snapshot/weekly-brief.mjs

  generateWeeklyBrief(region, snapshot, transitions, opts?)
    -> { region_id, generated_at, period_start, period_end,
         situation_recap, regime_trajectory, key_developments[],
         risk_outlook, provider, model }

  buildBriefPrompt()    — pure prompt builder
  parseBriefJson()      — JSON parser with prose-extraction fallback
  emptyBrief()          — canonical empty shape

Global region is skipped. Provider chain: Groq -> OpenRouter. Validate
callback ensures only parseable responses pass (narrative.mjs PR koala73#2960
review fix pattern).

## Proto + RPC: GetRegionalBrief

  proto/worldmonitor/intelligence/v1/get_regional_brief.proto

  - GetRegionalBriefRequest { region_id }
  - GetRegionalBriefResponse { brief: RegionalBrief }
  - RegionalBrief { region_id, generated_at, period_start, period_end,
                    situation_recap, regime_trajectory,
                    key_developments[], risk_outlook, provider, model }

## Server handler

  server/worldmonitor/intelligence/v1/get-regional-brief.ts

Simple getCachedJson read + adaptBrief snake->camel adapter.
Returns upstreamUnavailable: true on Redis failure so the gateway
skips caching (matching the get-regime-history pattern from koala73#2981).

## Premium gating + cache tier

  src/shared/premium-paths.ts + server/gateway.ts RPC_CACHE_TIER

## Tests — 27 new unit tests

  buildBriefPrompt (5): region/balance/transitions/narrative rendered,
                        empty transitions handled, missing fields tolerated
  parseBriefJson (5): valid JSON, garbage, all-empty, cap at 5, prose extraction
  generateWeeklyBrief (6): success, global skip, LLM fail, garbage, exception,
                           period_start/end delta
  emptyBrief (2): region_id + empty fields
  handler (4): key prefix, adapter export, upstreamUnavailable, registration
  security (2): premium path + cache tier
  proto (3): RPC declared, import wired, RegionalBrief fields

## Verification

- npm run test:data: 4651/4651 pass
- npm run typecheck + typecheck:api: clean
- biome lint: clean

* fix(intelligence): address 3 review findings on koala73#2989

P2 #1 — no consumer surface for GetRegionalBrief

Acknowledged. The consumer is the RegionalIntelligenceBoard panel,
which will call GetRegionalBrief and render a weekly brief block.
This wiring is Phase 3 PR3 (UI) scope — the RPC + Redis key are the
delivery mechanism, not the end surface. No code change in this commit;
the RPC is ready for the panel to consume.

P2 #2 — readRecentTransitions collapses failure to []

readRecentTransitions returned [] on Redis/network failure, which is
indistinguishable from a genuinely quiet week. The LLM then generates
a brief claiming "no regime transitions" when in reality the upstream
is down — fabricating false input.

Fix: return null on failure. The seeder skips the region with a clear
log message when transitions is null, so the brief is never written
with unreliable input. Empty array [] now only means genuinely no
transitions in the 7-day window.

P2 #3 — parseBriefJson accepts briefs the seeder rejects

parseBriefJson treated non-empty key_developments as valid even if
situation_recap was empty. The seeder gate only writes when
brief.situation_recap is truthy. That mismatch means the validator
pass + provider-fallback logic could accept a response that the seeder
then silently drops.

Fix: require situation_recap in parseBriefJson for valid=true, matching
the seeder gate. Now both checks agree on what constitutes a usable
brief, and the provider-fallback chain correctly falls through when
a provider returns a brief with developments but no recap.

* fix(intelligence): TTL path-segment fix + seed-meta always-write (Greptile P1+P2 on koala73#2989)

P1 — TTL silently not applied (briefs never expire)

Upstash REST ignores query-string SET options (?EX=N). The correct
form is path-segment: /set/{key}/{value}/EX/{seconds}. Without this
fix every brief persists indefinitely and Redis storage grows
unboundedly across weekly runs.

P2 — seed-meta not written when all regions skipped

writeExtraKeyWithMeta was gated on generated > 0. If every region
was skipped (no snapshot yet, or LLM failed), seed-meta was never
written, making the seeder indistinguishable from "never ran" in
health tooling. Now writes seed-meta whenever failed === 0,
carrying regionsSkipped count.

P2 #3 (validate gate) — already fixed in previous commit (parseBriefJson
now requires situation_recap for valid=true).

* fix(intelligence): register regional-briefs in health.js SEED_META + STANDALONE_KEYS (review P2 on koala73#2989)

* fix(intelligence): register regional-briefs in api/seed-health.js (review P2 on koala73#2989)

* fix(intelligence): raise brief TTL to 15 days to cover missed weekly cycle (review P2 on koala73#2989)

* fix(intelligence): distinguish missing-key from Redis-error + coverage-gated health (review P2s on koala73#2989)

P2 #1 — false upstreamUnavailable before first seed

getCachedJson returns null for both "key missing" and "Redis failed",
so the handler was advertising an outage for every region before the
first weekly seed ran. Switched to getRawJson (throws on Redis errors)
so null = genuinely missing key → clean empty 200, and thrown error =
upstream failure → upstreamUnavailable: true for gateway no-store.

P2 #2 — partial run hides coverage loss in health

The seed-meta was written with generated count even if only 1 of 7
regions produced a brief. /api/health treats any positive recordCount
as healthy, so broad regional failure was invisible to operators.

Fix: recordCount is set to 0 when generated < ceil(expectedRegions/2).
This makes /api/health report EMPTY_DATA for severely partial runs
while still writing seed-meta (so the seeder is confirmed to have run).
coverageOk flag in the summary payload lets operators drill into the
exact coverage state.

* fix(intelligence): tighten coverage gate to expectedRegions-1 (review P2 on koala73#2989)

…a73#3207) (koala73#3242)

* chore(api): enforce sebuf contract via exceptions manifest (koala73#3207)

Adds api/api-route-exceptions.json as the single source of truth for
non-proto /api/ endpoints, with scripts/enforce-sebuf-api-contract.mjs
gating every PR via npm run lint:api-contract. Fixes the root-only blind
spot in the prior allowlist (tests/edge-functions.test.mjs), which only
scanned top-level *.js files and missed nested paths and .ts endpoints —
the gap that let api/supply-chain/v1/country-products.ts and friends
drift under proto domain URL prefixes unchallenged.

Checks both directions: every api/<domain>/v<N>/[rpc].ts must pair with
a generated service_server.ts (so a deleted proto fails CI), and every
generated service must have an HTTP gateway (no orphaned generated code).

Manifest entries require category + reason + owner, with removal_issue
mandatory for temporary categories (deferred, migration-pending) and
forbidden for permanent ones. .github/CODEOWNERS pins the manifest to
@SebastienMelki so new exceptions don't slip through review.

The manifest only shrinks: migration-pending entries (19 today) will be
removed as subsequent commits in this PR land each migration.

* refactor(maritime): migrate /api/ais-snapshot → maritime/v1.GetVesselSnapshot (koala73#3207)

The proto VesselSnapshot was carrying density + disruptions but the frontend
also needed sequence, relay status, and candidate_reports to drive the
position-callback system. Those only lived on the raw relay passthrough, so
the client had to keep hitting /api/ais-snapshot whenever callbacks were
registered and fall back to the proto RPC only when the relay URL was gone.

This commit pushes all three missing fields through the proto contract and
collapses the dual-fetch-path into one proto client call.

Proto changes (proto/worldmonitor/maritime/v1/):
  - VesselSnapshot gains sequence, status, candidate_reports.
  - GetVesselSnapshotRequest gains include_candidates (query: include_candidates).

Handler (server/worldmonitor/maritime/v1/get-vessel-snapshot.ts):
  - Forwards include_candidates to ?candidates=... on the relay.
  - Separate 5-min in-memory caches for the candidates=on and candidates=off
    variants; they have very different payload sizes and should not share a slot.
  - Per-request in-flight dedup preserved per-variant.

Frontend (src/services/maritime/index.ts):
  - fetchSnapshotPayload now calls MaritimeServiceClient.getVesselSnapshot
    directly with includeCandidates threaded through. The raw-relay path,
    SNAPSHOT_PROXY_URL, DIRECT_RAILWAY_SNAPSHOT_URL and LOCAL_SNAPSHOT_FALLBACK
    are gone — production already routed via Vercel, the "direct" branch only
    ever fired on localhost, and the proto gateway covers both.
  - New toLegacyCandidateReport helper mirrors toDensityZone/toDisruptionEvent.

api/ais-snapshot.js deleted; manifest entry removed. Only reduced the codegen
scope to worldmonitor.maritime.v1 (buf generate --path) — regenerating the
full tree drops // @ts-nocheck from every client/server file and surfaces
pre-existing type errors across 30+ unrelated services, which is not in
scope for this PR.

Shape-diff vs legacy payload:
  - disruptions / density: proto carries the same fields, just with the
    GeoCoordinates wrapper and enum strings (remapped client-side via
    existing toDisruptionEvent / toDensityZone helpers).
  - sequence, status.{connected,vessels,messages}: now populated from the
    proto response — was hardcoded to 0/false in the prior proto fallback.
  - candidateReports: same shape; optional numeric fields come through as
    0 instead of undefined, which the legacy consumer already handled.

* refactor(sanctions): migrate /api/sanctions-entity-search → LookupSanctionEntity (koala73#3207)

The proto docstring already claimed "OFAC + OpenSanctions" coverage but the
handler only fuzzy-matched a local OFAC Redis index — narrower than the
legacy /api/sanctions-entity-search, which proxied OpenSanctions live (the
source advertised in docs/api-proxies.mdx). Deleting the legacy without
expanding the handler would have been a silent coverage regression for
external consumers.

Handler changes (server/worldmonitor/sanctions/v1/lookup-entity.ts):
  - Primary path: live search against api.opensanctions.org/search/default
    with an 8s timeout and the same User-Agent the legacy edge fn used.
  - Fallback path: the existing OFAC local fuzzy match, kept intact for when
    OpenSanctions is unreachable / rate-limiting.
  - Response source field flips between 'opensanctions' (happy path) and
    'ofac' (fallback) so clients can tell which index answered.
  - Query validation tightened: rejects q > 200 chars (matches legacy cap).

Rate limiting:
  - Added /api/sanctions/v1/lookup-entity to ENDPOINT_RATE_POLICIES at 30/min
    per IP — matches the legacy createIpRateLimiter budget. The gateway
    already enforces per-endpoint policies via checkEndpointRateLimit.

Docs:
  - docs/api-proxies.mdx — dropped the /api/sanctions-entity-search row
    (plus the orphaned /api/ais-snapshot row left over from the previous
    commit in this PR).
  - docs/panels/sanctions-pressure.mdx — points at the new RPC URL and
    describes the OpenSanctions-primary / OFAC-fallback semantics.

api/sanctions-entity-search.js deleted; manifest entry removed.

* refactor(military): migrate /api/military-flights → ListMilitaryFlights (koala73#3207)

Legacy /api/military-flights read a pre-baked Redis blob written by the
seed-military-flights cron and returned flights in a flat app-friendly
shape (lat/lon, lowercase enums, lastSeenMs). The proto RPC takes a bbox,
fetches OpenSky live, classifies server-side, and returns nested
GeoCoordinates + MILITARY_*_TYPE_* enum strings + lastSeenAt — same data,
different contract.

fetchFromRedis in src/services/military-flights.ts was doing nothing
sebuf-aware. Renamed it to fetchViaProto and rewrote to:

  - Instantiate MilitaryServiceClient against getRpcBaseUrl().
  - Iterate MILITARY_QUERY_REGIONS (PACIFIC + WESTERN) in parallel — same
    regions the desktop OpenSky path and the seed cron already use, so
    dashboard coverage tracks the analytic pipeline.
  - Dedup by hexCode across regions.
  - Map proto → app shape via new mapProtoFlight helper plus three reverse
    enum maps (AIRCRAFT_TYPE_REVERSE, OPERATOR_REVERSE, CONFIDENCE_REVERSE).

The seed cron (scripts/seed-military-flights.mjs) stays put: it feeds
regional-snapshot mobility, cross-source signals, correlation, and the
health freshness check (api/health.js: 'military:flights:v1'). None of
those read the legacy HTTP endpoint; they read the Redis key directly.
The proto handler uses its own per-bbox cache keys under the same prefix,
so dashboard traffic no longer races the seed cron's blob — the two paths
diverge by a small refresh lag, which is acceptable.

Docs: dropped the /api/military-flights row from docs/api-proxies.mdx.

api/military-flights.js deleted; manifest entry removed.

Shape-diff vs legacy:
  - f.location.{latitude,longitude} → f.lat, f.lon
  - f.aircraftType: MILITARY_AIRCRAFT_TYPE_TANKER → 'tanker' via reverse map
  - f.operator: MILITARY_OPERATOR_USAF → 'usaf' via reverse map
  - f.confidence: MILITARY_CONFIDENCE_LOW → 'low' via reverse map
  - f.lastSeenAt (number) → f.lastSeen (Date)
  - f.enrichment → f.enriched (with field renames)
  - Extra fields registration / aircraftModel / origin / destination /
    firstSeenAt now flow through where proto populates them.

* fix(supply-chain): thread includeCandidates through chokepoint status (koala73#3207)

Caught by tsconfig.api.json typecheck in the pre-push hook (not covered
by the plain tsc --noEmit run that ran before I pushed the ais-snapshot
commit). The chokepoint status handler calls getVesselSnapshot internally
with a static no-auth request — now required to include the new
includeCandidates bool from the proto extension.

Passing false: server-internal callers don't need per-vessel reports.

* test(maritime): update getVesselSnapshot cache assertions (koala73#3207)

The ais-snapshot migration replaced the single cachedSnapshot/cacheTimestamp
pair with a per-variant cache so candidates-on and candidates-off payloads
don't evict each other. Pre-push hook surfaced that tests/server-handlers
still asserted the old variable names. Rewriting the assertions to match
the new shape while preserving the invariants they actually guard:

  - Freshness check against slot TTL.
  - Cache read before relay call.
  - Per-slot in-flight dedup.
  - Stale-serve on relay failure (result ?? slot.snapshot).

* chore(proto): restore // @ts-nocheck on regenerated maritime files (koala73#3207)

I ran 'buf generate --path worldmonitor/maritime/v1' to scope the proto
regen to the one service I was changing (to avoid the toolchain drift
that drops @ts-nocheck from 60+ unrelated files — separate issue). But
the repo convention is the 'make generate' target, which runs buf and
then sed-prepends '// @ts-nocheck' to every generated .ts file. My
scoped command skipped the sed step. The proto-check CI enforces the
sed output, so the two maritime files need the directive restored.

* refactor(enrichment): decomm /api/enrichment/{company,signals} legacy edge fns (koala73#3207)

Both endpoints were already ported to IntelligenceService:
  - getCompanyEnrichment  (/api/intelligence/v1/get-company-enrichment)
  - listCompanySignals    (/api/intelligence/v1/list-company-signals)

No frontend callers of the legacy /api/enrichment/* paths exist. Removes:
  - api/enrichment/company.js, signals.js, _domain.js
  - api-route-exceptions.json migration-pending entries (58 remain)
  - docs/api-proxies.mdx rows for /api/enrichment/{company,signals}
  - docs/architecture.mdx reference updated to the IntelligenceService RPCs

Verified: typecheck, typecheck:api, lint:api-contract (89 files / 58 entries),
lint:boundaries, tests/edge-functions.test.mjs (136 pass),
tests/enrichment-caching.test.mjs (14 pass — still guards the intelligence/v1
handlers), make generate is zero-diff.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor(leads): migrate /api/{contact,register-interest} → LeadsService (koala73#3207)

New leads/v1 sebuf service with two POST RPCs:
  - SubmitContact    → /api/leads/v1/submit-contact
  - RegisterInterest → /api/leads/v1/register-interest

Handler logic ported 1:1 from api/contact.js + api/register-interest.js:
  - Turnstile verification (desktop sources bypass, preserved)
  - Honeypot (website field) silently accepts without upstream calls
  - Free-email-domain gate on SubmitContact (422 ApiError)
  - validateEmail (disposable/offensive/typo-TLD/MX) on RegisterInterest
  - Convex writes via ConvexHttpClient (contactMessages:submit, registerInterest:register)
  - Resend notification + confirmation emails (HTML templates unchanged)

Shared helpers moved to server/_shared/:
  - turnstile.ts (getClientIp + verifyTurnstile)
  - email-validation.ts (disposable/offensive/MX checks)

Rate limits preserved via ENDPOINT_RATE_POLICIES:
  - submit-contact:    3/hour per IP (was in-memory 3/hr)
  - register-interest: 5/hour per IP (was in-memory 5/hr; desktop
    sources previously capped at 2/hr via shared in-memory map —
    now 5/hr like everyone else, accepting the small regression in
    exchange for Upstash-backed global limiting)

Callers updated:
  - pro-test/src/App.tsx contact form → new submit-contact path
  - src-tauri/sidecar/local-api-server.mjs cloud-fallback rewrites
    /api/register-interest → /api/leads/v1/register-interest when
    proxying; keeps local path for older desktop builds
  - src/services/runtime.ts isKeyFreeApiTarget allows both old and
    new paths through the WORLDMONITOR_API_KEY-optional gate

Tests:
  - tests/contact-handler.test.mjs rewritten to call submitContact
    handler directly; asserts on ValidationError / ApiError
  - tests/email-validation.test.mjs + tests/turnstile.test.mjs
    point at the new server/_shared/ modules

Deleted: api/contact.js, api/register-interest.js, api/_ip-rate-limit.js,
api/_turnstile.js, api/_email-validation.js, api/_turnstile.test.mjs.
Manifest entries removed (58 → 56). Docs updated (api-platform,
api-commerce, usage-rate-limits).

Verified: npm run typecheck + typecheck:api + lint:api-contract
(88 files / 56 entries) + lint:boundaries pass; full test:data
(5852 tests) passes; make generate is zero-diff.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* chore(pro-test): rebuild bundle for leads/v1 contact form (koala73#3207)

Updates the enterprise contact form to POST to /api/leads/v1/submit-contact
(old path /api/contact removed in the previous commit).

Bundle is rebuilt from pro-test/src/App.tsx source change in 9ccd309.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review): address HIGH review findings 1-3 (koala73#3207)

Three review findings from @koala73 on the sebuf-migration PR, all
silent bugs that would have shipped to prod:

### 1. Sanctions rate-limit policy was dead code

ENDPOINT_RATE_POLICIES keyed the 30/min budget under
/api/sanctions/v1/lookup-entity, but the generated route (from the
proto RPC LookupSanctionEntity) is /api/sanctions/v1/lookup-sanction-entity.
hasEndpointRatePolicy / getEndpointRatelimit are exact-string pathname
lookups, so the mismatch meant the endpoint fell through to the
generic 600/min global limiter instead of the advertised 30/min.

Net effect: the live OpenSanctions proxy endpoint (unauthenticated,
external upstream) had 20x the intended rate budget. Fixed by renaming
the policy key to match the generated route.

### 2. Lost stale-seed fallback on military-flights

Legacy api/military-flights.js cascaded military:flights:v1 →
military:flights:stale:v1 before returning empty. The new proto
handler went straight to live OpenSky/relay and returned null on miss.

Relay or OpenSky hiccup used to serve stale seeded data (24h TTL);
under the new handler it showed an empty map. Both keys are still
written by scripts/seed-military-flights.mjs on every run — fix just
reads the stale key when the live fetch returns null, converts the
seed's app-shape flights (flat lat/lon, lowercase enums, lastSeenMs)
to the proto shape (nested GeoCoordinates, enum strings, lastSeenAt),
and filters to the request bbox.

Read via getRawJson (unprefixed) to match the seed cron's writes,
which bypass the env-prefix system.

### 3. Hex-code casing mismatch broke getFlightByHex

The seed cron writes hexCode: icao24.toUpperCase() (uppercase);
src/services/military-flights.ts:getFlightByHex uppercases the lookup
input: f.hexCode === hexCode.toUpperCase(). The new proto handler
preserved OpenSky's lowercase icao24, and mapProtoFlight is a
pass-through. getFlightByHex was silently returning undefined for
every call after the migration.

Fix: uppercase in the proto handler (live + stale paths), and document
the invariant in a comment on MilitaryFlight.hex_code in
military_flight.proto so future handlers don't re-break it.

### Verified

- typecheck + typecheck:api clean
- lint:api-contract (56 entries) / lint:boundaries clean
- tests/edge-functions.test.mjs 130 pass
- make generate zero-diff (openapi spec regenerated for proto comment)

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review): restore desktop 2/hr rate cap on register-interest (koala73#3207)

Addresses HIGH review finding #4 from @koala73. The legacy
api/register-interest.js applied a nested 2/hr per-IP cap when
`source === 'desktop-settings'`, on top of the generic 5/hr endpoint
budget. The sebuf migration lost this — desktop-source requests now
enjoy the full 5/hr cap.

Since `source` is an unsigned client-supplied field, anyone sending
`source: 'desktop-settings'` skips Turnstile AND gets 5/hr. Without
the tighter cap the Turnstile bypass is cheaper to abuse.

Added `checkScopedRateLimit` to `server/_shared/rate-limit.ts` — a
reusable second-stage Upstash limiter keyed on an opaque scope string
+ caller identifier. Fail-open on Redis errors to match existing
checkRateLimit / checkEndpointRateLimit semantics. Handlers that need
per-subscope caps on top of the gateway-level endpoint budget use this
helper.

In register-interest: when `isDesktopSource`, call checkScopedRateLimit
with scope `/api/leads/v1/register-interest#desktop`, limit=2, window=1h,
IP as identifier. On exceeded → throw ApiError(429).

### What this does not fix

This caps the blast radius of the Turnstile bypass but does not close
it — an attacker sending `source: 'desktop-settings'` still skips
Turnstile (just at 2/hr instead of 5/hr). The proper fix is a signed
desktop-secret header that authenticates the bypass; filed as
follow-up koala73#3252. That requires coordinated Tauri build + Vercel env
changes out of scope for koala73#3207.

### Verified

- typecheck + typecheck:api clean
- lint:api-contract (56 entries)
- tests/edge-functions.test.mjs + contact-handler.test.mjs (147 pass)

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review): MEDIUM + LOW + rate-limit-policy CI check (koala73#3207)

Closes out the remaining @koala73 review findings from koala73#3242 that
didn't already land in the HIGH-fix commits, plus the requested CI
check that would have caught HIGH #1 (dead-code policy key) at
review time.

### MEDIUM #5 — Turnstile missing-secret policy default

Flip `verifyTurnstile`'s default `missingSecretPolicy` from `'allow'`
to `'allow-in-development'`. Dev with no secret = pass (expected
local); prod with no secret = reject + log. submit-contact was
already explicitly overriding to `'allow-in-development'`;
register-interest was silently getting `'allow'`. Safe default now
means a future missing-secret misconfiguration in prod gets caught
instead of silently letting bots through. Removed the now-redundant
override in submit-contact.

### MEDIUM #6 — Silent enum fallbacks in maritime client

`toDisruptionEvent` mapped `AIS_DISRUPTION_TYPE_UNSPECIFIED` / unknown
enum values → `gap_spike` / `low` silently. Refactored to return null
when either enum is unknown; caller filters nulls out of the array.
Handler doesn't produce UNSPECIFIED today, but the `gap_spike`
default would have mislabeled the first new enum value the proto
ever adds — dropping unknowns is safer than shipping wrong labels.

### LOW — Copy drift in register-interest email

Email template hardcoded `435+ Sources`; PR koala73#3241 bumped marketing to
`500+`. Bumped in the rewritten file to stay consistent.

The `as any` on Convex mutation names carried over from legacy and
filed as follow-up koala73#3253.

### Rate-limit-policy coverage lint

`scripts/enforce-rate-limit-policies.mjs` validates every key in
`ENDPOINT_RATE_POLICIES` resolves to a proto-generated gateway route
by cross-referencing `docs/api/*.openapi.yaml`. Fails with the
sanctions-entity-search incident referenced in the error message so
future drift has a paper trail.

Wired into package.json (`lint:rate-limit-policies`) and the pre-push
hook alongside `lint:boundaries`. Smoke-tested both directions —
clean repo passes (5 policies / 175 routes), seeded drift (the exact
HIGH #1 typo) fails with the advertised remedy text.

### Verified
- `lint:rate-limit-policies` ✓
- `typecheck` + `typecheck:api` ✓
- `lint:api-contract` ✓ (56 entries)
- `lint:boundaries` ✓
- edge-functions + contact-handler tests (147 pass)

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor(commit 5): decomm /api/eia/* + migrate /api/satellites → IntelligenceService (koala73#3207)

Both targets turned out to be decomm-not-migration cases. The original
plan called for two new services (economic/v1.GetEiaSeries +
natural/v1.ListSatellitePositions) but research found neither was
needed:

### /api/eia/[[...path]].js — pure decomm, zero consumers

The "catch-all" is a misnomer — only two paths actually worked,
/api/eia/health and /api/eia/petroleum, both Redis-only readers.
Zero frontend callers in src/. Zero server-side readers. Nothing
consumes the `energy:eia-petroleum:v1` key that seed-eia-petroleum.mjs
writes daily.

The EIA data the frontend actually uses goes through existing typed
RPCs in economic/v1: GetEnergyPrices, GetCrudeInventories,
GetNatGasStorage, GetEnergyCapacity. None of those touch /api/eia/*.

Building GetEiaSeries would have been dead code. Deleted the legacy
file + its test (tests/api-eia-petroleum.test.mjs — it only covered
the legacy endpoint, no behavior to preserve). Empty api/eia/ dir
removed.

**Note for review:** the Redis seed cron keeps running daily and
nothing consumes it. If that stays unused, seed-eia-petroleum.mjs
should be retired too (separate PR). Out of scope for sebuf-migration.

### /api/satellites.js — Learning #2 strikes again

IntelligenceService.ListSatellites already exists at
/api/intelligence/v1/list-satellites, reads the same Redis key
(intelligence:satellites:tle:v1), and supports an optional country
filter the legacy didn't have.

One frontend caller in src/services/satellites.ts needed to switch
from `fetch(toApiUrl('/api/satellites'))` to the typed
IntelligenceServiceClient.listSatellites. Shape diff was tiny —
legacy `noradId` became proto `id` (handler line 36 already picks
either), everything else identical. alt/velocity/inclination in the
proto are ignored by the caller since it propagates positions
client-side via satellite.js.

Kept the client-side cache + failure cooldown + 20s timeout (still
valid concerns at the caller level).

### Manifest + docs
- api-route-exceptions.json: 56 → 54 entries (both removed)
- docs/api-proxies.mdx: dropped the two rows from the Raw-data
  passthroughs table

### Verified
- typecheck + typecheck:api ✓
- lint:api-contract (54 entries) / lint:boundaries / lint:rate-limit-policies ✓
- tests/edge-functions.test.mjs 127 pass (down from 130 — 3 tests were
  for the deleted eia endpoint)
- make generate zero-diff (no proto changes)

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor(commit 6): migrate /api/supply-chain/v1/{country-products,multi-sector-cost-shock} → SupplyChainService (koala73#3207)

Both endpoints were hand-rolled TS handlers sitting under a proto URL prefix —
the exact drift the manifest guardrail flagged. Promoted both to typed RPCs:

- GetCountryProducts → /api/supply-chain/v1/get-country-products
- GetMultiSectorCostShock → /api/supply-chain/v1/get-multi-sector-cost-shock

Handlers preserve the existing semantics: PRO-gate via isCallerPremium(ctx.request),
iso2 / chokepointId validation, raw bilateral-hs4 Redis read (skip env-prefix to
match seeder writes), CHOKEPOINT_STATUS_KEY for war-risk tier, and the math from
_multi-sector-shock.ts unchanged. Empty-data and non-PRO paths return the typed
empty payload (no 403 — the sebuf gateway pattern is empty-payload-on-deny).

Client wrapper switches from premiumFetch to client.getCountryProducts/
client.getMultiSectorCostShock. Legacy MultiSectorShock / MultiSectorShockResponse /
CountryProductsResponse names remain as type aliases of the generated proto types
so CountryBriefPanel + CountryDeepDivePanel callsites compile with zero churn.

Manifest 54 → 52. Rate-limit gateway routes 175 → 177.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(gateway): add cache-tier entries for new supply-chain RPCs (koala73#3207)

Pre-push tests/route-cache-tier.test.mjs caught the missing entries.
Both PRO-gated, request-varying — match the existing supply-chain PRO cohort
(get-country-cost-shock, get-bypass-options, etc.) at slow-browser tier.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor(commit 7): migrate /api/scenario/v1/{run,status,templates} → ScenarioService (koala73#3207)

Promote the three literal-filename scenario endpoints to a typed sebuf
service with three RPCs:

  POST /api/scenario/v1/run-scenario        (RunScenario)
  GET  /api/scenario/v1/get-scenario-status (GetScenarioStatus)
  GET  /api/scenario/v1/list-scenario-templates (ListScenarioTemplates)

Preserves all security invariants from the legacy handlers:
- 405 for wrong method (sebuf service-config method gate)
- scenarioId validation against SCENARIO_TEMPLATES registry
- iso2 regex ^[A-Z]{2}$
- JOB_ID_RE path-traversal guard on status
- Per-IP 10/min rate limit (moved to gateway ENDPOINT_RATE_POLICIES)
- Queue-depth backpressure (>100 → 429)
- PRO gating via isCallerPremium
- AbortSignal.timeout on every Redis pipeline (runRedisPipeline helper)

Wire-level diffs vs legacy:
- Per-user RL now enforced at the gateway (same 10/min/IP budget).
- Rate-limit response omits Retry-After header; retryAfter is in the
  body per error-mapper.ts convention.
- ListScenarioTemplates emits affectedHs2: [] when the registry entry
  is null (all-sectors sentinel); proto repeated cannot carry null.
- RunScenario returns { jobId, status } (no statusUrl field — unused
  by SupplyChainPanel, drop from wire).

Gateway wiring:
- server/gateway.ts RPC_CACHE_TIER: list-scenario-templates → 'daily'
  (matches legacy max-age=3600); get-scenario-status → 'slow-browser'
  (premium short-circuit target, explicit entry required by
  tests/route-cache-tier.test.mjs).
- src/shared/premium-paths.ts: swap old run/status for the new
  run-scenario/get-scenario-status paths.
- api/scenario/v1/{run,status,templates}.ts deleted; 3 manifest
  exceptions removed (63 → 52 → 49 migration-pending).

Client:
- src/services/scenario/index.ts — typed client wrapper using
  premiumFetch (injects Clerk bearer / API key).
- src/components/SupplyChainPanel.ts — polling loop swapped from
  premiumFetch strings to runScenario/getScenarioStatus. Hard 20s
  timeout on run preserved via AbortSignal.any.

Tests:
- tests/scenario-handler.test.mjs — 18 new handler-level tests
  covering every security invariant + the worker envelope coercion.
- tests/edge-functions.test.mjs — scenario sections removed,
  replaced with a breadcrumb pointer to the new test file.

Docs: api-scenarios.mdx, scenario-engine.mdx, usage-rate-limits.mdx,
usage-errors.mdx, supply-chain.mdx refreshed with new paths.

Verified: typecheck, typecheck:api, lint:api-contract (49 entries),
lint:rate-limit-policies (6/180), lint:boundaries, route-cache-tier
(parity), full edge-functions (117) + scenario-handler (18).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor(commit 8): migrate /api/v2/shipping/{route-intelligence,webhooks} → ShippingV2Service (koala73#3207)

Partner-facing endpoints promoted to a typed sebuf service. Wire shape
preserved byte-for-byte (camelCase field names, ISO-8601 fetchedAt, the
same subscriberId/secret formats, the same SET + SADD + EXPIRE 30-day
Redis pipeline). Partner URLs /api/v2/shipping/* are unchanged.

RPCs landed:
- GET  /route-intelligence  → RouteIntelligence  (PRO, slow-browser)
- POST /webhooks            → RegisterWebhook    (PRO)
- GET  /webhooks            → ListWebhooks       (PRO, slow-browser)

The existing path-parameter URLs remain on the legacy edge-function
layout because sebuf's HTTP annotations don't currently model path
params (grep proto/**/*.proto for `path: "{…}"` returns zero). Those
endpoints are split into two Vercel dynamic-route files under
api/v2/shipping/webhooks/, behaviorally identical to the previous
hybrid file but cleanly separated:
- GET  /webhooks/{subscriberId}                → [subscriberId].ts
- POST /webhooks/{subscriberId}/rotate-secret  → [subscriberId]/[action].ts
- POST /webhooks/{subscriberId}/reactivate     → [subscriberId]/[action].ts

Both get manifest entries under `migration-pending` pointing at koala73#3207.

Other changes
- scripts/enforce-sebuf-api-contract.mjs: extended GATEWAY_RE to accept
  api/v{N}/{domain}/[rpc].ts (version-first) alongside the canonical
  api/{domain}/v{N}/[rpc].ts; first-use of the reversed ordering is
  shipping/v2 because that's the partner contract.
- vite.config.ts: dev-server sebuf interceptor regex extended to match
  both layouts; shipping/v2 import + allRoutes entry added.
- server/gateway.ts: RPC_CACHE_TIER entries for /api/v2/shipping/
  route-intelligence + /webhooks (slow-browser; premium-gated endpoints
  short-circuit to slow-browser but the entries are required by
  tests/route-cache-tier.test.mjs).
- src/shared/premium-paths.ts: route-intelligence + webhooks added.
- tests/shipping-v2-handler.test.mjs: 18 handler-level tests covering
  PRO gate, iso2/cargoType/hs2 coercion, SSRF guards (http://, RFC1918,
  cloud metadata, IMDS), chokepoint whitelist, alertThreshold range,
  secret/subscriberId format, pipeline shape + 30-day TTL, cross-tenant
  owner isolation, `secret` omission from list response.

Manifest delta
- Removed: api/v2/shipping/route-intelligence.ts, api/v2/shipping/webhooks.ts
- Added:   api/v2/shipping/webhooks/[subscriberId].ts (migration-pending)
- Added:   api/v2/shipping/webhooks/[subscriberId]/[action].ts (migration-pending)
- Added:   api/internal/brief-why-matters.ts (internal-helper) — regression
  surface from the koala73#3248 main merge, which introduced the file without a
  manifest entry. Filed here to keep the lint green; not strictly in scope
  for commit 8 but unblocking.

Net result: 49 → 47 `migration-pending` entries (one net-removal even
though webhook path-params stay pending, because two files collapsed
into two dynamic routes).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review HIGH 1): SupplyChainServiceClient must use premiumFetch (koala73#3207)

Signed-in browser pro users were silently hitting 401 on 8 supply-chain
premium endpoints (country-products, multi-sector-cost-shock,
country-chokepoint-index, bypass-options, country-cost-shock,
sector-dependency, route-explorer-lane, route-impact). The shared
client was constructed with globalThis.fetch, so no Clerk bearer or
X-WorldMonitor-Key was injected. The gateway's validateApiKey runs
with forceKey=true for PREMIUM_RPC_PATHS and 401s before isCallerPremium
is consulted. The generated client's try/catch collapses the 401 into
an empty-fallback return, leaving panels blank with no visible error.

Fix is one line at the client constructor: swap globalThis.fetch for
premiumFetch. The same pattern is already in use for insider-transactions,
stock-analysis, stock-backtest, scenario, trade (premiumClient) — this
was an omission on this client, not a new pattern.

premiumFetch no-ops safely when no credentials are available, so the
5 non-premium methods on this client (shippingRates, chokepointStatus,
chokepointHistory, criticalMinerals, shippingStress) continue to work
unchanged.

This also fixes two panels that were pre-existing latently broken on
main (chokepoint-index, bypass-options, etc. — predating koala73#3207, not
regressions from it). Commit 6 expanded the surface by routing two more
methods through the same buggy client; this commit fixes the class.

From koala73 review (koala73#3242 second-pass, HIGH new #1):
> Exact class PR koala73#3233 fixed for RegionalIntelligenceBoard /
> DeductionPanel / trade / country-intel. Supply-chain was not in
> koala73#3233's scope.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review HIGH 2): restore 400 on input-shape errors for 2 supply-chain handlers (koala73#3207)

Commit 6 collapsed all non-happy paths into empty-200 on
`get-country-products` and `get-multi-sector-cost-shock`, including
caller-bug cases that legacy returned 400 for:

- get-country-products: malformed iso2 → empty 200 (was 400)
- get-multi-sector-cost-shock: malformed iso2 / missing chokepointId /
  unknown chokepointId → empty 200 (was 400)

The commit message for 6 called out the 403-for-non-pro → empty-200
shift ("sebuf gateway pattern is empty-payload-on-deny") but not the
400 shift. They're different classes:

- Empty-payload-200 for PRO-deny: intentional contract change, already
  documented and applied across the service. Generated clients treat
  "you lack PRO" as "no data" — fine.
- Empty-payload-200 for malformed input: caller bug silently masked.
  External API consumers can't distinguish "bad wiring" from "genuinely
  no data", test harnesses lose the signal, bad calling code doesn't
  surface in Sentry.

Fix: `throw new ValidationError(violations)` on the 3 input-shape
branches. The generated sebuf server maps ValidationError → HTTP 400
(see src/generated/server/.../service_server.ts and leads/v1 which
already uses this pattern).

PRO-gate deny stays as empty-200 — that contract shift was intentional
and is preserved.

Regression tests added at tests/supply-chain-validation.test.mjs (8
cases) pinning the three-way contract:
- bad input                         → 400 (ValidationError)
- PRO-gate deny on valid input      → 200 empty
- valid PRO input, no data in Redis → 200 empty (unchanged)

From koala73 review (koala73#3242 second-pass, HIGH new #2).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review HIGH 3): restore statusUrl on RunScenarioResponse + document 202→200 wire break (koala73#3207)

Commit 7 silently shifted /api/scenario/v1/run-scenario's response
contract in two ways that the commit message covered only partially:

1. HTTP 202 Accepted → HTTP 200 OK
2. Dropped `statusUrl` string from the response body

The `statusUrl` drop was mentioned as "unused by SupplyChainPanel" but
not framed as a contract change. The 202 → 200 shift was not mentioned
at all. This is a same-version (v1 → v1) migration, so external callers
that key off either signal — `response.status === 202` or
`response.body.statusUrl` — silently branch incorrectly.

Evaluated options:
  (a) sebuf per-RPC status-code config — not available. sebuf's
      HttpConfig only models `path` and `method`; no status annotation.
  (b) Bump to scenario/v2 — judged heavier than the break itself for
      a single status-code shift. No in-repo caller uses 202 or
      statusUrl; the docs-level impact is containable.
  (c) Accept the break, document explicitly, partially restore.

Took option (c):

- Restored `statusUrl` in the proto (new field `string status_url = 3`
  on RunScenarioResponse). Server computes
  `/api/scenario/v1/get-scenario-status?jobId=<encoded job_id>` and
  populates it on every successful enqueue. External callers that
  followed this URL keep working unchanged.
- 202 → 200 is not recoverable inside the sebuf generator, so it is
  called out explicitly in two places:
    - docs/api-scenarios.mdx now includes a prominent `<Warning>` block
      documenting the v1→v1 contract shift + the suggested migration
      (branch on response body shape, not HTTP status).
    - RunScenarioResponse proto comment explains why 200 is the new
      success status on enqueue.
  OpenAPI bundle regenerated to reflect the restored statusUrl field.

- Regression test added in tests/scenario-handler.test.mjs pinning
  `statusUrl` to the exact URL-encoded shape — locks the invariant so
  a future proto rename or handler refactor can't silently drop it
  again.

From koala73 review (koala73#3242 second-pass, HIGH new #3).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review HIGH 1/2): close webhook tenant-isolation gap on shipping/v2 (koala73#3207)

Koala flagged this as a merge blocker in PR koala73#3242 review.

server/worldmonitor/shipping/v2/{register-webhook,list-webhooks}.ts
migrated without reinstating validateApiKey(req, { forceKey: true }),
diverging from both the sibling api/v2/shipping/webhooks/[subscriberId]
routes and the documented "X-WorldMonitor-Key required" contract in
docs/api-shipping-v2.mdx.

Attack surface: the gateway accepts Clerk bearer auth as a pro signal.
A Clerk-authenticated pro user with no X-WorldMonitor-Key reaches the
handler, callerFingerprint() falls back to 'anon', and every such
caller collapses into a shared webhook:owner:anon:v1 bucket. The
defense-in-depth ownerTag !== ownerHash check in list-webhooks.ts
doesn't catch it because both sides equal 'anon' — every Clerk-session
holder could enumerate / overwrite every other Clerk-session pro
tenant's registered webhook URLs.

Fix: reinstate validateApiKey(ctx.request, { forceKey: true }) at the
top of each handler, throwing ApiError(401) when absent. Matches the
sibling routes exactly and the published partner contract.

Tests:
- tests/shipping-v2-handler.test.mjs: two existing "non-PRO → 403"
  tests for register/list were using makeCtx() with no key, which now
  fails at the 401 layer first. Renamed to "no API key → 401
  (tenant-isolation gate)" with a comment explaining the failure mode
  being tested. 18/18 pass.

Verified: typecheck:api, lint:api-contract (no change), lint:boundaries,
lint:rate-limit-policies, test:data (6005/6005).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review HIGH 2/2): restore v1 path aliases on scenario + supply-chain (koala73#3207)

Koala flagged this as a merge blocker in PR koala73#3242 review.

Commits 6 + 7 of koala73#3207 renamed five documented v1 URLs to the sebuf
method-derived paths and deleted the legacy edge-function files:

  POST /api/scenario/v1/run                       → run-scenario
  GET  /api/scenario/v1/status                    → get-scenario-status
  GET  /api/scenario/v1/templates                 → list-scenario-templates
  GET  /api/supply-chain/v1/country-products      → get-country-products
  GET  /api/supply-chain/v1/multi-sector-cost-shock → get-multi-sector-cost-shock

server/router.ts is an exact static-match table (Map keyed on `METHOD
PATH`), so any external caller — docs, partner scripts, grep-the-
internet — hitting the old documented URL would 404 on first request
after merge. Commit 8 (shipping/v2) preserved partner URLs byte-for-
byte; the scenario + supply-chain renames missed that discipline.

Fix: add five thin alias edge functions that rewrite the pathname to
the canonical sebuf path and delegate to the domain [rpc].ts gateway
via a new server/alias-rewrite.ts helper. Premium gating, rate limits,
entitlement checks, and cache-tier lookups all fire on the canonical
path — aliases are pure URL rewrites, not a duplicate handler pipeline.

  api/scenario/v1/{run,status,templates}.ts
  api/supply-chain/v1/{country-products,multi-sector-cost-shock}.ts

Vite dev parity: file-based routing at api/ is a Vercel concern, so the
dev middleware (vite.config.ts) gets a matching V1_ALIASES rewrite map
before the router dispatch.

Manifest: 5 new entries under `deferred` with removal_issue=koala73#3282
(tracking their retirement at the next v1→v2 break). lint:api-contract
stays green (89 files checked, 55 manifest entries validated).

Docs:
- docs/api-scenarios.mdx: migration callout at the top with the full
  old→new URL table and a link to the retirement issue.
- CHANGELOG.md + docs/changelog.mdx: Changed entry documenting the
  rename + alias compat + the 202→200 shift (from commit 23c821a).

Verified: typecheck:api, lint:api-contract, lint:rate-limit-policies,
lint:boundaries, test:data (6005/6005).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>

…koala73#3358)

* fix(insights): trust cluster rank, stop LLM from re-picking top story

WORLD BRIEF panel published "Iran's new supreme leader was seriously
wounded, leading him to delegate power to the Revolutionary Guards. This
development comes amid an ongoing war with Israel." to every visitor for
3h. Payload: openrouter / gemini-2.5-flash.

Root cause: callLLM sent all 10 clustered headlines with "pick the ONE
most significant and summarize ONLY that story". Clustering ranked
Lebanon journalist killing #1 (2 corroborating sources); News24 Iran
rumor ranked #3 (1 source). Gemini overrode the rank, picked #3, and
embellished with war framing from story #4. Objective rank (sourceCount,
velocity, isAlert) lost to model vibe.

Shrink the LLM's job to phrasing. Clustering already ranks — pass only
topStories[0].primaryTitle and instruct the model to rewrite it using
ONLY facts from the headline. No name/place/context invention.

Also:
- temperature 0.3 -> 0.1 (factual summary, not creative)
- CACHE_TTL 3h -> 30m so a bad brief ages out in one cron cycle
- Drop dead MAX_HEADLINES const

Payload shape unchanged; frontend untouched.

* fix(insights): corroboration gate + revert TTL + drop unconditional WHERE

Follow-up to review feedback on the ranking contract, TTL, and prompt:

1. Corroboration gate (P1a). scoreImportance() in scripts/_clustering.mjs
   is keyword-heavy (violence +125 on a single word, flashpoint +75, ^1.5
   multiplier when both hit), so a single-source sensational rumor can
   outrank a 2-source lead purely on lexical signals. Blindly trusting
   topStories[0] would let the ranker's keyword bias still pick bad
   stories. Walk topStories for sourceCount >= 2 instead — corroboration
   becomes a hard requirement, not a tiebreaker. If no cluster qualifies,
   publish status=degraded with no brief (frontend already handles this).

2. CACHE_TTL back to 10800 (P1b). 30m TTL == one cron cadence means the
   key expires on any missed or delayed run and /api/bootstrap loses
   insights entirely (api/bootstrap.js reads news:insights:v1 directly,
   no LKG across TTL-gap). The short TTL was defense-in-depth for bad
   content; the real safety is now upstream (corroboration gate + grounded
   prompt), so the LKG window doesn't need to be sacrificed for it.

3. Prompt: location conditional (P2). "Use ONLY facts present" + "Lead
   with WHAT happened and WHERE" conflicted for headlines without an
   explicit location and pushed the model toward inferred-place
   hallucination. Replaced with "Include a location, person, or
   organization ONLY if it appears in the headline."

* test(insights): lock corroboration gate + grounded-prompt invariants

Review P2: the corroboration gate and the prompt's no-invention rules
had no tests, so future edits to selectTopStories() ordering or prompt
text could silently reintroduce the original hallucination.

Extract the brief-selection helper and prompt builders into a pure
module (scripts/_insights-brief.mjs) so tests can import them without
triggering seed-insights.mjs's top-level runSeed() call:

- pickBriefCluster(topStories) returns first sourceCount>=2 cluster
- briefSystemPrompt(dateISO) returns the system prompt
- briefUserPrompt(headline) returns the user prompt

Regression tests (tests/seed-insights-brief.test.mjs, 12 cases) lock:
- pickBriefCluster skips single-source rumors even when ranked above a
  multi-sourced lead (explicit regression: News24 Iran supreme leader
  2026-04-23 scenario with realistic scores)
- pickBriefCluster tolerates missing/null entries
- briefSystemPrompt forbids invented facts and proper nouns
- briefSystemPrompt's "location" rule is conditional (no unconditional
  "Lead with WHAT and WHERE" directive that would push the model toward
  place-inference when the headline has no location)
- briefSystemPrompt does not contain "pick the most important" style
  language (ranking is done by pickBriefCluster upstream)
- briefUserPrompt passes the headline verbatim and instructs
  "only facts from this headline"

Also fix a misleading comment on CACHE_TTL: corroboration is gated at
brief-selection time, not on the topStories payload itself (which still
includes single-source clusters rendered as the headline list).

test:data: 6657/6657 pass (was 6645; +12).

… B) (koala73#3366)

* feat(energy-atlas): promote Atlas map layers to FULL variant (§R #3 = B)

Per plan §R/#3 decision B: the Redis-backed evidence registries
(75 gas + 75 oil pipelines, 200 storage facilities, 29 fuel shortages)
are now toggleable on the main worldmonitor.app map. Previously they
were hardcoded energy-variant-only, and FULL users who toggled
`pipelines: true` got the ~20-entry legacy static PIPELINES list.

Changes:
- `src/components/DeckGLMap.ts`: drop the `SITE_VARIANT === 'energy'`
  gates at :1511-1541. The pipelines layer now always uses
  `createEnergyPipelinesLayer()` (Redis-backed evidence registry);
  `createPipelinesLayer` (legacy static) is left in the file as dead
  code pending a separate cleanup PR that also retires
  `src/config/pipelines.ts`. Storage and fuel-shortage layers are
  now gated only on the variant's `mapLayers.storageFacilities` /
  `mapLayers.fuelShortages` booleans.
- `src/config/panels.ts`: add `storageFacilities: false` +
  `fuelShortages: false` to FULL_MAP_LAYERS (desktop + mobile) so
  the keys exist for toggle dispatch; default off so users opt in.
- `src/config/map-layer-definitions.ts`: extend the `full` variant's
  VARIANT_LAYER_ORDER to include `storageFacilities` and
  `fuelShortages`, so `getAllowedLayerKeys('full')` admits them and
  the layer picker surfaces them.
- `src/config/commands.ts`: add CMD+K toggles
  `layer:storageFacilities` and `layer:fuelShortages` next to the
  existing `layer:pipelines`.

Finance + commodity variants already had `pipelines: true`; they
now render the more comprehensive Redis-backed 150-entry dataset
instead of the ~20-entry legacy list. If a variant doesn't want
this, they set `pipelines: false` in their MAP_LAYERS config.

Part of docs/internal/energy-atlas-registry-expansion.md §R.

* fix(energy-atlas): restrict storageFacilities + fuelShortages to flat renderer

Reviewer (Codex) found two gaps in PR koala73#3366:

1. GlobeMap 3D toggles did nothing. LAYER_REGISTRY declared both new
   layers with the default ['flat', 'globe'] renderers, so the toggle
   showed up in globe mode. But GlobeMap.ts has no rendering support:
   ensureStaticDataForLayer (:2160) only handles cables/pipelines/etc.,
   and the layer-channel map (:2484) has no entries for either. Users
   in globe mode saw the toggle and got silent no-ops.

2. SVG/mobile fallback (Map.ts fullLayers at :381) also has no render
   path for these data types. The existing cyberThreats precedent at
   :387 documents this as an intentional DeckGL-only pattern.

Fix:
- Restrict both LAYER_REGISTRY entries to ['flat'] explicitly. The
  layer picker hides the toggle in globe mode instead of exposing a
  no-op. Comment points to the GlobeMap gap so a future globe-rendering
  PR knows what to undo.
- Extend the existing cyberThreats note in Map.ts:387 to cover
  storageFacilities + fuelShortages too, noting they're already
  hidden from globe mode via the LAYER_REGISTRY restriction.

This is the smallest possible fix consistent with the pre-existing
pattern. Full globe-mode rendering for these layers is out of scope —
tracked separately as a follow-up.

* fix(energy-atlas): gate layer:* CMD+K by current renderer + DeckGL state

Reviewer follow-up on PR koala73#3366: the previous fix restricted
LAYER_REGISTRY renderers to ['flat'] so the globe-mode layer picker
hides storageFacilities / fuelShortages toggles. But CMD+K was still
callable — SearchModal.matchCommands didn't filter `layer:*` commands
by renderer, so a user could CMD+K "storage layer" in globe or SVG
mode and trigger a silent no-op.

Fix — centralize "can this layer render right now?" in one helper:

- Add `deckGLOnly?: boolean` to LayerDefinition. `renderers: ['flat']`
  is not enough because `'flat'` covers both DeckGL-flat and SVG-flat,
  and the SVG/mobile fallback has no render path for either layer.
  Mark both as `deckGLOnly: true`.
- New `isLayerExecutable(key, renderer, isDeckGLActive)` helper in
  map-layer-definitions.ts. Returns true iff renderers include the
  current renderer AND (if deckGLOnly) DeckGL is active.
- `SearchModal.setLayerExecutableFn(fn)`: caller-supplied predicate
  used in both `matchCommands` (search results) and
  `renderAllCommandsList` (full picker).
- `search-manager` wires the predicate using `ctx.map.isGlobeMode()`
  + `ctx.map.isDeckGLActive()`, and also adds a symmetric guard in
  the `layer:` dispatch case so direct activations (keyboard
  accelerator, programmatic invocation) bail the same way.

Pre-existing resilienceScore DeckGL gate at search-manager:494 kept as
a belt-and-suspenders — the new isLayerExecutable check already
covers it since resilienceScore has `renderers: ['flat']` (though it
lacks deckGLOnly). Left the specific check in place to avoid scope
creep on a working guard.

Typecheck clean, 6694/6694 tests pass.

* fix(energy-atlas): filter CMD+K layer commands by variant too

Greptile P2 on commit 3f7a400: `layer:storageFacilities` and
`layer:fuelShortages` still surface in CMD+K on tech / finance /
commodity / happy variants (where they're not in VARIANT_LAYER_ORDER).
Renderer + DeckGL filter was passing because those variants run flat
DeckGL. Dispatch silently failed at the `variantAllowed` guard in
handleCommand (:491), producing an invisible no-op from the user's
POV.

Fix: extend `setLayerExecutableFn` predicate to also check
`getAllowedLayerKeys(SITE_VARIANT).has(key)` before the renderer
checks. SearchModal now hides these commands on non-full/non-energy
variants where they can't execute.

This also cleans up the pre-existing pattern for other
variant-specific layer commands flagged by Greptile as "consistent
with how other variant-specific layer commands (e.g. layer:nuclear
on tech variant) already behave today" — they now all route through
the same predicate.

* fix(energy-atlas): gate layers:* presets + add isLayerExecutable tests (review P2)

Two Codex P2 findings on this PR:

1. `layers:*` presets bypassed the renderer/DeckGL gate.
   `search-manager.ts:481` checked only `allowed.has(layer)` before
   flipping a preset layer on. A user in globe mode or on SVG
   fallback who ran `layers:all` or `layers:infra` would silently
   set `deckGLOnly` layers (storageFacilities, fuelShortages) to
   true — toggles with no rendered output, and since the picker
   hides those layers under the current renderer the user had no
   way to toggle them back off without switching modes.

   Fix: funnel presets through the same `isLayerExecutable`
   predicate per-layer CMD+K already uses. `executable(k)` combines
   the existing `allowed.has` variant check with the renderer + DeckGL
   gate, so presets now match the per-layer dispatch behavior exactly.

2. No regression tests for the `deckGLOnly` / `isLayerExecutable`
   contract, despite it being behavior-critical renderer gating.

   Fix: added `tests/map-layer-executable.test.mts` — 16 cases:
   - Flag assertions: storageFacilities + fuelShortages carry
     `deckGLOnly: true` and renderers: ['flat']. Layers without the
     flag (pipelines, conflicts, cables) have it `undefined`, not
     accidentally `false`.
   - Renderer-gate cases: deckGLOnly layers pass only on flat + DeckGL
     active, not on SVG fallback, not on globe. Flat-only non-deckGLOnly
     layers (ciiChoropleth) pass on flat regardless of DeckGL status.
     Dual-renderer layers (pipelines) pass on both flat and globe.
     Unknown layer keys return false.
   - Exhaustive 2×2×2 matrix across (renderer, isDeckGL, deckGLOnly)
     using representative layer keys for each shape.

All 16 new tests pass. Full test:data suite still green. Typecheck clean.

* fix(energy-atlas): add pipeline-status to finance + commodity panel sets (review P1)

Codex P1: FINANCE_MAP_LAYERS and COMMODITY_MAP_LAYERS both carry
`pipelines: true`, and PR koala73#3366 unified all variants on
`createEnergyPipelinesLayer` which dispatches
`energy:open-pipeline-detail` on row click. The listener for that
event lives in PipelineStatusPanel.

`PanelLayoutManager.createPanel()` only instantiates panels whose keys
are present in `panelSettings`, which derives from FULL_PANELS /
FINANCE_PANELS / etc. — so on finance and commodity variants the
listener never existed, and pipeline clicks were a silent no-op.

Fix: add `pipeline-status` to both FINANCE_PANELS and COMMODITY_PANELS
with `enabled: false` (panel slot not auto-opened; users invoke it by
clicking a pipeline on the map or via CMD+K). The panel now
instantiates on both variants and the click-through works end to end.

FULL_PANELS + ENERGY_PANELS already had the key from earlier PRs;
no change there.

Typecheck clean, test:data 6696/6696 pass.

…read (koala73#3385)

* feat(seed): BUNDLE_RUN_STARTED_AT_MS env + runSeed SIGTERM cleanup

Prereq for the re-export-share Comtrade seeder (plan 2026-04-24-003),
usable by any cohort seeder whose consumer needs bundle-level freshness.

Two coupled changes:

1. `_bundle-runner.mjs` injects `BUNDLE_RUN_STARTED_AT_MS` into every
   spawned child. All siblings in a single bundle run share one value
   (captured at `runBundle` start, not spawn time). Consumers use this
   to detect stale peer keys — if a peer's seed-meta predates the
   current bundle run, fall back to a hard default rather than read
   a cohort-peer's last-week output.

2. `_seed-utils.mjs::runSeed` registers a `process.once('SIGTERM')`
   handler that releases the acquired lock and extends existing-data
   TTL before exiting 143. `_bundle-runner.mjs` sends SIGTERM on
   section timeout, then SIGKILL after KILL_GRACE_MS (5s). Without
   this handler the `finally` path never runs on SIGKILL, leaving
   the 30-min acquireLock reservation in place until its own TTL
   expires — the next cron tick silently skips the resource.

Regression guard memory: `bundle-runner-sigkill-leaks-child-lock` (PR
koala73#3128 root cause).

Tests added:
- bundle-runner env injection (value within run bounds)
- sibling sections share the same timestamp (critical for the
  consumer freshness guard)
- runSeed SIGTERM path: exit 143 + cleanup log
- process.once contract: second SIGTERM does not re-enter handler

* fix(seed): address P1/P2 review findings on SIGTERM + bundle contracts

Addresses PR koala73#3384 review findings (todos 256, 257, 259, 260):

koala73#256 (P1) — SIGTERM handler narrowed to fetch phase only. Was installed
at runSeed entry and armed through every `process.exit` path; could
race `emptyDataIsFailure: true` strict-floor exits (IMF-External,
WB-bulk) and extend seed-meta TTL when the contract forbids it —
silently re-masking 30-day outages. Now the handler is attached
immediately before `withRetry(fetchFn)` and removed in a try/finally
that covers all fetch-phase exit branches.

koala73#257 (P1) — `BUNDLE_RUN_STARTED_AT_MS` now has a first-class helper.
Exported `getBundleRunStartedAtMs()` from `_seed-utils.mjs` with JSDoc
describing the bundle-freshness contract. Fleet-wide helper so the
next consumer seeder imports instead of rediscovering the idiom.

koala73#259 (P2) — SIGTERM cleanup runs `Promise.allSettled` on disjoint-key
ops (`releaseLock` + `extendExistingTtl`). Serialising compounded
Upstash latency during the exact failure mode (Redis degraded) this
handler exists to handle, risking breach of the 5s SIGKILL grace.

koala73#260 (P2) — `_bundle-runner.mjs` asserts topological order on
optional `dependsOn` section field. Throws on unknown-label refs and
on deps appearing at a later index. Fleet-wide contract replacing
the previous prose-comment ordering guarantee.

Tests added/updated:
- New: SIGTERM handler removed after fetchFn completes (narrowed-scope
  contract — post-fetch SIGTERM must NOT trigger TTL extension)
- New: dependsOn unknown-label + out-of-order + happy-path (3 tests)

Full test suite: 6,866 tests pass (+4 net).

* fix(seed): getBundleRunStartedAtMs returns null outside a bundle run

Review follow-up: the earlier `Math.floor(Date.now()/1000)*1000` fallback
regressed standalone (non-bundle) runs. A consumer seeder invoked
manually just after its peer wrote `fetchedAt = (now - 5s)` would see
`bundleStartMs = Date.now()`, reject the perfectly-fresh peer envelope
as "stale", and fall back to defaults — defeating the point of the
peer-read path outside the bundle.

Returning null when `BUNDLE_RUN_STARTED_AT_MS` is unset/invalid keeps
the freshness gate scoped to its real purpose (across-bundle-tick
staleness) and lets standalone runs skip the gate entirely. Consumers
check `bundleStartMs != null` before applying the comparison; see the
companion `seed-sovereign-wealth.mjs` change on the stacked PR.

* test(seed): SIGTERM cleanup test now verifies Redis DEL + EXPIRE calls

Greptile review P2 on PR koala73#3384: the existing test only asserted exit
code + log line, not that the Redis ops were actually issued. The
log claim was ahead of the test.

Fixture now logs every Upstash fetch call's shape (EVAL / pipeline-
EXPIRE / other) to stderr. Test asserts:

- >=1 EVAL op was issued during SIGTERM cleanup (releaseLock Lua
  script on the lock key)
- >=1 pipeline-EXPIRE op was issued (extendExistingTtl on canonical
  + seed-meta keys)
- The EVAL body carries the runSeed-generated runId (proves it's
  THIS run's release, not a phantom op)
- The EXPIRE pipeline touches both the canonicalKey AND the
  seed-meta key (proves the keys[] array was built correctly
  including the extraKeys merge path)

Full test suite: 6,866 tests pass, typecheck clean.

* feat(resilience): Comtrade-backed re-export-share seeder + SWF Redis read

Plan ref: docs/plans/2026-04-24-003-feat-reexport-share-comtrade-seeder-plan.md

Motivating case. Before this PR, the SWF `rawMonths` denominator for
the `sovereignFiscalBuffer` dimension used GROSS annual imports for
every country. For re-export hubs (goods transiting without domestic
settlement), this structurally under-reports resilience: UAE's 2023
$941B of imports include $334B of transit flow that never represents
domestic consumption. Net imports = gross × (1 − reexport_share).

The previous (PR 3A) design flattened a hand-curated YAML into Redis;
the YAML shipped empty and never populated, so the correction never
applied and the cohort audit showed no movement.

Gap #2 (this PR). Two coupled changes to make the correction actually
apply:

1. Comtrade-backed seeder (`scripts/seed-recovery-reexport-share.mjs`).
   Rewritten to fetch UN Comtrade `flowCode=RX` (re-exports) and
   `flowCode=M` (imports) per cohort member, compute share = RX/M at
   the latest co-populated year, clamp to [0.05, 0.95], publish the
   envelope. Header auth (`Ocp-Apim-Subscription-Key`) — subscription
   key never reaches URL/logs/Redis. `maxRecords=250000` cap with
   truncation detection. Sequential + retry-on-429 with backoff.

   Hub cohort resolved by Phase 0 empirical probe (plan §Phase 0):
   ['AE', 'PA']. Six candidates (SG/HK/NL/BE/MY/LT) return HTTP 200
   with zero RX rows — Comtrade doesn't expose RX for those reporters.

2. SWF seeder reads from Redis (`scripts/seed-sovereign-wealth.mjs`).
   Swaps `loadReexportShareByCountry()` (YAML) for
   `loadReexportShareFromRedis()` (Redis key written by #1). Guarded
   by bundle-run freshness: if the sibling Reexport-Share seeder's
   `seed-meta` predates `BUNDLE_RUN_STARTED_AT_MS` (set by the
   prereq PR's `_bundle-runner.mjs` env-injection), HARD fallback
   to gross imports rather than apply last-month's stale share.

Health registries. Both new keys registered in BOTH `api/health.js`
SEED_META (60-day alert threshold) and `api/seed-health.js`
SEED_DOMAINS (43200min interval). feedback_two_health_endpoints_must_match.

Bundle wiring. `seed-bundle-resilience-recovery` Reexport-Share
timeout bumped 60s → 300s (Comtrade + retry can take 2-3 min
worst-case). Ordering preserved: Reexport-Share before Sovereign-
Wealth so the SWF seeder reads a freshly-written key in the same
cron tick.

Deletions. YAML + loader + 7 obsolete loader tests removed; single
source of truth is now Comtrade → Redis.

Prereq. Stacks on PR koala73#3384 (feat/bundle-runner-env-sigterm)
which adds BUNDLE_RUN_STARTED_AT_MS env injection + runSeed
SIGTERM cleanup. This PR's bundle-freshness guard depends on
that env variable.

Tests (19 new, 7 deleted, +12 net):
- Pure math: parseComtradeFlowResponse, computeShareFromFlows,
  clampShare, declareRecords + credential-leak source scan (15)
- Integration (Gap #2 regression guards): SWF seeder loadReexport
  ShareFromRedis — fresh/absent/malformed/stale-meta/missing-meta (5)
- Health registry dual-registry drift guard — scoped to this PR's
  keys, respecting pre-existing asymmetry (4)
- Bundle-ordering + timeout assertions (2)

Phase 0 cohort validation committed to plan. Full test suite
passes: 6,881 tests.

* fix(resilience): address P1/P2 review findings — adopt shared helpers, pin freshness boundary

Addresses PR koala73#3385 review findings:

koala73#257 (P1) consumer — `seed-sovereign-wealth.mjs` imports the shared
`getBundleRunStartedAtMs` helper from `_seed-utils.mjs` (added in the
prereq commit) instead of its own `getBundleStartMs`. Single source of
truth for the bundle-freshness contract.

koala73#258 (P2) — `seed-recovery-reexport-share.mjs` isMain guard uses the
canonical `pathToFileURL(process.argv[1]).href === import.meta.url`
form instead of basename-suffix matching. Handles symlinks, case-
different paths on macOS HFS+, and Windows path separators without
string munging.

koala73#260 (P2) consumer — Sovereign-Wealth declares `dependsOn:
['Reexport-Share']` in the bundle spec. `_bundle-runner.mjs` (prereq
commit) now enforces topological order on load and throws on
violation — replaces the previous prose-comment ordering contract.

koala73#261 (P2) — added a test to `tests/seed-sovereign-wealth-reads-redis-
reexport-share.test.mts` pinning the inclusive-boundary semantic:
`fetchedAtMs === bundleStartMs` must be treated as FRESH. Guards
against a future refactor to `<=` that would silently reject peers
writing at the very first millisecond of the bundle run.

Rebased onto updated prereq. Full test suite: 6,886 tests pass (+5 net).

* fix(resilience): freshness gate skipped in standalone mode; meta still required

Review catch: the previous `bundleStartMs = Date.now()` fallback made
standalone/manual `seed-sovereign-wealth.mjs` runs ALWAYS reject any
previously-seeded re-export-share meta as "stale" — even when the
operator ran the Reexport seeder milliseconds beforehand. Defeated
the point of the peer-read path outside the bundle.

With `getBundleRunStartedAtMs()` now returning null outside a bundle
(companion commit on the prereq branch), the consumer only applies
the freshness gate when `bundleStartMs != null`. Standalone runs
accept any `fetchedAt` — the operator is responsible for ordering.

Two guards survive the change:
- Meta MUST exist (absence = peer-outage fail-safe, both modes)
- In-bundle: meta MUST be at or after `BUNDLE_RUN_STARTED_AT_MS`

Two new tests pin both modes:
- standalone: accepts meta written 10 min before this process started
- standalone: still rejects missing meta (peer-outage fail-safe
  survives gate bypass)

Rebased onto updated prereq. Full test suite: 6,888 tests (+2 net).

* fix(resilience): filter world-aggregate Comtrade rows + skip final-retry sleep

Greptile review of PR koala73#3385 flagged two P2s in the Comtrade seeder.

Finding #3 (parseComtradeFlowResponse double-count risk):
`cmdCode=TOTAL` without a partner filter currently returns only
world-aggregate rows in practice — but `parseComtradeFlowResponse`
summed every row unconditionally. A future refactor adding per-
partner querying would silently double-count (world-aggregate row +
partner-level rows for the same year), cutting the derived share in
half with no test signal.

Fix: explicit `partnerCode ∈ {'0', 0, null/undefined}` filter. Matches
current empirical behavior (aggregate-only responses) and makes the
construct robust to a future partner-level query.

Finding #4 (wasted backoff on final retry):
429 and 5xx branches slept `backoffMs` before `continue`, but on
`attempt === RETRY_MAX_ATTEMPTS` the loop condition fails immediately
after — the sleep was pure waste. Added early-return (parallel to the
existing pattern in the network-error catch branch) so the final
attempt exits the retry loop at the first non-success response
without extra latency.

Tests:
- 3 new `parseComtradeFlowResponse` variants: world-only filter,
  numeric-0 partnerCode shape, rows without partnerCode field
- Existing tests updated: the double-count assertion replaced with
  a "per-partner rows must NOT sum into the world-aggregate total"
  assertion that pins the new contract

Rebased onto updated prereq. Full test suite: 6,890 tests (+2 net).

…plan U1-U4) (koala73#3397)

* feat(energy-atlas): GEM pipeline import infrastructure (PR 1, plan U1-U4)

Lands the parser, dedup helper, validator extensions, and operator runbook
for the Global Energy Monitor (CC-BY 4.0) pipeline-data refresh — closing
~3.6× of the Energy Atlas pipeline-scale gap once the operator runs the
import.

Per docs/plans/2026-04-25-003-feat-energy-parity-pushup-plan.md PR 1.

U1 — Validator + schema extensions:
- Add `'gem'` to VALID_SOURCES in scripts/_pipeline-registry.mjs and to the
  evidence-bearing-source whitelist in derivePipelinePublicBadge so GEM-
  sourced offline rows derive a `disputed` badge via the external-signal
  rule (parity with `press`/`satellite`/`ais-relay`).
- Export VALID_SOURCES so tests assert against the same source-of-truth
  the validator uses (matches the VALID_OIL_PRODUCT_CLASSES pattern from
  PR koala73#3383).
- Floor bump (MIN_PIPELINES_PER_REGISTRY 8→200) intentionally DEFERRED
  to the follow-up data PR — bumping it now would gate the existing 75+75
  hand-curated rows below the new floor and break seeder publishes
  before the GEM data lands.

U2 — GEM parser (test-first):
- scripts/import-gem-pipelines.mjs reads a local JSON file (operator pre-
  converts GEM Excel externally — no `xlsx` dependency added). Schema-
  drift sentinel throws on missing columns. Status mapping covers
  Operating/Construction/Cancelled/Mothballed/Idle/Shut-in. ProductClass
  mapping covers Crude Oil / Refined Products / mixed-flow notes.
  Capacity-unit conversion handles bcm/y, bbl/d, Mbd, kbd.
- 22 tests in tests/import-gem-pipelines.test.mjs cover schema sentinel,
  fuel split, status mapping, productClass mapping, capacity conversion,
  minimum-viable-evidence shape, registry-shape conformance, and bad-
  coordinate rejection.

U3 — Deduplication (pure deterministic):
- scripts/_pipeline-dedup.mjs: dedupePipelines(existing, candidates) →
  { toAdd, skippedDuplicates }. Match rule: haversine ≤5km AND name
  Jaccard ≥0.6 (BOTH required). Reverse-direction-pair-aware.
- 19 tests cover internal helpers, match logic, id collision, determinism,
  and empty inputs.

U4 — Operator runbook (data import deferred):
- docs/methodology/pipelines.mdx: 7-step runbook for the operator to
  download GEM, pre-convert Excel→JSON, dry-run with --print-candidates,
  merge with --merge, bump the registry floor, and commit with
  provenance metadata.
- The actual data import is intentionally OUT OF SCOPE for this agent-
  authored PR because GEM downloads are registration-gated. A follow-up
  PR will commit the imported scripts/data/pipelines-{gas,oil}.json +
  bump MIN_PIPELINES_PER_REGISTRY → 200 + record the GEM release SHA256.

Tests: typecheck clean; 67 tests pass across the three test files.

Codex-approved through 8 review rounds against origin/main @ 0500733.

* fix(energy-atlas): wire --merge to dedupePipelines + within-batch dedup (PR1 review)

P1 — --merge was a TODO no-op (import-gem-pipelines.mjs:291):
- Previously exited with code 2 + a "TODO: wire dedup once U3 lands"
  message. The PR body and the methodology runbook both advertised
  --merge as the operator path.
- Add mergeIntoRegistry(filename, candidates) helper that loads the
  existing envelope, runs dedupePipelines() against the candidate
  list, sorts new entries alphabetically by id (stable diff on rerun),
  validates the merged registry via validateRegistry(), and writes
  to disk only after validation passes. CLI --merge now invokes it
  for both gas and oil + prints a per-fuel summary.
- Source attribution: the registry envelope's `source` field is
  upgraded to mention GEM (CC-BY 4.0) on first merge so the data file
  itself documents provenance.

P2 — dedup transitive-match bug (_pipeline-dedup.mjs:120):
- Pre-fix loop checked each candidate ONLY against the original
  `existing` array. Two GEM rows that match each other but not anything
  in `existing` would BOTH be added, defeating the dedup contract for
  same-batch duplicates (real example: a primary GEM entry plus a
  duplicate row from a regional supplemental sheet).
- Now compares against existing FIRST (existing wins on cross-set
  match — preserves richer hand-curated evidence), then falls back to
  the already-accepted toAdd set. Within-batch matches retain the FIRST
  accepted candidate (deterministic by candidate-list order).

Tests: 22 in tests/pipeline-dedup.test.mjs (3 new) cover the
within-batch dedup, transitive collapse, and existing-wins-over-
already-accepted scenarios. typecheck clean.

* fix(energy-atlas): cross-file-atomic --merge (PR1 review #2)

P1 — partial-import on disk if oil validation fails after gas writes
(import-gem-pipelines.mjs:329 / :350):
- Previous flow ran `mergeIntoRegistry('pipelines-gas.json', gas)` which
  wrote to disk, then `mergeIntoRegistry('pipelines-oil.json', oil)`. If
  oil validation failed, the operator was left with a half-imported
  state: gas had GEM rows committed to disk but oil didn't.
- Refactor into a two-phase API:
  1. prepareMerge(filename, candidates) — pure, no disk I/O. Builds the
     merged envelope, validates it, throws on validation failure.
  2. mergeBothRegistries(gasCandidates, oilCandidates) — calls
     prepareMerge for BOTH fuels first; only writes to disk after BOTH
     pass validation. If oil's prepareMerge throws, gas was never
     touched on disk.
- CLI --merge now invokes mergeBothRegistries. The atomicity guarantee
  is documented inline in the helper.

typecheck clean. No new tests because the existing dedup + validate
suites cover the underlying logic; the change is purely about call
ordering for atomicity.

* fix(energy-atlas): deterministic lastEvidenceUpdate + clarify test comment (PR1 review #3)

P2 — lastEvidenceUpdate was non-deterministic (Greptile P2):
- Previous code used new Date().toISOString() per parser run, so two runs
  of parseGemPipelines on the same input on different days produced
  byte-different output. Quarterly re-imports would produce noisy
  full-row diffs even when the upstream GEM data hadn't changed.
- New: resolveEvidenceTimestamp(envelope) derives the timestamp from
  envelope.downloadedAt (the operator-recorded date) or sourceVersion
  if it parses as ISO. Falls back to 1970-01-01 sentinel when neither
  is set — deliberately ugly so reviewers spot the missing field in
  the data file diff rather than getting silent today's date.
- Computed once per parse run so every emitted candidate gets the
  same timestamp.

P2 — misleading test comment (Greptile P2):
- Comment in tests/import-gem-pipelines.test.mjs:136 said "400_000 bbl/d
  ÷ 1000 = 400 Mbd" while the assertion correctly expects 0.4 (because
  the convention is millions, not thousands). Rewrote the comment to
  state the actual rule + arithmetic clearly.

3 new tests for determinism: (a) two parser runs produce identical
output, (b) timestamp derives from downloadedAt, (c) missing date
yields the epoch sentinel (loud failure mode).

…e mode (koala73#3422)

* fix(brief-ingest): READ-time freshness + direct-RSS migration + U6 age mode

Three coupled gap closures discovered after PR koala73#3419 merged. Brief
2026-04-26-0802 still surfaced two months-old Pentagon items DESPITE
PR koala73#3417's `when:1d` ingest gate working perfectly (verified live: the
gated query returns zero Pentagon items in the last 24h). The cause was
post-deploy residue, not the gate. See:
  skill: ingest-gate-tightening-leaves-residue-in-read-path

Fixes (all on the same PR because they share the publishedAt persistence
precondition):

1. READ-time freshness floor in buildDigest (seed-digest-notifications.mjs).
   Drops story:track:v1 rows whose source publishedAt is older than
   DIGEST_READ_MAX_AGE_HOURS (default 48h). Closes the residue window: when
   any future ingest-side gate is tightened, pre-deploy entries stop
   shipping in briefs even before they TTL-expire from the accumulator.
   Rows missing publishedAt (legacy pre-this-PR entries) fall through —
   back-compat. Operator kill switch: DIGEST_READ_MAX_AGE_HOURS=999.

2. Persist publishedAt in story:track:v1 HSET (list-feed-digest.ts).
   Adds 'publishedAt', String(item.publishedAt) to buildStoryTrackHsetFields.
   Required for #1 AND for U6's age-mode (#3) to actually find the field.
   Pre-existing rows pick it up on their next mention.
   Test: tests/news-story-track-description-persistence.test.mts asserts
   the field is emitted as a numeric string that round-trips through
   parseInt.

3. U6 audit gains --mode=age|url|both (audit-static-page-contamination.mjs).
   Original URL-pattern classifier is BLIND to Google-News-routed entries
   (track.link is news.google.com/rss/articles/CBMi opaque redirect).
   --mode=age matches by track.publishedAt > max-age-hours, the right
   primary signal for evicting post-deploy residue. Per-reason rollup
   surfaces url vs age hits. Operator runbook included in the script
   header. Default --mode=url is back-compat.

4. Feed swap: 3 of 5 gov sources to direct first-party RSS (_feeds.ts).
   Verified 2026-04-26 via live curl + end-to-end parseRssXml smoke test:
     - Pentagon -> https://www.war.gov/DesktopModules/ArticleCS/RSS.ashx?ContentType=1&Site=945
     - White House (briefings) -> /briefings-statements/feed/
     - White House Actions    -> /presidential-actions/feed/ (NEW entry)
   Direct RSS means the publisher's pubDate is authoritative — no
   re-indexing of months-old PDFs creating fresh-looking Google entries.
   State Dept, Treasury, DOJ retain gn(...) with an explanatory comment;
   no working public RSS at any verified path, Federal Register fallback
   bot-blocked. The READ-time freshness floor (#1) mitigates the residue
   gap for those.

5. source-tiers.json: White House Actions = 1 (with scripts/shared mirror).
   New feed gets the same tier-1 source-boost as the existing White House
   entry; without it, Actions items would default to tier-4 with no boost.

250/250 tests pass; importance-score-parity preserved.

Operator runbook for the immediate post-deploy residue:
  1. Wait ~1h for cron tick to start writing publishedAt on existing
     entries via HSET re-mention.
  2. Dry run: node scripts/audit-static-page-contamination.mjs --mode=age
  3. Inspect output, confirm matched titles look stale.
  4. Apply: --mode=age --apply

* fix: address review findings — window-aware floor + residue audit mode (P1×2 + P2/P3)

5 fixes responding to review:

P1 (reviewer): READ-time floor + audit --mode=age both fall through on
missing publishedAt — the legacy state of EVERY pre-PR-3422 row. So
the actual residue this PR was meant to evict (Pentagon items in brief
2026-04-26-0802) wasn't catchable by either gate. Add --mode=residue
that matches rows with missing/unparseable publishedAt — the explicit
opt-in cleanup signal for the post-deploy one-shot. Operator runbook
updated: wait at least 1 cron cycle (so still-active rows have publishedAt
re-mentioned in via HSET), then run --mode=residue --apply.

P1 (self-review): READ-time floor was hardcoded 48h, breaking weekly
users (their 7d window stories all dropped). Replaced with
readTimeAgeCutoffMs(windowStartMs) = windowStart - 24h buffer. Daily
user (24h window) -> 48h cutoff (matches old behavior); weekly user
(7d window) -> 8d cutoff (the broken case). DIGEST_READ_MAX_AGE_HOURS
env var removed — there's no scenario where an operator wants to drop
items younger than the rule's intended window.

P2: Defensive cast in buildStoryTrackHsetFields. Number.isFinite guard
on item.publishedAt prevents the literal "undefined"/"NaN" string from
ever being persisted; degraded input writes '' which the read-side
parseInt treats as legacy back-compat (fall through, not mis-classify
as a stale-row).

P2: Extract shouldDropTrackByAge + readTimeAgeCutoffMs into
scripts/lib/digest-orchestration-helpers.mjs (the existing module for
this purpose). 6 new unit tests cover the predicate matrix
(within/at-cutoff/before/missing/unparseable/zero/null-track) plus 2
integration cases that reproduce the exact failure modes:
  - weekly user with 5d-old story is KEPT (window-aware vs naive 48h)
  - 4-month-old residue (Jan 2026 Pentagon PDF) is DROPPED for daily user

P3: Audit parseArgs now rejects unknown args (--mode age space-not-equals)
with exit-2 + clear error message. classifyTrack refactored to a pure
function (mode + maxAgeMs as options) so tests can exercise it without
the script's module-load argv side effects. main() invocation gated
behind import.meta.url === file://process.argv[1] check (standard
ESM idiom). 10 new unit tests cover the url/age/residue/both modes
+ parseArgs flag handling.

316/316 tests pass; importance-score-parity preserved.

* fix(audit): residue mode safety gate + EOF whitespace (review fixes on koala73#3422)

P2 (reviewer): --mode=residue --apply was too broad. Classifier matched
ANY row with missing publishedAt — but every pre-PR-3422 row lacks the
field, so the recommended one-shot cleanup could delete legitimate recent
stories that simply hadn't been re-mentioned in the first post-deploy
cron cycle.

Fix: residue mode now requires BOTH conditions:
  1. publishedAt missing/unparseable (legacy or anomalous), AND
  2. lastSeen >= residueMinStaleMs ago (default 24h).

Active stories get HSET-touched on every re-mention, so a row that
hasn't been touched in 24h is genuinely abandoned — the new ingest gate
(when:1d) is correctly excluding it. Fresh-lastSeen rows are PROTECTED
from deletion.

New flag: --residue-min-stale-hours=N (default 24, positive-int only).
Operator runbook updated:
  1. Wait >=24h post-deploy.
  2. Dry run: node scripts/audit-static-page-contamination.mjs --mode=residue
  3. Inspect output, confirm rows are stale + look like residue.
  4. Apply: --mode=residue --apply

Defensive fall-throughs:
  - Missing lastSeen: treat as ancient (errs toward eviction). Acceptable
    in this opt-in destructive mode; rows without lastSeen are anomalous.
  - publishedAt present (any value): NOT residue (caught by --mode=age).

Tests:
  - 8 residue-mode cases incl. the explicit P2 fix ("fresh lastSeen must
    protect the row"), boundary at exactly the threshold, missing-lastSeen
    treated as ancient.
  - 4 parseArgs cases for the new --residue-min-stale-hours flag.

P3 (reviewer): tests/digest-orchestration-helpers.test.mjs had a trailing
blank line at EOF. Stripped.

323/323 tests pass; importance-score-parity preserved.

* fix(audit): align residue staleness gate with max digest window (P2 round-2)

Reviewer flagged that --mode=residue --apply with the 24h staleness
default would delete legitimate weekly-user stories. A pre-PR-3422 row
that lacks publishedAt with lastSeen 2-7d ago is still ship-able for a
weekly digest (whose readTimeAgeCutoffMs in
digest-orchestration-helpers.mjs is windowStart - 24h = 8d ago) — but
the 24h gate would mark it as residue and delete it.

Fix: default --residue-min-stale-hours from 24 to 192 (= 7d max digest
window + 24h buffer). Aligns with the readTimeAgeCutoffMs formula so
residue mode can never delete a row still legitimately ship-able for
ANY user. New explicit regression test:
  "SAFETY: does NOT match weekly-user 5d-old story" — pre-fix would
  have failed (deleted the row); now PROTECTED.

Operators with confidence the fleet is daily-only can drop to faster
cleanup via --residue-min-stale-hours=48 (documented escape hatch in
the script header runbook + test case).

The other safety guards remain:
  - publishedAt present → not residue (caught by --mode=age, not residue).
  - lastSeen fresh (< gate) → protected.
  - missing lastSeen → ancient (errs toward eviction in opt-in path).

325/325 tests pass; importance-score-parity preserved.

…a73#3431)

* feat(convex/broadcast): audience-export pipeline for PRO launch

Stacked on feat/customers-normalized-email-index — needs that branch's
customers.normalizedEmail field + index + backfill before this action
can produce a clean dedup.

Adds:
- convex/broadcast/audienceExport.ts: paginated internalAction plus
  three internalQueries that build the deduped audience and push
  contacts to a Resend Audience via the Contacts API.

Dedup formula: registrations − emailSuppressions − customers. Any
user who's been through Dodo checkout (active, cancelled, expired)
is excluded — pitching paid users a "buy PRO!" email is the worst-
case failure mode the launch is trying to avoid.

Idempotent on re-run: Resend's 422 already_exists response is counted
as `alreadyExists`, not `failed`. Cursor-paginated for safe resume
mid-export. Dry-run mode (counts only, no Resend calls) for verifying
exclusion math before the first real send.

Codegen catch-up:
- _generated/api.d.ts updated to declare broadcast/audienceExport
  (this file cross-references its own siblings via internal.* and
  needs the typed reference).
- Also catches up payments/backfillCustomerNormalizedEmail from the
  base PR — that file didn't cross-reference so its CI passed
  without the codegen update; adding now so future calls into it
  resolve correctly.
- Convex regenerates identically on deploy — no drift risk.

Operational:
  npx convex run broadcast/audienceExport:exportProLaunchAudience \
    '{"audienceId":"aud_xxx"}'
  # Loop, passing continueCursor from each response, until isDone:true

* fix(audienceExport): address PR review (P1×3)

All three findings valid; verified line numbers match commit 7a440c1
and the Resend API claim against current docs at
https://resend.com/docs/api-reference/contacts/create-contact.

P1 #1 — Paid-customer exclusion fails open when normalizedEmail missing:
`getPaidEmails` now derives the join key from `customers.email` (the
existing `email.trim().toLowerCase()` convention) when `normalizedEmail`
is unset. The backfill becomes a perf optimization — a missed/incomplete
run can no longer silently leak paid users into the launch audience.

P1 #2 — Wrong Resend endpoint:
Switched from `POST /audiences/{id}/contacts` (legacy, may still resolve
but no longer the canonical 2026 path) to `POST /contacts` with
`segments: [{ id: segmentId }]` in the body, per current Resend docs.
Renamed the action arg from `audienceId` to `segmentId` to match
Resend's renaming of Audiences → Segments. Updated JSDoc usage examples.

P1 #3 — 409/422 catch-all masking validation errors:
Added `isDuplicateContactError()` helper that parses the 422 body and
matches on `name` (`*_already_exists` / `*_duplicate`) and `message`
(`/already (exists|in)|duplicate/`). Only duplicate-shaped responses
increment `alreadyExists`; everything else (missing segment, invalid
email, unauthorized field, etc.) increments `failed` and logs the
parsed body. Also dropped the speculative 409 branch — Resend doesn't
use 409 for contacts; 422 is the real status.

* fix(audienceExport): two-step upsert guarantees segment membership (P1)

Round-2 review finding: previous fix counted POST /contacts 422 duplicates
as alreadyExists without verifying the contact ended up in OUR segment.
Resend's global-Contacts model means a 422 duplicate could mean the
contact exists in a different segment or no segment at all — and the
`segments` field on the duplicate path is not applied — so existing
global contacts could be silently OMITTED from the launch segment while
the export reports success.

New `upsertContactToSegment(apiKey, email, segmentId)` helper does a
deterministic two-step:

  1. POST /contacts with `segments: [{ id }]`. Brand-new globally →
     creates and assigns in one call → outcome=created.
  2. If step 1 returns a duplicate-shaped 422, the contact exists
     globally. Disambiguate with POST /contacts/{email}/segments/{id}:
       - 2xx → was global-only or in another segment, now linked here
         → outcome=linkedExisting
       - 422 duplicate-shaped → was already in this segment
         → outcome=alreadyInSegment
       - anything else → outcome=failed

Stats updated:
- `upserted` now counts BOTH `created` AND `linkedExisting` (anything
  that ended up in the segment via this call).
- `linkedExisting` added as a separate diagnostic counter — operator
  can compare to verify how many were pre-existing global contacts.
- `alreadyExists` now strictly means "was already in this segment
  before this call" — never inferred from a global-duplicate response
  without segment-membership verification.

* fix(audienceExport): address PR review round 3 (P2×3)

Stale: P1 "422 conflates duplicate with validation errors" was cited
on commit 7a440c1; addressed in ebf2bf8 — replied as resolved.

Three valid P2s on current code, all fixed:

P2 — Missing User-Agent header on Resend fetches:
AGENTS.md:185 requires User-Agent on every server-side fetch. Added
USER_AGENT constant and applied to both /contacts and
/contacts/{email}/segments/{id} POSTs.

P2 — Raw email PII written to Convex dashboard logs:
The action's failure-path console.error interpolated `${email}` into
log lines that anyone with project access can read. Added maskEmail()
helper (`[email protected]` → `jo******@example.com`) and applied
to the failure-path log.

P2 — `upserted` semantics differ between dry-run and live:
The previous draft incremented stats.upserted in BOTH dry-run (every
email passing dedup) and live (only successful Resend calls). Operators
comparing dry-run to live totals to validate dedup math would see a
spurious discrepancy on any failure / linkedExisting / alreadyInSegment.

Reshaped ExportStats:
- Dry-run-only: `wouldUpsertAfterDedup` — count of emails that pass
  the dedup filters and would be attempted on a live run.
- Live-only: `upserted`, `linkedExisting`, `alreadyExists`, `failed`
  — strictly result of Resend interactions; all zero in dry-run mode.
- Shared (dedup-only): `suppressedSkipped`, `paidSkipped`, `emptyEmail`.

Operator can now compare dry-run `wouldUpsertAfterDedup` to live
`upserted + alreadyExists` and any divergence is a real Resend issue,
not a counter-semantics artefact.

…eads (koala73#3667)

* fix(brief): grounding gate on digest synthesis — block hallucinated leads

The 2026-05-12 0801 send shipped a "President Biden announced a new
executive order targeting cryptocurrency mixers and privacy coins" lead
to users whose actual story pool was Trump-era geopolitics (Iran
ceasefire, Israeli strikes in Lebanon, Sudan drones, Cuba rhetoric,
Russia/Ukraine, EU sanctions). The magazine envelope rendered the
correct grounded lead; the email Executive Summary block rendered a
fully fabricated narrative with four threads all about the imaginary
Biden EO. Shape was valid — content was a hallucination from training-
data priors.

Root cause: validateDigestProseShape only checks shape (lengths,
types, array presence). Any well-formed JSON whose lead names entities
absent from the input pool was happily cached for 4h and re-served on
every cache hit until the row TTL'd out. The canonical-brain promise
from PR koala73#3396 ("compose-phase synthesis is reused on send-phase cache
read") only guarantees same-output-per-cache-hit; it does not guard
against the LLM committing to a fabricated row in the first place.

This adds checkLeadGrounding(synthesis, stories): extract proper-noun
tokens (capitalised, length ≥4) from story headlines; require ≥2
distinct hits in the lead + thread teasers (relaxed to 1 when the
corpus has <4 anchor tokens). Length cap of 4 deliberately filters
short-form acronyms (US/EU/UN/RSF) which are too generic to
discriminate. Plumbed through validateDigestProseShape (new optional
stories arg, back-compat preserved) → parseDigestProse →
generateDigestProse cache-hit path. Cache key bumped v4 → v5 so
existing v4 rows (which may carry shape-valid hallucinations) are
evicted on rollout; 4h paid-for-once cost matching the established
pattern at this function.

When a cached row fails the grounding gate, the cron's existing 3-level
fallback chain falls through to L2 (capped pool, no profile/greeting)
and ultimately L3 (stub with "Digest" subject). A user gets either a
re-rolled grounded lead or a degraded subject line — never a
hallucinated headline.

Test coverage: the verbatim May 12 hallucinated lead + the verbatim
2026-05-12 story headlines as fixtures, asserting the validator
rejects. Plus the actual magazine lead from the same day as positive
control, plus skip-on-empty-stories back-compat, plus single-story
threshold relaxation, plus short-acronym filtering, plus a
generateDigestProse cache-hit re-LLM regression test.

86/86 brief-llm tests pass. 179/179 brief-related tests pass.
typecheck clean. biome lint clean.

* fix(brief): address PR koala73#3667 review — lead must ground independently + token-set matching

Two real bugs caught on review of the grounding validator.

#1 — Combined haystack let a hallucinated lead pass when teasers
happened to mention real entities.

  Before: lead + threads[].teaser were joined into one string and
  any ≥2 anchor hits across the combination passed. So a synthesis
  with a fabricated "President Biden announced..." lead and grounded
  teasers like "Trump rejected Tehran's response" would accept —
  even though the visible top-of-email lead stayed fabricated.

  After: lead must independently hit ≥1 anchor. Combined check
  (≥2 hits, or ≥1 for sparse corpora) still applies on top of that.
  A hallucinated lead with grounded teasers now correctly rejects.

#2 — haystack.includes(tok) accepted unrelated entities via
substring match.

  Before: anchor "iran" hit on "tirana", "oman" on "romania",
  "india" on "indiana". Any anchor that happens to appear as a
  substring of an unrelated word in the synthesis prose counted.

  After: both sides tokenise on the same delimiter regex into Sets
  and match by membership. Story-side keeps the proper-noun
  capitalisation filter; synthesis-side does not (so possessive
  forms / sentence-medial mentions still count as anchor hits).

Both regressions covered by new golden tests:
  - hallucinated-lead-grounded-threads: rejects with the new lead
    independence requirement, accepts when the lead is also grounded
  - substring-trap-corpus (Iran/Oman/India + Tirana/Romania/Indiana):
    rejects under token-set matching, accepts under real word matches

Tests: 88/88 brief-llm, 8604/8604 full suite, typecheck clean,
biome clean.

* fix(brief): address PR koala73#3667 review round 2 — stopword filter + Unicode delimiters

Two more bugs in the grounding validator caught on second review.

#1 (HIGH) — Generic capitalised words leaked into the anchor set.

  Before: any capitalised word ≥4 chars from a headline became an
  anchor. So "President Trump signed Iran sanctions" added
  "president" to storyTokens. A hallucinated lead like "President
  Biden announced..." passed the lead-anchor check via the shared
  "president" token, and a teaser mentioning Iran satisfied the
  combined threshold — recreating the exact failure mode this PR
  is trying to block.

  After: GROUNDING_ANCHOR_STOPWORDS filters honorifics ("President",
  "Senator", "Minister"), generic role labels ("Officials",
  "Members", "Forces"), quasi-adjectives ("Senior", "Federal",
  "Former"), sentence-start filler ("After", "Following",
  "Despite"), and calendar words. Specific entity names (Iran,
  Trump, Israel, EU) are deliberately NOT on the list. "May" is
  also omitted (Theresa May, May Day, May the month all collide).

#2 (MEDIUM) — Unicode apostrophes weren't delimiters.

  Before: GROUNDING_TOKEN_DELIMS only included ASCII apostrophe.
  Reuters/AP/Guardian headlines use U+2019 ("China's", "Iran's",
  "DPRK's"). The regex didn't split on U+2019, so "China's" became
  a single token "china's" that no normal lead saying "China"
  could ever match — a false negative that would reject genuinely
  grounded leads.

  After: U+2018, U+2019, U+201C, U+201D, U+00B4 added alongside
  ASCII counterparts.

Both regressions covered by new golden tests:
  - "President Trump signed..." headlines + "President Biden
    announced..." hallucinated lead must REJECT
  - U+2019 headlines + ASCII-quote grounded lead must ACCEPT

Tests: 90/90 brief-llm, 8606/8606 full suite, typecheck clean,
biome clean.

* fix(brief): address PR koala73#3667 review round 3 — bigram-leading title stopwords

Round 2 added "President" to the anchor stopword list. Round 3 review
caught that other common bigram titles still leak through.

  Before: "Prime Minister Netanyahu says Iran threats continue" added
  "prime" to anchors. A hallucinated "Prime Minister Trudeau announced
  cryptocurrency restrictions..." passed the lead-anchor check via the
  shared "prime" token. A teaser mentioning Iran satisfied the combined
  threshold. Same shape worked for Chief Justice / Cardinal X /
  Chancellor X / Speaker X / Ambassador X.

  After: GROUNDING_ANCHOR_STOPWORDS extended with bigram-leading
  titles: prime, chief, premier, chancellor, speaker, ambassador,
  envoy, commissioner, attorney, cardinal, archbishop, monsignor,
  reverend, pastor, bishop, lord, lady, dame, congressman/woman/person,
  representative, delegate, baron(ess).

Regression test covers Prime Minister / Chief Justice / Cardinal
ride-along leads, plus a counter-control naming the actual entity
(Netanyahu) which still passes.

Tests: 91/91 brief-llm, 8607/8607 full suite, typecheck clean,
biome clean.

* fix(brief): observability for digest synthesis failures (PR koala73#3667 review #3)

Greptile review caught: when generateDigestProse returns null, ops
can't tell whether the LLM threw, returned no content, returned
malformed JSON, or returned a shape-valid hallucination that the
grounding gate rejected. All four classes look identical in logs —
just "L1 returned null" — so a sustained model regression looks
identical to an infra blip during on-call triage.

Add two distinct console.warn lines:
  - "LLM call threw" — provider/network failure (catches the actual
    error message for triage)
  - "ungrounded or malformed output" — provider returned but parsing
    or grounding rejected (logs text length so 0 vs ~900-char
    distinguishes "no content" from "model drift/hallucination")

Cost note documented inline: we deliberately do NOT cache the
failure with a sentinel under the v5 key. At temperature 0.4 the
next cron tick may roll a grounded output for the same prompt;
caching the failure would block legitimate retries. The L1→L2→L3
fallback in runSynthesisWithFallback handles user-visible
degradation; these logs handle ops visibility.

Tests: 91/91 brief-llm pass, typecheck clean, biome clean.

* fix(brief): PR koala73#3667 review round 4 cleanup — extract grounding helpers + fixture comment

Two reviewer-flagged cleanups:

- P2: extract `extractAnchors` / `tokensetOf` from inside
  checkLeadGrounding to file-level `extractAnchorTokens` /
  `groundingTokenSet`. Avoids closure re-instantiation per call,
  separates the two helpers cleanly, and makes them individually
  inspectable / unit-testable. Behaviour unchanged — same callers,
  same inputs, same outputs.

- P3: add a load-bearing comment on the `story()` test factory
  documenting that the default headline ("Iran threatens to close
  Strait of Hormuz...") is what grounds every `validJson` lead used
  by `generateDigestProse` / `generateDigestProsePublic` cache-shape
  tests. If a future contributor changes the default headline so it
  no longer mentions Iran/Hormuz, those tests would silently reject
  via the v5 grounding gate with cascading "expected truthy, got
  null" failures whose root cause is invisible. Comment names the
  invariant + the escape hatch (override headline + update
  validJson).

Tests: 91/91 brief-llm pass, typecheck clean, biome clean.

* fix(brief): PR koala73#3667 review round 5 — JSDoc attachment, blind-spot warn, stopword heuristic

Three reviewer findings:

#1 — JSDoc block was orphaned by the round-4 helper extraction.
  The big "Cheap content-grounding check..." doc sat at line 519,
  BEFORE extractAnchorTokens and groundingTokenSet — JSDoc parsers
  attach a doc block to the NEXT declaration, so the rich docs were
  describing the wrong function. Moved directly above
  checkLeadGrounding where they belong.

#2 — Lowercase-headline blind spot. If a feed ever produces all-
  lowercase or all-≤3-char headlines, extractAnchorTokens returns
  empty for every story, storyTokens.size === 0, and the gate
  silently returns true (skip). Added a console.warn gated on
  stories.length >= 3 so synthetic single-headline test corpora
  don't spam logs but real production feed regressions surface in
  cron output.

#3 — Stopword maintenance heuristic. Added a comment to
  GROUNDING_ANCHOR_STOPWORDS describing the detection rule: dump a
  week of real headlines, tokenise with stopwords disabled, count
  frequencies, inspect any token appearing in >~10% of headlines
  that isn't a known proper noun. The Prime/Chief/Cardinal gaps
  caught on rounds 2-3 would have surfaced from such a frequency
  audit. Captures the maintenance burden as an actionable signal
  rather than guesswork.

Tests: 91/91 brief-llm pass, typecheck clean, biome clean. The
new console.warn calls are intentionally surface in test output
when triggered (reviewer round 5 #4 — informational, no action).

…+ lock down (koala73#3832)

* fix(route-explorer): close Asia→DE lane gap for TW/KR/JP/VN/TH/PH/IN + lock down

PR koala73#3828 fixed HK→Germany by adding `china-europe-suez` and `asia-europe-cape`
to HK's `nearestRouteIds` in `scripts/shared/country-port-clusters.json`, and
explicitly deferred TW/KR/JP/VN/TH/PH because each needed review against actual
shipping data. While triaging a reporter's HK→DE / Premium Stock / WM Analyst
empty-state screenshots (pr-3718 session), I confirmed those six countries plus
IN still fail with `noModeledLane: true` against DE — server reports the gap
honestly, UI shows "No modeled lane for this pair."

Root cause for each is identical to HK's: the country has only
Pacific/intra-Asia/Gulf-oil route IDs, none of which appear in DE's cluster
(`china-europe-suez`, `asia-europe-cape`, `transatlantic`). Server intersection
at server/worldmonitor/supply-chain/v1/get-route-explorer-lane.ts:233-234
returns zero shared routes → noModeledLane=true.

Fix: add `china-europe-suez` and `asia-europe-cape` (the trunk Asia↔Europe
container routes terminating at Rotterdam) to TW, KR, JP, VN, TH, PH, IN.
Both routes are tagged `category: 'container', status: 'active'` in
src/config/trade-routes.ts; they are factually correct for every major Asian
export hub shipping to Northern Europe.

Lock-down: tests/country-port-clusters-asia-europe-lane.test.mts (3 cases)

  1. Data invariant — every Asian port country in {CN,HK,TW,JP,KR,SG,MY,ID,
     TH,VN,PH,IN} must have a non-empty `nearestRouteIds` intersection with
     DE in the cluster JSON. Reverting any of the seven new entries flips
     this test red with a clear "add china-europe-suez and/or asia-europe-
     cape to <ISO2>'s nearestRouteIds" error.
  2. Computed lane — `computeLane('HK','DE','85','container')` must return
     `noModeledLane: false` and a non-empty `primaryRouteId`. This is the
     EXACT shape of the reporter's screenshot (HK→Germany, Electrical &
     Electronics HS2=85, Container, AUTO badge); the original PR koala73#3828 had
     no test pinning this case, so a future contributor reverting the HK
     entry would not be caught.
  3. Full Asian-port matrix — same `noModeledLane: false` assertion for
     all 12 Asian export hubs against DE, defending the algorithm boundary
     (computeLane) on top of the data invariant.

Bite-test: reverting just the data change (git stash on the JSON) flips
tests #1 and #3 red, naming the exact 7 offenders (TW/JP/KR/TH/VN/PH/IN);
test #2 stays green because HK was already fixed by koala73#3828. Confirms each
assertion bites its intended regression.

The 30-query smoke matrix in tests/route-explorer-lane.test.mts (38 cases)
stays 38/38 green. tsc clean.

This complements PR koala73#3828's HK fix and clears the deferred-follow-up note
without touching unrelated areas. The reporter's third screenshot (HK→DE
"No modeled lane") will resolve on next deploy regardless of this PR
(PR koala73#3828 already fixed HK); this PR additionally prevents TW/KR/JP/VN/TH/
PH/IN from showing the same dead-end to other Asia-export-hub users, and
makes both fixes regression-proof.

* fix(route-explorer): address PR koala73#3832 review P1 — IN→DE picks india-europe, not china-europe-suez

Reviewer (correct): adding `china-europe-suez` + `asia-europe-cape` to IN
satisfied `noModeledLane === false` for IN→DE but pushed the resolver to
pick `china-europe-suez` (Shanghai → Rotterdam) as the primary route. The
Route Explorer UI highlights the route's from→to ports, so an Indian
shipment to Germany rendered a Shanghai origin port. The test was vacuous
on this dimension — it only asserted `noModeledLane === false`, never
which route resolved.

Root cause of my original choice: DE's `nearestRouteIds` didn't list
`india-europe`, so no intersection existed for IN→DE via that route. I
papered over the missing intersection by polluting IN's profile with
China-origin trunk routes instead of fixing DE's profile.

Correct fix (follows existing convention): every European country that
already has `china-europe-suez` (the Asia-Europe trunk to Rotterdam) also
gets `india-europe` (the India-Europe trunk to Rotterdam). Both routes
terminate at the same Northern-European hub; the data model treats trunk
routes as broadly serving any European port country. Applied uniformly to
DE, GB, FR, IT, ES, NL, BE, PL, SE, DK, FI, GR, PT (the 13 European
countries with `china-europe-suez`).

IN's `nearestRouteIds` reverts to `["india-europe", "india-se-asia",
"gulf-asia-oil"]` — no more China-origin pollution. IN→DE now intersects
on `india-europe` (only shared route), so the resolver picks it
unambiguously.

Test tightening (the actual lock the reviewer asked for):
`tests/country-port-clusters-asia-europe-lane.test.mts` adds an
`EXPECTED_PRIMARY_ROUTE_TO_DE` map and asserts `computeLane(<iso2>, 'DE',
'85', 'container').primaryRouteId === expected` for all 12 Asian-port
countries. IN→DE must equal `'india-europe'`; all others must equal
`'china-europe-suez'`. Removed the weaker "noModeledLane: false for the
matrix" test — the stricter primaryRouteId assertion subsumes it (a
`noModeledLane=true` response fails the equality check with a clear
message naming the offender).

Bite-test of the new assertion:
  - Revert `india-europe` from DE only → both tests #1 and #3 flip red
    naming IN: data invariant says "IN → DE: shared routes = []", lane
    assertion says "IN → DE: noModeledLane=true (expected
    primaryRouteId='india-europe')". Confirms the assertion bites the
    exact reviewer-flagged regression.
  - HK→DE remains green (china-europe-suez is correct for an HK origin).
  - 38/38 of the existing `tests/route-explorer-lane.test.mts` smoke
    matrix stays green. tsc clean.

The other Asian origins (TW/JP/KR/VN/TH/PH/SG/MY/ID) already correctly
intersect DE on `china-europe-suez` (added in this PR's first commit) —
that's the trunk route their containers actually take to Europe, so
their primaryRouteId assertion was already correct.

* test(route-explorer): drop redundant HK-specific case (PR koala73#3832 review P2)

Greptile P2: the standalone HK → DE test was fully subsumed by the matrix
test, which iterates over ASIAN_PORT_COUNTRIES (includes HK at index 1)
and asserts `primaryRouteId === EXPECTED_PRIMARY_ROUTE_TO_DE[iso2]`.
HK's expected value is 'china-europe-suez'; a regression on HK fails the
equality check with a clear message, just like every other origin.

Renamed the surviving matrix test to call out HK→DE explicitly so the
provenance trail to the pr-3718 reporter symptom isn't lost.

…9 regression) (koala73#3835)

The 2026-05-19 Pro brief shipped "How nuclear war would impact the
global food system" from Bulletin of Atomic Scientists as CRITICAL
story #6, sitting alongside breaking news about the Iran-Israel war
and the WHO Ebola declaration. The brief promises event-driven
intelligence — a Bulletin analysis essay is not an event.

The existing classifier missed it on all three current signals:
- STRONG #1 (URL section /opinion/): Bulletin URLs have no opinion-
  style path because the WHOLE SITE is commentary, no hard-news
  section to distinguish from.
- STRONG #2 (headline prefix "Opinion:"): hard-news-shaped title.
- CORROBORATING (quote-wrap + columnist framing): description reads
  like a news lede.

The signal IS the publisher. Add STRONG #3: a hand-curated allowlist
of commentary-only publishers, matched by hostname (suffix-anchored to
permit `newsletter.<host>` / `m.<host>` while rejecting typo-domains
like `evilthebulletin.org`).

Initial list (per docs/plans/2026-05-19-001 U1):
- thebulletin.org      — Bulletin of Atomic Scientists
- project-syndicate.org — op-eds from world leaders / academics
- foreignaffairs.com   — CFR's analysis quarterly
- foreignpolicy.com    — Foreign Policy magazine
- warontherocks.com    — defense analysis blog

Maintenance commitment: quarterly review against `droppedOpinion`
telemetry to catch (a) new commentary publishers to add, (b) listed
publishers that launched a hard-news section. Rollback path is
remove-from-Set, never per-URL exceptions (cruft).

Pattern mirrors the existing safePathname/STRONG_URL_SEGMENTS shape;
new safeHostname helper parses URL().hostname so the match is on
the parsed host (not raw .includes() on the link string — closes the
tracking-param injection vector documented in PR koala73#3748).

7 new test cases cover the May 19 regression, the 4 other publishers,
subdomain matching, typo-domain rejection, tracking-param / fragment
spoofing, malformed URLs, and the "allowlisted host PLUS hard-news
content → still opinion" rule.

This is PR-1 of the 4-PR Phase 1 wave from
docs/plans/2026-05-19-001-fix-brief-content-quality-regressions-plan.md.