- Rename cronbachAlpha param `items` → `observations` (matches row layout)
- CI fallback uses ±max(5, 10%) so value=0 gets [0, 5] not [0, 0]
- probabilityDown rounds from already-rounded probabilityUp (sum=1 invariant)

* feat(resilience): add shared statistical utilities

Port the resilience statistics helpers needed for country resilience scoring into server/_shared with a focused test suite.

Refs koala73#2478

Validation:
- npx tsx --test tests/resilience-stats.test.mts
- npm run typecheck:api (fails on upstream main because server/__tests__/entitlement-check.test.ts imports vitest without the dependency in tsconfig.api scope)

* fix(resilience): harden stats utilities from review feedback

- cronbachAlpha: reject jagged rows (unequal row lengths → 0)
- CUSUM: simplify double-normalized slack to constant 0.5
- nrcForecast: use modelError-scaled probability instead of raw delta
- Add boundary test for nrcForecast at exactly 3 values
- Add test for cronbachAlpha jagged row guard

* fix(resilience): fix CUSUM changepoint location and nrcForecast horizon guard

detectChangepoints:
- Track onset index (where accumulation began) instead of detection index
- Reduce CUSUM slack from 0.5 to 0.25 to detect moderate step shifts
- Require onset >= 2 to filter false positives from initial-regime artifacts
- Tighten tests: assert exact count and onset range, add moderate shift case

nrcForecast:
- Return neutral empty result for non-positive horizonDays
- Add test covering horizon=0 and negative values

* fix(resilience): address Greptile review items #2-4

- Rename cronbachAlpha param `items` → `observations` (matches row layout)
- CI fallback uses ±max(5, 10%) so value=0 gets [0, 5] not [0, 0]
- probabilityDown rounds from already-rounded probabilityUp (sum=1 invariant)

* test(resilience): cover zero-value CI floor in short-history fallback

---------

Co-authored-by: Elie Habib <[email protected]>

…ass) (koala73#2961)

* feat(resilience): dimension freshness propagation (T1.5 propagation pass)

Ships the Phase 1 T1.5 propagation pass of the country-resilience
reference-grade upgrade plan. PR koala73#2947 shipped the staleness
classifier foundation (classifyStaleness, cadence taxonomy, three
staleness levels) and explicitly deferred the dimension-level
propagation. This PR consumes the classifier and surfaces per
dimension freshness on the ResilienceDimension response.

What this PR commits

- Proto: new DimensionFreshness message + `freshness` field on
  ResilienceDimension (last_observed_at_ms, staleness string).
- New module server/worldmonitor/resilience/v1/_dimension-freshness.ts
  that reads seed-meta values for every sourceKey in INDICATOR_REGISTRY
  and aggregates the worst staleness + oldest fetchedAt across the
  constituent indicators of each dimension.
- scoreAllDimensions decorates each dimension score with its freshness
  result before returning. The 13 dimension scorer function bodies are
  untouched: aggregation is a decoration pass at the caller level so
  this PR stays mechanical.
- Response builder: _shared.ts buildDimensionList propagates the
  freshness field to the proto output.
- Tests: 10 classifyDimensionFreshness + readFreshnessMap cases in a
  new test file + response-shape case on the release-gate test.

Aggregation rules

- last_observed_at_ms: MIN fetchedAt across the dimension's indicators
  (oldest signal = most conservative bound). 0 when no signal has
  ever been observed.
- staleness: MAX staleness level across the dimension's indicators
  (stale > aging > fresh). Empty string when the dimension has no
  indicators in the registry (defensive path).

What is deliberately NOT in this PR

- No changes to the 13 individual dimension scorer function bodies.
  Per-signal freshness inside scorers is a future enhancement.
- No widget rendering of the freshness badge (T1.6 full grid, PR 3).
- No cache key bump: additive int64/string fields with zero defaults.

Verified

- make generate clean, new interface in regenerated types
- typecheck + typecheck:api clean
- tests/resilience-dimension-freshness.test.mts all new cases pass
- tests/resilience-*.test.mts full suite pass
- test:data clean
- lint exits 0 on touched files

* fix(resilience): resolve templated sourceKeys to real seed-meta (koala73#2961 P1)

Greptile P1 finding on PR koala73#2961: readFreshnessMap() assumed every
INDICATOR_REGISTRY sourceKey could be fetched as seed-meta:<sourceKey>,
but most entries use placeholder templates like resilience:static:{ISO2},
energy:mix:v1:{ISO2}, and displacement:summary:v1:{year}. Those produce
literal lookups like seed-meta:resilience:static:{ISO2} which don't
exist in Redis, so the freshness map missed every templated entry and
classifyDimensionFreshness marked the affected dimensions stale even
with healthy seeds. Most Phase 1 T1.5 freshness badges were broken on
arrival.

Fix: two-layer resolution in _dimension-freshness.ts.

Layer 1 stripTemplateTokens: drop :{placeholder} and :* segments.
  'resilience:static:{ISO2}'        -> 'resilience:static'
  'resilience:static:*'             -> 'resilience:static'
  'energy:mix:v1:{ISO2}'            -> 'energy:mix:v1'
  'displacement:summary:v1:{year}'  -> 'displacement:summary:v1'

Layer 2 stripTrailingVersion: strip trailing :v\d+, mirroring
writeExtraKeyWithMeta + runSeed() in scripts/_seed-utils.mjs which
never persist the trailing version in seed-meta keys. Handles
cyber:threats:v2, infra:outages:v1, unrest:events:v1,
conflict:ucdp-events:v1, sanctions:country-counts:v1, and the
displacement v1 case above.

Layer 3 SOURCE_KEY_META_OVERRIDES: explicit table for drift cases
where the two strips still do not match the real seed-meta key.
Verified against api/seed-health.js, api/health.js, and scripts/seed-*.
Drift cases covered:
  economic:imf:macro       -> economic:imf-macro
  economic:bis:eer         -> economic:bis
  economic:energy:v1:all   -> economic:energy-prices
  energy:mix               -> economic:owid-energy-mix
  energy:gas-storage       -> energy:gas-storage-countries
  news:threat:summary      -> news:threat-summary
  intelligence:social:reddit -> intelligence:social-reddit

readFreshnessMap now deduplicates reads by resolved meta key (so
the 15+ resilience:static indicators share one Redis read) and
projects per-meta-key results back onto per-sourceKey map entries so
classifyDimensionFreshness can keep its existing interface.

Regression coverage:
- stripTemplateTokens cases for {ISO2}, {year}, and *.
- stripTrailingVersion cases for :v1 / :v2 suffixes.
- Embedded :v1 carve-out (trade:restrictions:v1:tariff-overview:50
  stays unchanged because :v1 is not trailing).
- Override cases for the seven drift entries.
- Integration test that proves every resilience:static:* / {ISO2}
  registry entry resolves to the same seed-meta and is marked fresh
  when that one key has a recent fetchedAt.
- healthPublicService end-to-end test: classifies fresh when
  seed-meta:resilience:static is recent (was stale before the fix).
- Registry-coverage assertion: every INDICATOR_REGISTRY sourceKey
  must resolve to a seed-meta key that either lives in
  api/seed-health.js, api/health.js, or the test's
  KNOWN_SEEDS_NOT_IN_HEALTH allowlist (which covers the four seeds
  written by writeExtraKeyWithMeta / runSeed that no health monitor
  tracks yet: trade:restrictions, trade:barriers,
  sanctions:country-counts, economic:energy-prices). Fails loudly if
  a future registry entry introduces an unknown sourceKey.

Note on P1 #2 (scoreCurrencyExternal absence-branch delete): that is
PR koala73#2964's scope (T1.7 source-failure wiring), not koala73#2961 (T1.5
propagation pass). koala73#2961 never claimed to delete the fallback branch;
no test in this branch expects the new IMPUTE.bisEer fallback. The
reviewer conflated the two stacked PRs. koala73#2964 owns the delete.

…oala73#2962)

* feat(resilience): dimension freshness propagation (T1.5 propagation pass)

Ships the Phase 1 T1.5 propagation pass of the country-resilience
reference-grade upgrade plan. PR koala73#2947 shipped the staleness
classifier foundation (classifyStaleness, cadence taxonomy, three
staleness levels) and explicitly deferred the dimension-level
propagation. This PR consumes the classifier and surfaces per
dimension freshness on the ResilienceDimension response.

What this PR commits

- Proto: new DimensionFreshness message + `freshness` field on
  ResilienceDimension (last_observed_at_ms, staleness string).
- New module server/worldmonitor/resilience/v1/_dimension-freshness.ts
  that reads seed-meta values for every sourceKey in INDICATOR_REGISTRY
  and aggregates the worst staleness + oldest fetchedAt across the
  constituent indicators of each dimension.
- scoreAllDimensions decorates each dimension score with its freshness
  result before returning. The 13 dimension scorer function bodies are
  untouched: aggregation is a decoration pass at the caller level so
  this PR stays mechanical.
- Response builder: _shared.ts buildDimensionList propagates the
  freshness field to the proto output.
- Tests: 10 classifyDimensionFreshness + readFreshnessMap cases in a
  new test file + response-shape case on the release-gate test.

Aggregation rules

- last_observed_at_ms: MIN fetchedAt across the dimension's indicators
  (oldest signal = most conservative bound). 0 when no signal has
  ever been observed.
- staleness: MAX staleness level across the dimension's indicators
  (stale > aging > fresh). Empty string when the dimension has no
  indicators in the registry (defensive path).

What is deliberately NOT in this PR

- No changes to the 13 individual dimension scorer function bodies.
  Per-signal freshness inside scorers is a future enhancement.
- No widget rendering of the freshness badge (T1.6 full grid, PR 3).
- No cache key bump: additive int64/string fields with zero defaults.

Verified

- make generate clean, new interface in regenerated types
- typecheck + typecheck:api clean
- tests/resilience-dimension-freshness.test.mts all new cases pass
- tests/resilience-*.test.mts full suite pass
- test:data clean
- lint exits 0 on touched files

* fix(resilience): resolve templated sourceKeys to real seed-meta (koala73#2961 P1)

Greptile P1 finding on PR koala73#2961: readFreshnessMap() assumed every
INDICATOR_REGISTRY sourceKey could be fetched as seed-meta:<sourceKey>,
but most entries use placeholder templates like resilience:static:{ISO2},
energy:mix:v1:{ISO2}, and displacement:summary:v1:{year}. Those produce
literal lookups like seed-meta:resilience:static:{ISO2} which don't
exist in Redis, so the freshness map missed every templated entry and
classifyDimensionFreshness marked the affected dimensions stale even
with healthy seeds. Most Phase 1 T1.5 freshness badges were broken on
arrival.

Fix: two-layer resolution in _dimension-freshness.ts.

Layer 1 stripTemplateTokens: drop :{placeholder} and :* segments.
  'resilience:static:{ISO2}'        -> 'resilience:static'
  'resilience:static:*'             -> 'resilience:static'
  'energy:mix:v1:{ISO2}'            -> 'energy:mix:v1'
  'displacement:summary:v1:{year}'  -> 'displacement:summary:v1'

Layer 2 stripTrailingVersion: strip trailing :v\d+, mirroring
writeExtraKeyWithMeta + runSeed() in scripts/_seed-utils.mjs which
never persist the trailing version in seed-meta keys. Handles
cyber:threats:v2, infra:outages:v1, unrest:events:v1,
conflict:ucdp-events:v1, sanctions:country-counts:v1, and the
displacement v1 case above.

Layer 3 SOURCE_KEY_META_OVERRIDES: explicit table for drift cases
where the two strips still do not match the real seed-meta key.
Verified against api/seed-health.js, api/health.js, and scripts/seed-*.
Drift cases covered:
  economic:imf:macro       -> economic:imf-macro
  economic:bis:eer         -> economic:bis
  economic:energy:v1:all   -> economic:energy-prices
  energy:mix               -> economic:owid-energy-mix
  energy:gas-storage       -> energy:gas-storage-countries
  news:threat:summary      -> news:threat-summary
  intelligence:social:reddit -> intelligence:social-reddit

readFreshnessMap now deduplicates reads by resolved meta key (so
the 15+ resilience:static indicators share one Redis read) and
projects per-meta-key results back onto per-sourceKey map entries so
classifyDimensionFreshness can keep its existing interface.

Regression coverage:
- stripTemplateTokens cases for {ISO2}, {year}, and *.
- stripTrailingVersion cases for :v1 / :v2 suffixes.
- Embedded :v1 carve-out (trade:restrictions:v1:tariff-overview:50
  stays unchanged because :v1 is not trailing).
- Override cases for the seven drift entries.
- Integration test that proves every resilience:static:* / {ISO2}
  registry entry resolves to the same seed-meta and is marked fresh
  when that one key has a recent fetchedAt.
- healthPublicService end-to-end test: classifies fresh when
  seed-meta:resilience:static is recent (was stale before the fix).
- Registry-coverage assertion: every INDICATOR_REGISTRY sourceKey
  must resolve to a seed-meta key that either lives in
  api/seed-health.js, api/health.js, or the test's
  KNOWN_SEEDS_NOT_IN_HEALTH allowlist (which covers the four seeds
  written by writeExtraKeyWithMeta / runSeed that no health monitor
  tracks yet: trade:restrictions, trade:barriers,
  sanctions:country-counts, economic:energy-prices). Fails loudly if
  a future registry entry introduces an unknown sourceKey.

Note on P1 #2 (scoreCurrencyExternal absence-branch delete): that is
PR koala73#2964's scope (T1.7 source-failure wiring), not koala73#2961 (T1.5
propagation pass). koala73#2961 never claimed to delete the fallback branch;
no test in this branch expects the new IMPUTE.bisEer fallback. The
reviewer conflated the two stacked PRs. koala73#2964 owns the delete.

* feat(resilience): T1.6 full grid with imputation + freshness columns

Ships the Phase 1 T1.6 full grid slice of the country-resilience
reference-grade upgrade plan. PR koala73#2949 shipped the per-dimension
confidence grid scaffold (label + coverage bar + percentage) and
explicitly deferred two columns because their proto fields had not
landed yet. PR koala73#2959 (imputationClass) and koala73#2961 (freshness) added
those fields to the ResilienceDimension response type. This PR wires
both into the widget, completing the grid and satisfying the Phase 1
acceptance criterion "Widget shows per-dimension coverage + imputation
class + freshness badge".

What this PR commits

- DimensionConfidence type gains imputationClass and staleness plus
  a numeric lastObservedAtMs coerced from the proto int64 string.
- formatDimensionConfidence normalizes empty string, unknown, and
  missing fields into typed nulls so downstream rendering can branch
  safely.
- New helpers getImputationClassIcon, getImputationClassLabel,
  getStalenessLabel for compact glyphs and tooltip strings.
- ResilienceWidget.renderDimensionConfidenceCell renders two new
  columns:
    * imputation-icon column between the coverage bar and the
      percentage (color-coded per class, empty when null)
    * freshness-dot column at the far right (green/yellow/red dot
      per staleness, empty when null)
  aria-label and title attributes explain each glyph for a11y.
- country-deep-dive.css: cell grid template expanded from 3 columns
  to 5. New .resilience-widget__dimension-imputation and
  .resilience-widget__dimension-freshness rules with per-class and
  per-level color modifiers. Mobile media-query adjusted to keep the
  5-column layout readable below 480px.
- LOCKED_PREVIEW fixture populated with a realistic mix of classes
  and staleness levels so the gated-UI preview shows off the new
  columns.
- Tests: normalization cases for imputationClass and freshness,
  glyph / label helpers, LOCKED_PREVIEW smoke test.

What is deliberately NOT in this PR

- No scorer changes. source-failure class is exposed for rendering
  but no scorer path emits it yet; PR 4 of 5 wires the seed-meta
  failedDatasets consultation.
- formatResilienceConfidence single-label fallback is untouched; the
  cleanup is out of scope.
- No new SVG assets; text glyphs and CSS colors only.

Verified

- typecheck + typecheck:api clean
- tests/resilience-widget-utils.test.mts passing
- tests/resilience-*.test.mts full suite passing
- test:data clean
- lint exit 0

…koala73#2976)

* feat(intelligence): Mobility v1 population (Phase 2 PR2)

Phase 2 PR2 of the Regional Intelligence Model. Replaces the Phase 0
empty `mobility` stub with a real MobilityState built from existing
Redis inputs — no new seeders, no new API keys, no new dependencies.

## New module: scripts/regional-snapshot/mobility.mjs

Single public entry point:

  buildMobilityState(regionId, sources) -> MobilityState

Pure function. Never throws. Wrapped in a defensive try/catch that
returns the empty shape on any unexpected builder bug so snapshot
persist can never be blocked by mobility data issues.

## Data sources (all already in the repo)

- aviation:delays:faa:v1       — US airports (FAA ASWS, Vercel seeder)
- aviation:delays:intl:v3      — ~51 non-US airports across Europe/APAC/
                                 MENA/Africa/Americas (AviationStack,
                                 Railway ais-relay, 30min refresh)
- aviation:notam:closures:v2   — global ICAO NOTAM closures (FAA-side
                                 seeder hits ICAO API with monitored
                                 ICAO list spanning all regions)
- intelligence:gpsjam:v2       — global GPS jamming hexes (Wingbits,
                                 fetch-gpsjam.mjs)
- military:flights:v1          — global military ADSB tracks (OpenSky,
                                 seed-military-flights.mjs)

## Per-field output mapping

| Field                 | Source                          | Coverage      |
|-----------------------|--------------------------------|---------------|
| airspace[]            | GPS-jam hexes aggregated 1:region | All regions  |
| flight_corridors[]    | (empty in v1)                   | -             |
| airports[]            | FAA + intl delays, severity ≥ MAJOR | All regions |
| reroute_intensity     | clip(military count / 50, 0, 1) | All regions   |
| notam_closures[]      | NOTAM reasons for region airports | All regions  |

## Region classification helpers

Three pure classifiers handle the three different region conventions
the source data uses:

  airportToSnapshotRegion(alert) — splits AviationStack AIRPORT_REGION_*
    by country so americas → {north-america, latam} and apac →
    {south-asia, east-asia}

  gpsjamRegionToSnapshotRegion(label) — maps the 17 fetch-gpsjam.mjs
    classifyRegion labels (iran-iraq, levant, ukraine-russia, etc.) to
    the 7 snapshot regions. Turkey/Caucasus intentionally lands in MENA
    per the geography.js WB override.

  latLonToSnapshotRegion(lat, lon) — bbox classifier for military
    flights. Coarse but shippable; v2 could use h3/geometry.

## Airspace aggregation (P2 design decision)

v1 emits ONE AirspaceStatus per region instead of one per GPS-jam hex.
Emitting per-hex would flood the UI (current MENA data has 40+ hexes).
The aggregated entry's `reason` lists sub-region names and high/med/low
counts so the analyst can still drill into the underlying data via the
evidence chain.

## Reroute intensity caveat

Military flight count is a CRUDE proxy for civil rerouting pressure.
A dedicated civil ADSB track-diversion pipeline would be more rigorous,
but this gets us a defensible 0-1 scalar today. Documented inline.

## Freshness registry

Added 5 new input keys to FRESHNESS_REGISTRY so the snapshot's
missing_inputs / stale_inputs classification tracks mobility sources.
Per-key maxAgeMin set to 2x the source's cron cadence:

  aviation:delays:faa:v1        — 60  min (30min cron)
  aviation:delays:intl:v3       — 90  min (30min cron + Railway jitter)
  aviation:notam:closures:v2    — 120 min (60min cron)
  intelligence:gpsjam:v2        — 240 min (120min cron)
  military:flights:v1           — 30  min (10min cron)

## seed-regional-snapshots.mjs Step 8

Replaced the hardcoded empty stub with a single call:

  const mobility = buildMobilityState(regionId, sources);

No reordering of the compute flow. Pure data, no async.

## Tests — 35 new unit tests

tests/regional-snapshot-mobility.test.mjs:

  airportToSnapshotRegion (5):
    - US/CA/MX → north-america
    - Other Americas → latam
    - IN/PK/BD → south-asia, rest of APAC → east-asia
    - europe/mena/africa direct
    - lowercase labels accepted
    - null/unknown safety

  gpsjamRegionToSnapshotRegion (4): 17 labels + "other" + undefined

  latLonToSnapshotRegion (3): major cities + oceans + invalid inputs

  buildAirports (5): empty sources, region filter, severity filter,
    severity → status (disrupted vs closed), reason passthrough

  buildAirspace (4): empty, region-scoped aggregation, no-match,
    low-level still restricted

  buildRerouteIntensity (4): zero, region-scoped count, saturation,
    missing lat/lon

  buildNotamClosures (4): empty, region attribution via airport alerts,
    unattributable ICAOs skipped, long-reason truncation

  buildMobilityState (4): full shape, all-empty sources, malformed
    inputs don't throw, region isolation across fields

## Verification

- node --test tests/regional-snapshot-mobility.test.mjs: 35/35 pass
- npm run test:data: 4537/4537 pass
- npm run typecheck: clean
- npm run typecheck:api: clean
- scripts/jsconfig.json typecheck: clean on touched files
- biome lint: clean on touched files

## Deferred to future iterations

- Direct civil flight track diversion counts per corridor
- flight_corridors[] stress level + rerouted_flights_24h
- LLM-assisted NOTAM structured parse
- h3 geometry for smarter hex-to-region attribution
- Mobility-based alert triggers in the diff engine

* fix(intelligence): address 2 review findings on koala73#2976

P2 #1 — Mexico bbox mismatch between airport and flight classifiers

airportToSnapshotRegion() routes Mexico to north-america by country,
but latLonToSnapshotRegion()'s NA bbox started at lat 20°N. Mexico City
(19.43°N), Guadalajara (20.67°N), and most Mexican airspace fell into
latam, so NA's reroute_intensity understated its actual military
pressure and the two classifiers disagreed on the same country.

Fix: lower the NA bbox southern edge to lat 16.0°N. That captures
every major Mexican city and state capital (Tuxtla Gutiérrez at
16.75°N is the southernmost) while still routing Guatemala City
(14.6°N), San Salvador, Belize, and Honduras to latam. Tiny airports
at lat ~14.9°N (Tapachula in southern Chiapas) route to latam —
acceptable because they're not in the monitored set and don't carry
meaningful military traffic.

Added 8 regression tests asserting classifier parity across major
Mexican cities and correct routing of Central American capitals.

P2 #2 — seed-meta freshness path for undated payloads

classifyInputs() was treating undated payloads as "fresh" via its
fallback branch, so FAA/NOTAM/AviationStack/GPS-jam stalls would
never drag down snapshot_confidence. None of those four payloads
carry a top-level fetchedAt field; the seeders track freshness in
companion seed-meta:* keys only.

Fix: added an optional `metaKey` field to FreshnessRegistry entries.
classifyInputs() now prefers the metaKey companion's fetchedAt when
the spec declares one, falling back to the primary payload's
top-level timestamp (existing behavior) and finally to "fresh"
(existing fallback for legacy undated payloads).

Wired through the full pipeline:
  - freshness.mjs: added metaKey type, ALL_META_KEYS export, and
    metaPayloads parameter to classifyInputs()
  - snapshot-meta.mjs: buildPreMeta() forwards metaSources
  - seed-regional-snapshots.mjs: readAllInputs() now fetches both
    ALL_INPUT_KEYS and ALL_META_KEYS in a single pipeline and
    returns { sources, metaSources }; main() destructures and
    passes metaSources through computeSnapshot → buildPreMeta
  - computeSnapshot() signature gained an optional metaSources arg
    (defaults to {} for back-compat with existing tests)

Registered metaKey for each mobility input:
  aviation:delays:faa:v1       → seed-meta:aviation:faa
  aviation:delays:intl:v3      → seed-meta:aviation:intl
  aviation:notam:closures:v2   → seed-meta:aviation:notam
  intelligence:gpsjam:v2       → seed-meta:intelligence:gpsjam

military:flights:v1 does NOT declare a metaKey because its payload
already carries top-level fetchedAt — this is asserted in a test.

Added 11 regression tests covering:
  - every undated mobility input has a metaKey
  - military:flights:v1 does NOT need a metaKey
  - ALL_META_KEYS aggregation
  - undated payload + fresh meta → fresh
  - undated payload + STALE meta → stale (the core bug fix)
  - undated payload + missing meta → falls back to fresh
  - missing payload → missing regardless of meta
  - legacy payloads with top-level timestamp still work
  - meta without fetchedAt falls through to payload timestamp

## Verification

- node --test tests/regional-snapshot-mobility.test.mjs: 54/54 pass
  (35 original + 19 new regression tests)
- npm run test:data: 4556/4556 pass
- npm run typecheck: clean
- biome lint on touched files: clean

* feat(intelligence): regime drift history (Phase 3 PR1)

Phase 3 PR1 of the Regional Intelligence Model. Adds an append-only
regime transition log per region plus a premium-gated RPC to read it.

## What landed

### New writer module: scripts/regional-snapshot/regime-history.mjs

Single public entry point:

  recordRegimeTransition(region, snapshot, diff, opts?)
    -> { recorded, entry, pushed, trimmed }

Pure builder + Redis-ops orchestrator + dependency-injected publisher.

Flow:
  1. buildTransitionEntry() returns null when diff has no regime_changed
     (steady-state snapshots produce no entry — pure transition stream)
  2. publishTransitionWithOps() LPUSHes onto
     intelligence:regime-history:v1:{region}, then LTRIMs to keep the
     most recent REGIME_HISTORY_MAX (100) entries
  3. defaultPublisher binds real Upstash REST calls; tests inject an
     in-memory ops object for offline coverage

LTRIM failure is non-fatal — entry already landed, next cycle will
re-trim. LPUSH failure short-circuits and reports pushed=false. The
recorder NEVER throws and is wrapped in its own try/catch in the seed
loop so snapshot persist is never blocked.

### seed-regional-snapshots.mjs hook

Added a regime-history call alongside the existing alert-emitter call,
right after persistSnapshot success. Same best-effort contract:
unconditional try/catch, log warn on throw, continue main loop.

### Proto + RPC: GetRegimeHistory

  proto/worldmonitor/intelligence/v1/get_regime_history.proto

  - GetRegimeHistoryRequest { region_id, limit (0..100) }
  - GetRegimeHistoryResponse { transitions: RegimeTransition[] }
  - RegimeTransition { region_id, label, previous_label,
                       transitioned_at, transition_driver, snapshot_id }

region_id validated as strict kebab-case (same regex as
get-regional-snapshot). limit capped server-side at MAX_LIMIT=100,
defaulting to 50 when omitted.

Added to IntelligenceService in service.proto. Generated openapi
JSON/YAML committed via `make generate`.

### Server handler: server/worldmonitor/intelligence/v1/get-regime-history.ts

LRANGE-based read (newest-first because the writer LPUSHes). adapter
is a dedicated exported function adaptTransition(raw) for testability.

LRANGE helper is inlined here because server/_shared/redis.ts has no
list helpers yet — this is the first list-reading handler in the
intelligence service. If a second list reader lands, the helper can
be promoted to a shared util.

Empty list / Redis miss / failed JSON parse all return
{ transitions: [] } so the client can distinguish "never changed" from
"upstream broken" via the HTTP status code, not the body.

Registered in handler.ts.

### Premium gating + cache tier

  src/shared/premium-paths.ts:   added /api/intelligence/v1/get-regime-history
  server/gateway.ts RPC_CACHE_TIER: same path with 'slow' tier (matches
                                    route-parity contract enforced by
                                    tests/route-cache-tier.test.mjs)

## Tests — 44 new unit tests

tests/regional-snapshot-regime-history.test.mjs (22 tests):

  buildTransitionEntry (7):
    - null on missing diff/region/snapshot
    - returns entry on regime change
    - first-ever transition (empty previous_label)
    - falls back to generated_at when transitioned_at is missing
    - preserves snapshot_id

  publishTransitionWithOps (8):
    - happy path (LPUSH + LTRIM both succeed)
    - canonical key prefix
    - LTRIM uses REGIME_HISTORY_MAX-1 stop
    - LPUSH failure → not pushed, LTRIM not called
    - LTRIM failure → pushed=true, trimmed=false (non-fatal)
    - LPUSH/LTRIM throwing caught and reported
    - null/empty entry → no-op

  recordRegimeTransition (5):
    - no-op on no regime change
    - records on regime change
    - publisher returning false → recorded=false
    - publisher exceptions swallowed
    - critical escalation labels preserved

  module constants (2): key prefix + max are valid

tests/get-regime-history.test.mts (22 tests):

  adaptTransition (4):
    - all fields snake → camel
    - missing fields → empty/zero defaults
    - first-ever transition shape preserved
    - non-numeric transitioned_at → 0

  handler structural checks (7): canonical key prefix, LRANGE usage,
    adapter export, handler export signature, MAX_LIMIT cap matches
    writer, missing-region short-circuit, malformed-entry filter

  intelligence handler registration (2): import + registration

  security wiring (2): premium path + cache-tier entry

  proto definition (7): RPC method declared, import wired, request
    shape, kebab regex, limit bounds, RegimeTransition fields,
    response shape

## Verification

- node --test tests/regional-snapshot-regime-history.test.mjs: 22/22 pass
- npx tsx --test tests/get-regime-history.test.mts: 22/22 pass
- npm run test:data: 4621/4621 pass
- npm run typecheck: clean
- npm run typecheck:api: clean
- biome lint on touched files: clean

## Deferred to future iterations

- Phase 3 PR2: weekly regional briefs LLM seeder (consumes regime history
  to highlight drift events in the weekly summary)
- Phase 3 PR3: UI block in RegionalIntelligenceBoard for regime drift
  timeline (can ride alongside or after PR2)
- Drift analytics: % of last N days spent in each regime, transition
  frequency rolling window, regime cycle detection
- Alert triggers on drift cycles (e.g., "thrashed between regimes 3 times
  in 7 days")

* fix(intelligence): address 2 review findings on koala73#2981

P2 #1 — transition_driver always empty in the live path

buildRegimeState(balance, previousLabel, '') at Step 11 passed an empty
driver because the diff hasn't been computed yet. The regime-history
recorder reads snapshot.regime.transition_driver which was therefore
always '' in production, despite tests exercising synthetic fixtures
with a populated driver.

Fix: after Step 15 derives triggerReason via inferTriggerReason(diff),
backfill regime.transition_driver = triggerReason when a genuine regime
change occurred. This ensures both the persisted snapshot's regime block
AND the regime-history entry carry the real driver (e.g., 'regime_shift',
'trigger_activation', 'corridor_break').

Added 2 regression tests: populated driver flows through, and pre-fix
empty-driver snapshots remain back-compatible.

P2 #2 — Redis failure returns cached false-empty history

get-regime-history.ts returned 200 {transitions:[]} on LRANGE failure.
The gateway caches 200 GET responses at the slow tier, so a transient
Upstash outage would be pinned as a false-empty history until the cache
TTL expired.

Fix: when redisLrange returns null (Redis unavailable or credentials
missing), the response now includes upstreamUnavailable: true in the
body. The gateway already checks for this flag in the response body
(line 434) and sets Cache-Control: no-store, so transient failures are
not cached.

Added 1 structural test asserting the upstreamUnavailable flag is set.

Verification:
- 24/24 writer tests, 23/23 handler tests, 4624/4624 full suite pass
- npm run typecheck: clean
- biome lint on touched files: clean

* fix(intelligence): correct misleading 'log once per region' comment (Greptile P2)

* feat(intelligence): weekly regional briefs (Phase 3 PR2)

Phase 3 PR2 of the Regional Intelligence Model. Adds LLM-powered
weekly intelligence briefs per region, completing the core feature set.

## New seeder: scripts/seed-regional-briefs.mjs

Standalone weekly cron script (not part of the 6h derived-signals bundle).
For each non-global region:
  1. Read the latest snapshot via two-hop Redis read
  2. Read recent regime transitions from the history log (koala73#2981)
  3. Call the LLM once per region with regime trajectory + balance +
     triggers + narrative context
  4. Write structured brief to intelligence:regional-briefs:v1:weekly:{region}
     with 8-day TTL (survives one missed weekly run)

Reuses the same injectable-callLlm + parse-validation + provider-chain
pattern from narrative.mjs and weekly-brief.mjs.

## New module: scripts/regional-snapshot/weekly-brief.mjs

  generateWeeklyBrief(region, snapshot, transitions, opts?)
    -> { region_id, generated_at, period_start, period_end,
         situation_recap, regime_trajectory, key_developments[],
         risk_outlook, provider, model }

  buildBriefPrompt()    — pure prompt builder
  parseBriefJson()      — JSON parser with prose-extraction fallback
  emptyBrief()          — canonical empty shape

Global region is skipped. Provider chain: Groq -> OpenRouter. Validate
callback ensures only parseable responses pass (narrative.mjs PR koala73#2960
review fix pattern).

## Proto + RPC: GetRegionalBrief

  proto/worldmonitor/intelligence/v1/get_regional_brief.proto

  - GetRegionalBriefRequest { region_id }
  - GetRegionalBriefResponse { brief: RegionalBrief }
  - RegionalBrief { region_id, generated_at, period_start, period_end,
                    situation_recap, regime_trajectory,
                    key_developments[], risk_outlook, provider, model }

## Server handler

  server/worldmonitor/intelligence/v1/get-regional-brief.ts

Simple getCachedJson read + adaptBrief snake->camel adapter.
Returns upstreamUnavailable: true on Redis failure so the gateway
skips caching (matching the get-regime-history pattern from koala73#2981).

## Premium gating + cache tier

  src/shared/premium-paths.ts + server/gateway.ts RPC_CACHE_TIER

## Tests — 27 new unit tests

  buildBriefPrompt (5): region/balance/transitions/narrative rendered,
                        empty transitions handled, missing fields tolerated
  parseBriefJson (5): valid JSON, garbage, all-empty, cap at 5, prose extraction
  generateWeeklyBrief (6): success, global skip, LLM fail, garbage, exception,
                           period_start/end delta
  emptyBrief (2): region_id + empty fields
  handler (4): key prefix, adapter export, upstreamUnavailable, registration
  security (2): premium path + cache tier
  proto (3): RPC declared, import wired, RegionalBrief fields

## Verification

- npm run test:data: 4651/4651 pass
- npm run typecheck + typecheck:api: clean
- biome lint: clean

* fix(intelligence): address 3 review findings on koala73#2989

P2 #1 — no consumer surface for GetRegionalBrief

Acknowledged. The consumer is the RegionalIntelligenceBoard panel,
which will call GetRegionalBrief and render a weekly brief block.
This wiring is Phase 3 PR3 (UI) scope — the RPC + Redis key are the
delivery mechanism, not the end surface. No code change in this commit;
the RPC is ready for the panel to consume.

P2 #2 — readRecentTransitions collapses failure to []

readRecentTransitions returned [] on Redis/network failure, which is
indistinguishable from a genuinely quiet week. The LLM then generates
a brief claiming "no regime transitions" when in reality the upstream
is down — fabricating false input.

Fix: return null on failure. The seeder skips the region with a clear
log message when transitions is null, so the brief is never written
with unreliable input. Empty array [] now only means genuinely no
transitions in the 7-day window.

P2 #3 — parseBriefJson accepts briefs the seeder rejects

parseBriefJson treated non-empty key_developments as valid even if
situation_recap was empty. The seeder gate only writes when
brief.situation_recap is truthy. That mismatch means the validator
pass + provider-fallback logic could accept a response that the seeder
then silently drops.

Fix: require situation_recap in parseBriefJson for valid=true, matching
the seeder gate. Now both checks agree on what constitutes a usable
brief, and the provider-fallback chain correctly falls through when
a provider returns a brief with developments but no recap.

* fix(intelligence): TTL path-segment fix + seed-meta always-write (Greptile P1+P2 on koala73#2989)

P1 — TTL silently not applied (briefs never expire)

Upstash REST ignores query-string SET options (?EX=N). The correct
form is path-segment: /set/{key}/{value}/EX/{seconds}. Without this
fix every brief persists indefinitely and Redis storage grows
unboundedly across weekly runs.

P2 — seed-meta not written when all regions skipped

writeExtraKeyWithMeta was gated on generated > 0. If every region
was skipped (no snapshot yet, or LLM failed), seed-meta was never
written, making the seeder indistinguishable from "never ran" in
health tooling. Now writes seed-meta whenever failed === 0,
carrying regionsSkipped count.

P2 #3 (validate gate) — already fixed in previous commit (parseBriefJson
now requires situation_recap for valid=true).

* fix(intelligence): register regional-briefs in health.js SEED_META + STANDALONE_KEYS (review P2 on koala73#2989)

* fix(intelligence): register regional-briefs in api/seed-health.js (review P2 on koala73#2989)

* fix(intelligence): raise brief TTL to 15 days to cover missed weekly cycle (review P2 on koala73#2989)

* fix(intelligence): distinguish missing-key from Redis-error + coverage-gated health (review P2s on koala73#2989)

P2 #1 — false upstreamUnavailable before first seed

getCachedJson returns null for both "key missing" and "Redis failed",
so the handler was advertising an outage for every region before the
first weekly seed ran. Switched to getRawJson (throws on Redis errors)
so null = genuinely missing key → clean empty 200, and thrown error =
upstream failure → upstreamUnavailable: true for gateway no-store.

P2 #2 — partial run hides coverage loss in health

The seed-meta was written with generated count even if only 1 of 7
regions produced a brief. /api/health treats any positive recordCount
as healthy, so broad regional failure was invisible to operators.

Fix: recordCount is set to 0 when generated < ceil(expectedRegions/2).
This makes /api/health report EMPTY_DATA for severely partial runs
while still writing seed-meta (so the seeder is confirmed to have run).
coverageOk flag in the summary payload lets operators drill into the
exact coverage state.

* fix(intelligence): tighten coverage gate to expectedRegions-1 (review P2 on koala73#2989)

…ss (koala73#3134)

* fix(sector-valuations): proxy Yahoo quoteSummary via Decodo curl egress

Yahoo's /v10/finance/quoteSummary returns HTTP 401 from Railway container
IPs. Railway logs 2026-04-16 show all 12 sector ETFs failing every 5-min
cron:

  [Sector] Yahoo quoteSummary XLK HTTP 401  (x12 per tick)
  [Market] Seeded 12/12 sectors, 0 valuations

Add a curl-based proxy fallback that matches scripts/_yahoo-fetch.mjs:
hit us.decodo.com (curl egress pool) NOT gate.decodo.com (CONNECT egress
pool). Per the 2026-04-16 probe documented in _yahoo-fetch.mjs header,
Yahoo blocks Decodo's CONNECT egress IPs but accepts the curl egress.
Reusing ytFetchViaProxy here would keep failing silently.

Shares the existing _yahooProxyFailCount / _yahooProxyCooldownUntil
cooldown state with fetchYahooChartDirect so both Yahoo paths pause
together if Decodo's curl pool also gets blocked.

No change to direct-path behavior when Yahoo is healthy.

* fix(sector-valuations): don't proxy on empty quoteSummary result (review)

Direct 200 with data.quoteSummary.result[0] absent is an app-level "no
data for this symbol" signal (e.g. delisted ETF). Proxy won't return
different data for a symbol Yahoo itself doesn't carry — falling back
would burn the 5-failure cooldown budget on structurally empty symbols
and mask a genuine proxy outage.

Resolve null on !result; keep JSON.parse catch going to proxy (garbage
body IS a transport-level signal — captive portal, Cloudflare challenge).

Review feedback from PR koala73#3134.

* fix(sector-valuations): split cooldown per egress route, cover transport failures (review)

Review feedback on PR koala73#3134, both P1.

P1 #1 — transport failures bypassed cooldown
  execFileSync timeouts, proxy-connect refusals, and JSON.parse on garbage
  bodies all went through the catch block and returned null without
  ticking _yahooProxyFailCount. In the exact failure mode this PR hardens
  against, the relay would have thrashed through 12 × 20s curl attempts
  per tick with no backoff. Extract a bumpCooldown() helper and call it
  from both the non-2xx branch and the catch block.

P1 #2 — two Decodo egress pools shared one cooldown budget
  fetchYahooChartDirect uses CONNECT via gate.decodo.com.
  _yahooQuoteSummaryProxyFallback uses curl via us.decodo.com.
  These are independent egress IP pools — per the 2026-04-16 probe,
  Yahoo blocks CONNECT but accepts curl. Sharing cooldown means 5
  CONNECT failures suppress the healthy curl path (and vice versa).
  Split into _yahooConnectProxy* (chart) and _yahooCurlProxy* (sector
  valuations).

Also: on proxy 200 with empty result, reset the curl counter. The route
is healthy even if this specific symbol has no data — don't pretend it's
a failure.

* fix(sector-valuations): non-blocking curl + settle guard (review round 3)

Review feedback on PR koala73#3134, both P1.

P1 #1 - double proxy invocation on timeout/error race
  req.destroy() inside the timeout handler can still emit 'error', and
  both handlers eagerly called resolve(_yahooQuoteSummaryProxyFallback(...)).
  A single upstream timeout therefore launched two curl subprocesses,
  double-ticked the cooldown counter, and blocked twice. Add a settled
  flag; settle() exits early on the second handler before evaluating the
  fallback.

P1 #2 - execFileSync blocks the relay event loop
  The relay serves HTTP/WS traffic on the same thread that awaits
  seedSectorSummary's per-symbol Yahoo fetch. execFileSync for up to 20s
  per failure x 5 failures before cooldown = ~100s of frozen event loop.
  Switch to promisify(execFile). resolve(promise) chains the Promise
  through fetchYahooQuoteSummary's outer Promise, so the main-loop await
  yields while curl runs. Other traffic continues during the fetch.

tests/sector-valuations.test.mjs: bump the static-analysis window from
1500 to 2000 chars so the field-extraction markers (ytdReturn etc.)
stay inside the window after the settle guard was added.

… gauges (koala73#3141)

* feat(positioning): standalone 24/7 Positioning panel with directional gauges

Extract Hyperliquid perp positioning from the "Perp Flow" tab inside
CommoditiesPanel into a dedicated "24/7 Positioning" panel (koala73#3140).

New panel features:
- SVG arc gauge (0-100) per asset with color encoding:
  green = bullish/longs crowded, red = bearish/shorts crowded
- Grouped by asset class: Commodities, Crypto, FX
- Visual elevation (glow + border) for scores above 40
- Click-through to relevant panels (crypto, commodities)
- Hover tooltips with score, funding rate, OI delta
- Warmup banner, stale badges, unavailable state
- Compact secondary metrics (funding rate + OI delta 1h)
- Mobile responsive (2-column grid below 600px)

Registration: panel-layout.ts, App.ts (primeTask + 5min refresh),
commands.ts (CMD+K), panels.ts (all variants), index.ts export,
en.json i18n keys.

Removals from CommoditiesPanel:
- "Perp Flow" tab and its tab type
- fetchHyperliquidFlow() method
- _renderFlow() and _renderFlowGrid() methods
- Associated state (_flow, _flowLoading)
- App.ts primeTask and refreshScheduler entries

No backend changes. Reuses existing getHyperliquidFlow RPC and
market:hyperliquid-flow:v1 Redis key.

* fix(positioning): correct oiDelta lookback, click targets, warmup state (review)

P1 #1: oiDelta1h used 2-point lookback (5min delta mislabeled as 1h).
  Restored to 13-point lookback matching the original MarketPanel.ts
  implementation (12 samples back = 1h at 5min cadence).

P1 #2: CLICK_TARGETS used display-style names (GOLD, WTI, BRENT) but
  seeded symbols are xyz:-prefixed Hyperliquid names (xyz:GOLD, xyz:CL,
  xyz:BRENTOIL). Fixed to match actual seeder output from
  seed-hyperliquid-flow.mjs.

P2: Empty/unavailable snapshots (cold seed, fresh deploy) now show
  warmup guidance instead of a hard error. The prior Perp Flow tab
  intentionally treated this as normal bootstrap. The new panel
  regressed to showError which is confusing on first deploy.

* fix(positioning): use data-panel selector, guard missing target panels (review)

P1: Click-through used getElementById('panel-X') but panels mount with
data-panel="X" attribute. Changed to querySelector('[data-panel="X"]').

P2: Crypto cards (BTC/ETH/SOL) target the 'crypto' panel which doesn't
exist in the commodity variant. resolveClickTarget() now checks if the
target panel is actually in the DOM before adding clickable styling and
data-pos-navigate. Cards whose target panel is absent render as
non-clickable (no cursor pointer, no navigate attribute).

koala73#3204)

* fix(brief): bundle resvg linux-x64-gnu native binding with carousel fn

Real root cause of every Telegram carousel WEBPAGE_CURL_FAILED
since PR koala73#3174 merged. Not middleware (last PR fixed that
theoretical path but not the observed failure). The Vercel
function itself crashes HTTP 500 FUNCTION_INVOCATION_FAILED on
every request including OPTIONS - the isolate can't initialise.

The handler imports brief-carousel-render which lazy-imports
@resvg/resvg-js. That package's js-binding.js does runtime
require(@resvg/resvg-js-<platform>-<arch>-<libc>). On Vercel
Lambda (Amazon Linux 2 glibc) that resolves to
@resvg/resvg-js-linux-x64-gnu. Vercel nft tracing does NOT
follow this conditional require so the optional peer package
isnt bundled. Cold start throws MODULE_NOT_FOUND, isolate
crashes, Vercel returns FUNCTION_INVOCATION_FAILED, Telegram
reports WEBPAGE_CURL_FAILED.

Fix: vercel.json functions.includeFiles forces linux-x64-gnu
binding into the carousel functions bundle. Only this route
needs it; every other api route is unaffected.

Verified:
- deploy-config tests 21/21 pass
- JSON valid
- Reproduced 500 via curl on all methods and UAs
- resvg-js/js-binding.js confirms linux-x64-gnu is the runtime
  binary on Amazon Linux 2 glibc

Post-merge: curl with TelegramBot UA should return 200 image/png
instead of 500; next cron tick should clear the Railway
[digest] Telegram carousel 400 line.

* Address Greptile P2s: regression guard + arch-assumption reasoning

Two P2 findings on PR koala73#3204:

P2 #1 (inline on vercel.json:6): Platform architecture assumption
undocumented. If Vercel migrates to Graviton/arm64 Lambda the
cold-start crash silently returns. vercel.json is strict JSON so
comments aren't possible inline.

P2 #2 (tests/deploy-config.test.mjs:17): No regression guard for
the carousel includeFiles rule. A future vercel.json tidy-up
could silently revert the fix with no CI signal.

Fixed both in a single block:

- New describe() in deploy-config.test.mjs asserts the carousel
  route's functions entry exists AND its includeFiles points at
  @resvg/resvg-js-linux-x64-gnu. Any drift fails the build.
- The block comment above it documents the Amazon Linux 2 x86_64
  glibc assumption that would have lived next to the includeFiles
  entry if JSON supported comments. Includes the Graviton/arm64
  migration pointer.

tests 22/22 pass (was 21, +1 new).

…a73#3207) (koala73#3242)

* chore(api): enforce sebuf contract via exceptions manifest (koala73#3207)

Adds api/api-route-exceptions.json as the single source of truth for
non-proto /api/ endpoints, with scripts/enforce-sebuf-api-contract.mjs
gating every PR via npm run lint:api-contract. Fixes the root-only blind
spot in the prior allowlist (tests/edge-functions.test.mjs), which only
scanned top-level *.js files and missed nested paths and .ts endpoints —
the gap that let api/supply-chain/v1/country-products.ts and friends
drift under proto domain URL prefixes unchallenged.

Checks both directions: every api/<domain>/v<N>/[rpc].ts must pair with
a generated service_server.ts (so a deleted proto fails CI), and every
generated service must have an HTTP gateway (no orphaned generated code).

Manifest entries require category + reason + owner, with removal_issue
mandatory for temporary categories (deferred, migration-pending) and
forbidden for permanent ones. .github/CODEOWNERS pins the manifest to
@SebastienMelki so new exceptions don't slip through review.

The manifest only shrinks: migration-pending entries (19 today) will be
removed as subsequent commits in this PR land each migration.

* refactor(maritime): migrate /api/ais-snapshot → maritime/v1.GetVesselSnapshot (koala73#3207)

The proto VesselSnapshot was carrying density + disruptions but the frontend
also needed sequence, relay status, and candidate_reports to drive the
position-callback system. Those only lived on the raw relay passthrough, so
the client had to keep hitting /api/ais-snapshot whenever callbacks were
registered and fall back to the proto RPC only when the relay URL was gone.

This commit pushes all three missing fields through the proto contract and
collapses the dual-fetch-path into one proto client call.

Proto changes (proto/worldmonitor/maritime/v1/):
  - VesselSnapshot gains sequence, status, candidate_reports.
  - GetVesselSnapshotRequest gains include_candidates (query: include_candidates).

Handler (server/worldmonitor/maritime/v1/get-vessel-snapshot.ts):
  - Forwards include_candidates to ?candidates=... on the relay.
  - Separate 5-min in-memory caches for the candidates=on and candidates=off
    variants; they have very different payload sizes and should not share a slot.
  - Per-request in-flight dedup preserved per-variant.

Frontend (src/services/maritime/index.ts):
  - fetchSnapshotPayload now calls MaritimeServiceClient.getVesselSnapshot
    directly with includeCandidates threaded through. The raw-relay path,
    SNAPSHOT_PROXY_URL, DIRECT_RAILWAY_SNAPSHOT_URL and LOCAL_SNAPSHOT_FALLBACK
    are gone — production already routed via Vercel, the "direct" branch only
    ever fired on localhost, and the proto gateway covers both.
  - New toLegacyCandidateReport helper mirrors toDensityZone/toDisruptionEvent.

api/ais-snapshot.js deleted; manifest entry removed. Only reduced the codegen
scope to worldmonitor.maritime.v1 (buf generate --path) — regenerating the
full tree drops // @ts-nocheck from every client/server file and surfaces
pre-existing type errors across 30+ unrelated services, which is not in
scope for this PR.

Shape-diff vs legacy payload:
  - disruptions / density: proto carries the same fields, just with the
    GeoCoordinates wrapper and enum strings (remapped client-side via
    existing toDisruptionEvent / toDensityZone helpers).
  - sequence, status.{connected,vessels,messages}: now populated from the
    proto response — was hardcoded to 0/false in the prior proto fallback.
  - candidateReports: same shape; optional numeric fields come through as
    0 instead of undefined, which the legacy consumer already handled.

* refactor(sanctions): migrate /api/sanctions-entity-search → LookupSanctionEntity (koala73#3207)

The proto docstring already claimed "OFAC + OpenSanctions" coverage but the
handler only fuzzy-matched a local OFAC Redis index — narrower than the
legacy /api/sanctions-entity-search, which proxied OpenSanctions live (the
source advertised in docs/api-proxies.mdx). Deleting the legacy without
expanding the handler would have been a silent coverage regression for
external consumers.

Handler changes (server/worldmonitor/sanctions/v1/lookup-entity.ts):
  - Primary path: live search against api.opensanctions.org/search/default
    with an 8s timeout and the same User-Agent the legacy edge fn used.
  - Fallback path: the existing OFAC local fuzzy match, kept intact for when
    OpenSanctions is unreachable / rate-limiting.
  - Response source field flips between 'opensanctions' (happy path) and
    'ofac' (fallback) so clients can tell which index answered.
  - Query validation tightened: rejects q > 200 chars (matches legacy cap).

Rate limiting:
  - Added /api/sanctions/v1/lookup-entity to ENDPOINT_RATE_POLICIES at 30/min
    per IP — matches the legacy createIpRateLimiter budget. The gateway
    already enforces per-endpoint policies via checkEndpointRateLimit.

Docs:
  - docs/api-proxies.mdx — dropped the /api/sanctions-entity-search row
    (plus the orphaned /api/ais-snapshot row left over from the previous
    commit in this PR).
  - docs/panels/sanctions-pressure.mdx — points at the new RPC URL and
    describes the OpenSanctions-primary / OFAC-fallback semantics.

api/sanctions-entity-search.js deleted; manifest entry removed.

* refactor(military): migrate /api/military-flights → ListMilitaryFlights (koala73#3207)

Legacy /api/military-flights read a pre-baked Redis blob written by the
seed-military-flights cron and returned flights in a flat app-friendly
shape (lat/lon, lowercase enums, lastSeenMs). The proto RPC takes a bbox,
fetches OpenSky live, classifies server-side, and returns nested
GeoCoordinates + MILITARY_*_TYPE_* enum strings + lastSeenAt — same data,
different contract.

fetchFromRedis in src/services/military-flights.ts was doing nothing
sebuf-aware. Renamed it to fetchViaProto and rewrote to:

  - Instantiate MilitaryServiceClient against getRpcBaseUrl().
  - Iterate MILITARY_QUERY_REGIONS (PACIFIC + WESTERN) in parallel — same
    regions the desktop OpenSky path and the seed cron already use, so
    dashboard coverage tracks the analytic pipeline.
  - Dedup by hexCode across regions.
  - Map proto → app shape via new mapProtoFlight helper plus three reverse
    enum maps (AIRCRAFT_TYPE_REVERSE, OPERATOR_REVERSE, CONFIDENCE_REVERSE).

The seed cron (scripts/seed-military-flights.mjs) stays put: it feeds
regional-snapshot mobility, cross-source signals, correlation, and the
health freshness check (api/health.js: 'military:flights:v1'). None of
those read the legacy HTTP endpoint; they read the Redis key directly.
The proto handler uses its own per-bbox cache keys under the same prefix,
so dashboard traffic no longer races the seed cron's blob — the two paths
diverge by a small refresh lag, which is acceptable.

Docs: dropped the /api/military-flights row from docs/api-proxies.mdx.

api/military-flights.js deleted; manifest entry removed.

Shape-diff vs legacy:
  - f.location.{latitude,longitude} → f.lat, f.lon
  - f.aircraftType: MILITARY_AIRCRAFT_TYPE_TANKER → 'tanker' via reverse map
  - f.operator: MILITARY_OPERATOR_USAF → 'usaf' via reverse map
  - f.confidence: MILITARY_CONFIDENCE_LOW → 'low' via reverse map
  - f.lastSeenAt (number) → f.lastSeen (Date)
  - f.enrichment → f.enriched (with field renames)
  - Extra fields registration / aircraftModel / origin / destination /
    firstSeenAt now flow through where proto populates them.

* fix(supply-chain): thread includeCandidates through chokepoint status (koala73#3207)

Caught by tsconfig.api.json typecheck in the pre-push hook (not covered
by the plain tsc --noEmit run that ran before I pushed the ais-snapshot
commit). The chokepoint status handler calls getVesselSnapshot internally
with a static no-auth request — now required to include the new
includeCandidates bool from the proto extension.

Passing false: server-internal callers don't need per-vessel reports.

* test(maritime): update getVesselSnapshot cache assertions (koala73#3207)

The ais-snapshot migration replaced the single cachedSnapshot/cacheTimestamp
pair with a per-variant cache so candidates-on and candidates-off payloads
don't evict each other. Pre-push hook surfaced that tests/server-handlers
still asserted the old variable names. Rewriting the assertions to match
the new shape while preserving the invariants they actually guard:

  - Freshness check against slot TTL.
  - Cache read before relay call.
  - Per-slot in-flight dedup.
  - Stale-serve on relay failure (result ?? slot.snapshot).

* chore(proto): restore // @ts-nocheck on regenerated maritime files (koala73#3207)

I ran 'buf generate --path worldmonitor/maritime/v1' to scope the proto
regen to the one service I was changing (to avoid the toolchain drift
that drops @ts-nocheck from 60+ unrelated files — separate issue). But
the repo convention is the 'make generate' target, which runs buf and
then sed-prepends '// @ts-nocheck' to every generated .ts file. My
scoped command skipped the sed step. The proto-check CI enforces the
sed output, so the two maritime files need the directive restored.

* refactor(enrichment): decomm /api/enrichment/{company,signals} legacy edge fns (koala73#3207)

Both endpoints were already ported to IntelligenceService:
  - getCompanyEnrichment  (/api/intelligence/v1/get-company-enrichment)
  - listCompanySignals    (/api/intelligence/v1/list-company-signals)

No frontend callers of the legacy /api/enrichment/* paths exist. Removes:
  - api/enrichment/company.js, signals.js, _domain.js
  - api-route-exceptions.json migration-pending entries (58 remain)
  - docs/api-proxies.mdx rows for /api/enrichment/{company,signals}
  - docs/architecture.mdx reference updated to the IntelligenceService RPCs

Verified: typecheck, typecheck:api, lint:api-contract (89 files / 58 entries),
lint:boundaries, tests/edge-functions.test.mjs (136 pass),
tests/enrichment-caching.test.mjs (14 pass — still guards the intelligence/v1
handlers), make generate is zero-diff.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor(leads): migrate /api/{contact,register-interest} → LeadsService (koala73#3207)

New leads/v1 sebuf service with two POST RPCs:
  - SubmitContact    → /api/leads/v1/submit-contact
  - RegisterInterest → /api/leads/v1/register-interest

Handler logic ported 1:1 from api/contact.js + api/register-interest.js:
  - Turnstile verification (desktop sources bypass, preserved)
  - Honeypot (website field) silently accepts without upstream calls
  - Free-email-domain gate on SubmitContact (422 ApiError)
  - validateEmail (disposable/offensive/typo-TLD/MX) on RegisterInterest
  - Convex writes via ConvexHttpClient (contactMessages:submit, registerInterest:register)
  - Resend notification + confirmation emails (HTML templates unchanged)

Shared helpers moved to server/_shared/:
  - turnstile.ts (getClientIp + verifyTurnstile)
  - email-validation.ts (disposable/offensive/MX checks)

Rate limits preserved via ENDPOINT_RATE_POLICIES:
  - submit-contact:    3/hour per IP (was in-memory 3/hr)
  - register-interest: 5/hour per IP (was in-memory 5/hr; desktop
    sources previously capped at 2/hr via shared in-memory map —
    now 5/hr like everyone else, accepting the small regression in
    exchange for Upstash-backed global limiting)

Callers updated:
  - pro-test/src/App.tsx contact form → new submit-contact path
  - src-tauri/sidecar/local-api-server.mjs cloud-fallback rewrites
    /api/register-interest → /api/leads/v1/register-interest when
    proxying; keeps local path for older desktop builds
  - src/services/runtime.ts isKeyFreeApiTarget allows both old and
    new paths through the WORLDMONITOR_API_KEY-optional gate

Tests:
  - tests/contact-handler.test.mjs rewritten to call submitContact
    handler directly; asserts on ValidationError / ApiError
  - tests/email-validation.test.mjs + tests/turnstile.test.mjs
    point at the new server/_shared/ modules

Deleted: api/contact.js, api/register-interest.js, api/_ip-rate-limit.js,
api/_turnstile.js, api/_email-validation.js, api/_turnstile.test.mjs.
Manifest entries removed (58 → 56). Docs updated (api-platform,
api-commerce, usage-rate-limits).

Verified: npm run typecheck + typecheck:api + lint:api-contract
(88 files / 56 entries) + lint:boundaries pass; full test:data
(5852 tests) passes; make generate is zero-diff.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* chore(pro-test): rebuild bundle for leads/v1 contact form (koala73#3207)

Updates the enterprise contact form to POST to /api/leads/v1/submit-contact
(old path /api/contact removed in the previous commit).

Bundle is rebuilt from pro-test/src/App.tsx source change in 9ccd309.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review): address HIGH review findings 1-3 (koala73#3207)

Three review findings from @koala73 on the sebuf-migration PR, all
silent bugs that would have shipped to prod:

### 1. Sanctions rate-limit policy was dead code

ENDPOINT_RATE_POLICIES keyed the 30/min budget under
/api/sanctions/v1/lookup-entity, but the generated route (from the
proto RPC LookupSanctionEntity) is /api/sanctions/v1/lookup-sanction-entity.
hasEndpointRatePolicy / getEndpointRatelimit are exact-string pathname
lookups, so the mismatch meant the endpoint fell through to the
generic 600/min global limiter instead of the advertised 30/min.

Net effect: the live OpenSanctions proxy endpoint (unauthenticated,
external upstream) had 20x the intended rate budget. Fixed by renaming
the policy key to match the generated route.

### 2. Lost stale-seed fallback on military-flights

Legacy api/military-flights.js cascaded military:flights:v1 →
military:flights:stale:v1 before returning empty. The new proto
handler went straight to live OpenSky/relay and returned null on miss.

Relay or OpenSky hiccup used to serve stale seeded data (24h TTL);
under the new handler it showed an empty map. Both keys are still
written by scripts/seed-military-flights.mjs on every run — fix just
reads the stale key when the live fetch returns null, converts the
seed's app-shape flights (flat lat/lon, lowercase enums, lastSeenMs)
to the proto shape (nested GeoCoordinates, enum strings, lastSeenAt),
and filters to the request bbox.

Read via getRawJson (unprefixed) to match the seed cron's writes,
which bypass the env-prefix system.

### 3. Hex-code casing mismatch broke getFlightByHex

The seed cron writes hexCode: icao24.toUpperCase() (uppercase);
src/services/military-flights.ts:getFlightByHex uppercases the lookup
input: f.hexCode === hexCode.toUpperCase(). The new proto handler
preserved OpenSky's lowercase icao24, and mapProtoFlight is a
pass-through. getFlightByHex was silently returning undefined for
every call after the migration.

Fix: uppercase in the proto handler (live + stale paths), and document
the invariant in a comment on MilitaryFlight.hex_code in
military_flight.proto so future handlers don't re-break it.

### Verified

- typecheck + typecheck:api clean
- lint:api-contract (56 entries) / lint:boundaries clean
- tests/edge-functions.test.mjs 130 pass
- make generate zero-diff (openapi spec regenerated for proto comment)

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review): restore desktop 2/hr rate cap on register-interest (koala73#3207)

Addresses HIGH review finding #4 from @koala73. The legacy
api/register-interest.js applied a nested 2/hr per-IP cap when
`source === 'desktop-settings'`, on top of the generic 5/hr endpoint
budget. The sebuf migration lost this — desktop-source requests now
enjoy the full 5/hr cap.

Since `source` is an unsigned client-supplied field, anyone sending
`source: 'desktop-settings'` skips Turnstile AND gets 5/hr. Without
the tighter cap the Turnstile bypass is cheaper to abuse.

Added `checkScopedRateLimit` to `server/_shared/rate-limit.ts` — a
reusable second-stage Upstash limiter keyed on an opaque scope string
+ caller identifier. Fail-open on Redis errors to match existing
checkRateLimit / checkEndpointRateLimit semantics. Handlers that need
per-subscope caps on top of the gateway-level endpoint budget use this
helper.

In register-interest: when `isDesktopSource`, call checkScopedRateLimit
with scope `/api/leads/v1/register-interest#desktop`, limit=2, window=1h,
IP as identifier. On exceeded → throw ApiError(429).

### What this does not fix

This caps the blast radius of the Turnstile bypass but does not close
it — an attacker sending `source: 'desktop-settings'` still skips
Turnstile (just at 2/hr instead of 5/hr). The proper fix is a signed
desktop-secret header that authenticates the bypass; filed as
follow-up koala73#3252. That requires coordinated Tauri build + Vercel env
changes out of scope for koala73#3207.

### Verified

- typecheck + typecheck:api clean
- lint:api-contract (56 entries)
- tests/edge-functions.test.mjs + contact-handler.test.mjs (147 pass)

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review): MEDIUM + LOW + rate-limit-policy CI check (koala73#3207)

Closes out the remaining @koala73 review findings from koala73#3242 that
didn't already land in the HIGH-fix commits, plus the requested CI
check that would have caught HIGH #1 (dead-code policy key) at
review time.

### MEDIUM #5 — Turnstile missing-secret policy default

Flip `verifyTurnstile`'s default `missingSecretPolicy` from `'allow'`
to `'allow-in-development'`. Dev with no secret = pass (expected
local); prod with no secret = reject + log. submit-contact was
already explicitly overriding to `'allow-in-development'`;
register-interest was silently getting `'allow'`. Safe default now
means a future missing-secret misconfiguration in prod gets caught
instead of silently letting bots through. Removed the now-redundant
override in submit-contact.

### MEDIUM #6 — Silent enum fallbacks in maritime client

`toDisruptionEvent` mapped `AIS_DISRUPTION_TYPE_UNSPECIFIED` / unknown
enum values → `gap_spike` / `low` silently. Refactored to return null
when either enum is unknown; caller filters nulls out of the array.
Handler doesn't produce UNSPECIFIED today, but the `gap_spike`
default would have mislabeled the first new enum value the proto
ever adds — dropping unknowns is safer than shipping wrong labels.

### LOW — Copy drift in register-interest email

Email template hardcoded `435+ Sources`; PR koala73#3241 bumped marketing to
`500+`. Bumped in the rewritten file to stay consistent.

The `as any` on Convex mutation names carried over from legacy and
filed as follow-up koala73#3253.

### Rate-limit-policy coverage lint

`scripts/enforce-rate-limit-policies.mjs` validates every key in
`ENDPOINT_RATE_POLICIES` resolves to a proto-generated gateway route
by cross-referencing `docs/api/*.openapi.yaml`. Fails with the
sanctions-entity-search incident referenced in the error message so
future drift has a paper trail.

Wired into package.json (`lint:rate-limit-policies`) and the pre-push
hook alongside `lint:boundaries`. Smoke-tested both directions —
clean repo passes (5 policies / 175 routes), seeded drift (the exact
HIGH #1 typo) fails with the advertised remedy text.

### Verified
- `lint:rate-limit-policies` ✓
- `typecheck` + `typecheck:api` ✓
- `lint:api-contract` ✓ (56 entries)
- `lint:boundaries` ✓
- edge-functions + contact-handler tests (147 pass)

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor(commit 5): decomm /api/eia/* + migrate /api/satellites → IntelligenceService (koala73#3207)

Both targets turned out to be decomm-not-migration cases. The original
plan called for two new services (economic/v1.GetEiaSeries +
natural/v1.ListSatellitePositions) but research found neither was
needed:

### /api/eia/[[...path]].js — pure decomm, zero consumers

The "catch-all" is a misnomer — only two paths actually worked,
/api/eia/health and /api/eia/petroleum, both Redis-only readers.
Zero frontend callers in src/. Zero server-side readers. Nothing
consumes the `energy:eia-petroleum:v1` key that seed-eia-petroleum.mjs
writes daily.

The EIA data the frontend actually uses goes through existing typed
RPCs in economic/v1: GetEnergyPrices, GetCrudeInventories,
GetNatGasStorage, GetEnergyCapacity. None of those touch /api/eia/*.

Building GetEiaSeries would have been dead code. Deleted the legacy
file + its test (tests/api-eia-petroleum.test.mjs — it only covered
the legacy endpoint, no behavior to preserve). Empty api/eia/ dir
removed.

**Note for review:** the Redis seed cron keeps running daily and
nothing consumes it. If that stays unused, seed-eia-petroleum.mjs
should be retired too (separate PR). Out of scope for sebuf-migration.

### /api/satellites.js — Learning #2 strikes again

IntelligenceService.ListSatellites already exists at
/api/intelligence/v1/list-satellites, reads the same Redis key
(intelligence:satellites:tle:v1), and supports an optional country
filter the legacy didn't have.

One frontend caller in src/services/satellites.ts needed to switch
from `fetch(toApiUrl('/api/satellites'))` to the typed
IntelligenceServiceClient.listSatellites. Shape diff was tiny —
legacy `noradId` became proto `id` (handler line 36 already picks
either), everything else identical. alt/velocity/inclination in the
proto are ignored by the caller since it propagates positions
client-side via satellite.js.

Kept the client-side cache + failure cooldown + 20s timeout (still
valid concerns at the caller level).

### Manifest + docs
- api-route-exceptions.json: 56 → 54 entries (both removed)
- docs/api-proxies.mdx: dropped the two rows from the Raw-data
  passthroughs table

### Verified
- typecheck + typecheck:api ✓
- lint:api-contract (54 entries) / lint:boundaries / lint:rate-limit-policies ✓
- tests/edge-functions.test.mjs 127 pass (down from 130 — 3 tests were
  for the deleted eia endpoint)
- make generate zero-diff (no proto changes)

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor(commit 6): migrate /api/supply-chain/v1/{country-products,multi-sector-cost-shock} → SupplyChainService (koala73#3207)

Both endpoints were hand-rolled TS handlers sitting under a proto URL prefix —
the exact drift the manifest guardrail flagged. Promoted both to typed RPCs:

- GetCountryProducts → /api/supply-chain/v1/get-country-products
- GetMultiSectorCostShock → /api/supply-chain/v1/get-multi-sector-cost-shock

Handlers preserve the existing semantics: PRO-gate via isCallerPremium(ctx.request),
iso2 / chokepointId validation, raw bilateral-hs4 Redis read (skip env-prefix to
match seeder writes), CHOKEPOINT_STATUS_KEY for war-risk tier, and the math from
_multi-sector-shock.ts unchanged. Empty-data and non-PRO paths return the typed
empty payload (no 403 — the sebuf gateway pattern is empty-payload-on-deny).

Client wrapper switches from premiumFetch to client.getCountryProducts/
client.getMultiSectorCostShock. Legacy MultiSectorShock / MultiSectorShockResponse /
CountryProductsResponse names remain as type aliases of the generated proto types
so CountryBriefPanel + CountryDeepDivePanel callsites compile with zero churn.

Manifest 54 → 52. Rate-limit gateway routes 175 → 177.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(gateway): add cache-tier entries for new supply-chain RPCs (koala73#3207)

Pre-push tests/route-cache-tier.test.mjs caught the missing entries.
Both PRO-gated, request-varying — match the existing supply-chain PRO cohort
(get-country-cost-shock, get-bypass-options, etc.) at slow-browser tier.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor(commit 7): migrate /api/scenario/v1/{run,status,templates} → ScenarioService (koala73#3207)

Promote the three literal-filename scenario endpoints to a typed sebuf
service with three RPCs:

  POST /api/scenario/v1/run-scenario        (RunScenario)
  GET  /api/scenario/v1/get-scenario-status (GetScenarioStatus)
  GET  /api/scenario/v1/list-scenario-templates (ListScenarioTemplates)

Preserves all security invariants from the legacy handlers:
- 405 for wrong method (sebuf service-config method gate)
- scenarioId validation against SCENARIO_TEMPLATES registry
- iso2 regex ^[A-Z]{2}$
- JOB_ID_RE path-traversal guard on status
- Per-IP 10/min rate limit (moved to gateway ENDPOINT_RATE_POLICIES)
- Queue-depth backpressure (>100 → 429)
- PRO gating via isCallerPremium
- AbortSignal.timeout on every Redis pipeline (runRedisPipeline helper)

Wire-level diffs vs legacy:
- Per-user RL now enforced at the gateway (same 10/min/IP budget).
- Rate-limit response omits Retry-After header; retryAfter is in the
  body per error-mapper.ts convention.
- ListScenarioTemplates emits affectedHs2: [] when the registry entry
  is null (all-sectors sentinel); proto repeated cannot carry null.
- RunScenario returns { jobId, status } (no statusUrl field — unused
  by SupplyChainPanel, drop from wire).

Gateway wiring:
- server/gateway.ts RPC_CACHE_TIER: list-scenario-templates → 'daily'
  (matches legacy max-age=3600); get-scenario-status → 'slow-browser'
  (premium short-circuit target, explicit entry required by
  tests/route-cache-tier.test.mjs).
- src/shared/premium-paths.ts: swap old run/status for the new
  run-scenario/get-scenario-status paths.
- api/scenario/v1/{run,status,templates}.ts deleted; 3 manifest
  exceptions removed (63 → 52 → 49 migration-pending).

Client:
- src/services/scenario/index.ts — typed client wrapper using
  premiumFetch (injects Clerk bearer / API key).
- src/components/SupplyChainPanel.ts — polling loop swapped from
  premiumFetch strings to runScenario/getScenarioStatus. Hard 20s
  timeout on run preserved via AbortSignal.any.

Tests:
- tests/scenario-handler.test.mjs — 18 new handler-level tests
  covering every security invariant + the worker envelope coercion.
- tests/edge-functions.test.mjs — scenario sections removed,
  replaced with a breadcrumb pointer to the new test file.

Docs: api-scenarios.mdx, scenario-engine.mdx, usage-rate-limits.mdx,
usage-errors.mdx, supply-chain.mdx refreshed with new paths.

Verified: typecheck, typecheck:api, lint:api-contract (49 entries),
lint:rate-limit-policies (6/180), lint:boundaries, route-cache-tier
(parity), full edge-functions (117) + scenario-handler (18).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor(commit 8): migrate /api/v2/shipping/{route-intelligence,webhooks} → ShippingV2Service (koala73#3207)

Partner-facing endpoints promoted to a typed sebuf service. Wire shape
preserved byte-for-byte (camelCase field names, ISO-8601 fetchedAt, the
same subscriberId/secret formats, the same SET + SADD + EXPIRE 30-day
Redis pipeline). Partner URLs /api/v2/shipping/* are unchanged.

RPCs landed:
- GET  /route-intelligence  → RouteIntelligence  (PRO, slow-browser)
- POST /webhooks            → RegisterWebhook    (PRO)
- GET  /webhooks            → ListWebhooks       (PRO, slow-browser)

The existing path-parameter URLs remain on the legacy edge-function
layout because sebuf's HTTP annotations don't currently model path
params (grep proto/**/*.proto for `path: "{…}"` returns zero). Those
endpoints are split into two Vercel dynamic-route files under
api/v2/shipping/webhooks/, behaviorally identical to the previous
hybrid file but cleanly separated:
- GET  /webhooks/{subscriberId}                → [subscriberId].ts
- POST /webhooks/{subscriberId}/rotate-secret  → [subscriberId]/[action].ts
- POST /webhooks/{subscriberId}/reactivate     → [subscriberId]/[action].ts

Both get manifest entries under `migration-pending` pointing at koala73#3207.

Other changes
- scripts/enforce-sebuf-api-contract.mjs: extended GATEWAY_RE to accept
  api/v{N}/{domain}/[rpc].ts (version-first) alongside the canonical
  api/{domain}/v{N}/[rpc].ts; first-use of the reversed ordering is
  shipping/v2 because that's the partner contract.
- vite.config.ts: dev-server sebuf interceptor regex extended to match
  both layouts; shipping/v2 import + allRoutes entry added.
- server/gateway.ts: RPC_CACHE_TIER entries for /api/v2/shipping/
  route-intelligence + /webhooks (slow-browser; premium-gated endpoints
  short-circuit to slow-browser but the entries are required by
  tests/route-cache-tier.test.mjs).
- src/shared/premium-paths.ts: route-intelligence + webhooks added.
- tests/shipping-v2-handler.test.mjs: 18 handler-level tests covering
  PRO gate, iso2/cargoType/hs2 coercion, SSRF guards (http://, RFC1918,
  cloud metadata, IMDS), chokepoint whitelist, alertThreshold range,
  secret/subscriberId format, pipeline shape + 30-day TTL, cross-tenant
  owner isolation, `secret` omission from list response.

Manifest delta
- Removed: api/v2/shipping/route-intelligence.ts, api/v2/shipping/webhooks.ts
- Added:   api/v2/shipping/webhooks/[subscriberId].ts (migration-pending)
- Added:   api/v2/shipping/webhooks/[subscriberId]/[action].ts (migration-pending)
- Added:   api/internal/brief-why-matters.ts (internal-helper) — regression
  surface from the koala73#3248 main merge, which introduced the file without a
  manifest entry. Filed here to keep the lint green; not strictly in scope
  for commit 8 but unblocking.

Net result: 49 → 47 `migration-pending` entries (one net-removal even
though webhook path-params stay pending, because two files collapsed
into two dynamic routes).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review HIGH 1): SupplyChainServiceClient must use premiumFetch (koala73#3207)

Signed-in browser pro users were silently hitting 401 on 8 supply-chain
premium endpoints (country-products, multi-sector-cost-shock,
country-chokepoint-index, bypass-options, country-cost-shock,
sector-dependency, route-explorer-lane, route-impact). The shared
client was constructed with globalThis.fetch, so no Clerk bearer or
X-WorldMonitor-Key was injected. The gateway's validateApiKey runs
with forceKey=true for PREMIUM_RPC_PATHS and 401s before isCallerPremium
is consulted. The generated client's try/catch collapses the 401 into
an empty-fallback return, leaving panels blank with no visible error.

Fix is one line at the client constructor: swap globalThis.fetch for
premiumFetch. The same pattern is already in use for insider-transactions,
stock-analysis, stock-backtest, scenario, trade (premiumClient) — this
was an omission on this client, not a new pattern.

premiumFetch no-ops safely when no credentials are available, so the
5 non-premium methods on this client (shippingRates, chokepointStatus,
chokepointHistory, criticalMinerals, shippingStress) continue to work
unchanged.

This also fixes two panels that were pre-existing latently broken on
main (chokepoint-index, bypass-options, etc. — predating koala73#3207, not
regressions from it). Commit 6 expanded the surface by routing two more
methods through the same buggy client; this commit fixes the class.

From koala73 review (koala73#3242 second-pass, HIGH new #1):
> Exact class PR koala73#3233 fixed for RegionalIntelligenceBoard /
> DeductionPanel / trade / country-intel. Supply-chain was not in
> koala73#3233's scope.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review HIGH 2): restore 400 on input-shape errors for 2 supply-chain handlers (koala73#3207)

Commit 6 collapsed all non-happy paths into empty-200 on
`get-country-products` and `get-multi-sector-cost-shock`, including
caller-bug cases that legacy returned 400 for:

- get-country-products: malformed iso2 → empty 200 (was 400)
- get-multi-sector-cost-shock: malformed iso2 / missing chokepointId /
  unknown chokepointId → empty 200 (was 400)

The commit message for 6 called out the 403-for-non-pro → empty-200
shift ("sebuf gateway pattern is empty-payload-on-deny") but not the
400 shift. They're different classes:

- Empty-payload-200 for PRO-deny: intentional contract change, already
  documented and applied across the service. Generated clients treat
  "you lack PRO" as "no data" — fine.
- Empty-payload-200 for malformed input: caller bug silently masked.
  External API consumers can't distinguish "bad wiring" from "genuinely
  no data", test harnesses lose the signal, bad calling code doesn't
  surface in Sentry.

Fix: `throw new ValidationError(violations)` on the 3 input-shape
branches. The generated sebuf server maps ValidationError → HTTP 400
(see src/generated/server/.../service_server.ts and leads/v1 which
already uses this pattern).

PRO-gate deny stays as empty-200 — that contract shift was intentional
and is preserved.

Regression tests added at tests/supply-chain-validation.test.mjs (8
cases) pinning the three-way contract:
- bad input                         → 400 (ValidationError)
- PRO-gate deny on valid input      → 200 empty
- valid PRO input, no data in Redis → 200 empty (unchanged)

From koala73 review (koala73#3242 second-pass, HIGH new #2).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review HIGH 3): restore statusUrl on RunScenarioResponse + document 202→200 wire break (koala73#3207)

Commit 7 silently shifted /api/scenario/v1/run-scenario's response
contract in two ways that the commit message covered only partially:

1. HTTP 202 Accepted → HTTP 200 OK
2. Dropped `statusUrl` string from the response body

The `statusUrl` drop was mentioned as "unused by SupplyChainPanel" but
not framed as a contract change. The 202 → 200 shift was not mentioned
at all. This is a same-version (v1 → v1) migration, so external callers
that key off either signal — `response.status === 202` or
`response.body.statusUrl` — silently branch incorrectly.

Evaluated options:
  (a) sebuf per-RPC status-code config — not available. sebuf's
      HttpConfig only models `path` and `method`; no status annotation.
  (b) Bump to scenario/v2 — judged heavier than the break itself for
      a single status-code shift. No in-repo caller uses 202 or
      statusUrl; the docs-level impact is containable.
  (c) Accept the break, document explicitly, partially restore.

Took option (c):

- Restored `statusUrl` in the proto (new field `string status_url = 3`
  on RunScenarioResponse). Server computes
  `/api/scenario/v1/get-scenario-status?jobId=<encoded job_id>` and
  populates it on every successful enqueue. External callers that
  followed this URL keep working unchanged.
- 202 → 200 is not recoverable inside the sebuf generator, so it is
  called out explicitly in two places:
    - docs/api-scenarios.mdx now includes a prominent `<Warning>` block
      documenting the v1→v1 contract shift + the suggested migration
      (branch on response body shape, not HTTP status).
    - RunScenarioResponse proto comment explains why 200 is the new
      success status on enqueue.
  OpenAPI bundle regenerated to reflect the restored statusUrl field.

- Regression test added in tests/scenario-handler.test.mjs pinning
  `statusUrl` to the exact URL-encoded shape — locks the invariant so
  a future proto rename or handler refactor can't silently drop it
  again.

From koala73 review (koala73#3242 second-pass, HIGH new #3).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review HIGH 1/2): close webhook tenant-isolation gap on shipping/v2 (koala73#3207)

Koala flagged this as a merge blocker in PR koala73#3242 review.

server/worldmonitor/shipping/v2/{register-webhook,list-webhooks}.ts
migrated without reinstating validateApiKey(req, { forceKey: true }),
diverging from both the sibling api/v2/shipping/webhooks/[subscriberId]
routes and the documented "X-WorldMonitor-Key required" contract in
docs/api-shipping-v2.mdx.

Attack surface: the gateway accepts Clerk bearer auth as a pro signal.
A Clerk-authenticated pro user with no X-WorldMonitor-Key reaches the
handler, callerFingerprint() falls back to 'anon', and every such
caller collapses into a shared webhook:owner:anon:v1 bucket. The
defense-in-depth ownerTag !== ownerHash check in list-webhooks.ts
doesn't catch it because both sides equal 'anon' — every Clerk-session
holder could enumerate / overwrite every other Clerk-session pro
tenant's registered webhook URLs.

Fix: reinstate validateApiKey(ctx.request, { forceKey: true }) at the
top of each handler, throwing ApiError(401) when absent. Matches the
sibling routes exactly and the published partner contract.

Tests:
- tests/shipping-v2-handler.test.mjs: two existing "non-PRO → 403"
  tests for register/list were using makeCtx() with no key, which now
  fails at the 401 layer first. Renamed to "no API key → 401
  (tenant-isolation gate)" with a comment explaining the failure mode
  being tested. 18/18 pass.

Verified: typecheck:api, lint:api-contract (no change), lint:boundaries,
lint:rate-limit-policies, test:data (6005/6005).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review HIGH 2/2): restore v1 path aliases on scenario + supply-chain (koala73#3207)

Koala flagged this as a merge blocker in PR koala73#3242 review.

Commits 6 + 7 of koala73#3207 renamed five documented v1 URLs to the sebuf
method-derived paths and deleted the legacy edge-function files:

  POST /api/scenario/v1/run                       → run-scenario
  GET  /api/scenario/v1/status                    → get-scenario-status
  GET  /api/scenario/v1/templates                 → list-scenario-templates
  GET  /api/supply-chain/v1/country-products      → get-country-products
  GET  /api/supply-chain/v1/multi-sector-cost-shock → get-multi-sector-cost-shock

server/router.ts is an exact static-match table (Map keyed on `METHOD
PATH`), so any external caller — docs, partner scripts, grep-the-
internet — hitting the old documented URL would 404 on first request
after merge. Commit 8 (shipping/v2) preserved partner URLs byte-for-
byte; the scenario + supply-chain renames missed that discipline.

Fix: add five thin alias edge functions that rewrite the pathname to
the canonical sebuf path and delegate to the domain [rpc].ts gateway
via a new server/alias-rewrite.ts helper. Premium gating, rate limits,
entitlement checks, and cache-tier lookups all fire on the canonical
path — aliases are pure URL rewrites, not a duplicate handler pipeline.

  api/scenario/v1/{run,status,templates}.ts
  api/supply-chain/v1/{country-products,multi-sector-cost-shock}.ts

Vite dev parity: file-based routing at api/ is a Vercel concern, so the
dev middleware (vite.config.ts) gets a matching V1_ALIASES rewrite map
before the router dispatch.

Manifest: 5 new entries under `deferred` with removal_issue=koala73#3282
(tracking their retirement at the next v1→v2 break). lint:api-contract
stays green (89 files checked, 55 manifest entries validated).

Docs:
- docs/api-scenarios.mdx: migration callout at the top with the full
  old→new URL table and a link to the retirement issue.
- CHANGELOG.md + docs/changelog.mdx: Changed entry documenting the
  rename + alias compat + the 202→200 shift (from commit 23c821a).

Verified: typecheck:api, lint:api-contract, lint:rate-limit-policies,
lint:boundaries, test:data (6005/6005).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>

…ala73#3370)

* feat(news/parser): extract RSS/Atom description for LLM grounding (U1)

Add description field to ParsedItem, extract from the first non-empty of
description/content:encoded (RSS) or summary/content (Atom), picking the
longest after HTML-strip + entity-decode + whitespace-normalize. Clip to
400 chars. Reject empty, <40 chars after strip, or normalize-equal to the
headline — downstream consumers fall back to the cleaned headline on '',
preserving current behavior for feeds without a description.

CDATA end is anchored to the closing tag so internal ]]> sequences do not
truncate the match. Preserves cached rss:feed:v1 row compatibility during
the 1h TTL bleed since the field is additive.

Part of fix: pipe RSS description end-to-end so LLM surfaces stop
hallucinating named actors (docs/plans/2026-04-24-001-...).

Covers R1, R7.

* feat(news/story-track): persist description on story:track:v1 HSET (U2)

Append description to the story:track:v1 HSET only when non-empty. Additive
— no key version bump. Old rows and rows from feeds without a description
return undefined on HGETALL, letting downstream readers fall back to the
cleaned headline (R6).

Extract buildStoryTrackHsetFields as a pure helper so the inclusion gate is
unit-testable without Redis.

Update the contract comment in cache-keys.ts so the next reader of the
schema sees description as an optional field.

Covers R2, R6.

* feat(proto): NewsItem.snippet + SummarizeArticleRequest.bodies (U3)

Add two additive proto fields so the article description can ride to every
LLM-adjacent consumer without a breaking change:

- NewsItem.snippet (field 12): RSS/Atom description, HTML-stripped,
  ≤400 chars, empty when unavailable. Wired on toProtoItem.
- SummarizeArticleRequest.bodies (field 8): optional article bodies
  paired 1:1 with headlines for prompt grounding. Empty array is today's
  headline-only behavior.

Regenerated TS client/server stubs and OpenAPI YAML/JSON via sebuf v0.11.1
(PATH=~/go/bin required — Homebrew's protoc-gen-openapiv3 is an older
pre-bundle-mode build that collides on duplicate emission).

Pre-emptive bodies:[] placeholders at the two existing SummarizeArticle
call sites in src/services/summarization.ts; U6 replaces them with real
article bodies once SummarizeArticle handler reads the field.

Covers R3, R5.

* feat(brief/digest): forward RSS description end-to-end through brief envelope (U4)

Digest accumulator reader (seed-digest-notifications.mjs::buildDigest) now
plumbs the optional `description` field off each story:track:v1 HGETALL into
the digest story object. The brief adapter (brief-compose.mjs::
digestStoryToUpstreamTopStory) prefers the real RSS description over the
cleaned headline; when the upstream row has no description (old rows in the
48h bleed, feeds that don't carry one), we fall back to the cleaned headline
so today behavior is preserved (R6).

This is the upstream half of the description cache path. U5 lands the LLM-
side grounding + cache-prefix bump so Gemini actually sees the article body
instead of hallucinating a named actor from the headline.

Covers R4 (upstream half), R6.

* feat(brief/llm): RSS grounding + sanitisation + 4 cache prefix bumps (U5)

The actual fix for the headline-only named-actor hallucination class:
Gemini 2.5 Flash now receives the real article body as grounding context,
so it paraphrases what the article says instead of filling role-label
headlines from parametric priors ("Iran's new supreme leader" → "Ali
Khamenei" was the 2026-04-24 reproduction; with grounding, it becomes
the actual article-named actor).

Changes:

- buildStoryDescriptionPrompt interpolates a `Context: <body>` line
  between the metadata block and the "One editorial sentence" instruction
  when description is non-empty AND not normalise-equal to the headline.
  Clips to 400 chars as a second belt-and-braces after the U1 parser cap.
  No Context line → identical prompt to pre-fix (R6 preserved).

- sanitizeStoryForPrompt extended to cover `description`. Closes the
  asymmetry where whyMatters was sanitised and description wasn't —
  untrusted RSS bodies now flow through the same injection-marker
  neutraliser before prompt interpolation. generateStoryDescription wraps
  the story in sanitizeStoryForPrompt before calling the builder,
  matching generateWhyMatters.

- Four cache prefixes bumped atomically to evict pre-grounding rows:
    scripts/lib/brief-llm.mjs:
      brief:llm:description:v1 → v2  (Railway, description path)
      brief:llm:whymatters:v2 → v3   (Railway, whyMatters fallback)
    api/internal/brief-why-matters.ts:
      brief:llm:whymatters:v6 → v7                (edge, primary)
      brief:llm:whymatters:shadow:v4 → shadow:v5  (edge, shadow)
  hashBriefStory already includes description in the 6-field material
  (v5 contract) so identity naturally drifts; the prefix bump is the
  belt-and-braces that guarantees a clean cold-start on first tick.

- Tests: 8 new + 2 prefix-match updates on tests/brief-llm.test.mjs.
  Covers Context-line injection, empty/dup-of-headline rejection,
  400-char clip, sanitisation of adversarial descriptions, v2 write,
  and legacy-v1 row dark (forced cold-start).

Covers R4 + new sanitisation requirement.

* feat(news/summarize): accept bodies + bump summary cache v5→v6 (U6)

SummarizeArticle now grounds on per-headline article bodies when callers
supply them, so the dashboard "News summary" path stops hallucinating
across unrelated headlines when the upstream RSS carried context.

Three coordinated changes:

1. SummarizeArticleRequest handler reads req.bodies, sanitises each entry
   through sanitizeForPrompt (same trust treatment as geoContext — bodies
   are untrusted RSS text), clips to 400 chars, and pads to the headlines
   length so pair-wise identity is stable.

2. buildArticlePrompts accepts optional bodies and interleaves a
   `    Context: <body>` line under each numbered headline that has a
   non-empty body. Skipped in translate mode (headline[0]-only) and when
   all bodies are empty — yielding a byte-identical prompt to pre-U6
   for every current caller (R6 preserved).

3. summary-cache-key bumps CACHE_VERSION v5→v6 so the pre-grounding rows
   (produced from headline-only prompts) cold-start cleanly. Extends
   canonicalizeSummaryInputs + buildSummaryCacheKey with a pair-wise
   bodies segment `:bd<hash>`; the prefix is `:bd` rather than `:b` to
   avoid colliding with `:brief:` when pattern-matching keys. Translate
   mode is headline[0]-only and intentionally does not shift on bodies.

Dedup reorder preserved: the handler re-pairs bodies to the deduplicated
top-5 via findIndex, so layout matches without breaking cache identity.

New tests: 7 on buildArticlePrompts (bodies interleave, partial fill,
translate-mode skip, clip, short-array tolerance), 8 on
buildSummaryCacheKey (pair-wise sort, cache-bust on body drift, translate
skip). Existing summary-cache-key assertions updated v5→v6.

Covers R3, R4.

* feat(consumers): surface RSS snippet across dashboard, email, relay, MCP + audit (U7)

Thread the RSS description from the ingestion path (U1-U5) into every
user-facing LLM-adjacent surface. Audit the notification producers so
RSS-origin and domain-origin events stay on distinct contracts.

Dashboard (proto snippet → client → panel):
- src/types/index.ts NewsItem.snippet?:string (client-side field).
- src/app/data-loader.ts proto→client mapper propagates p.snippet.
- src/components/NewsPanel.ts renders snippet as a truncated (~200 chars,
  word-boundary ellipsis) `.item-snippet` line under each headline.
- NewsPanel.currentBodies tracks per-headline bodies paired 1:1 with
  currentHeadlines; passed as options.bodies to generateSummary so the
  server-side SummarizeArticle LLM grounds on the article body.

Summary plumbing:
- src/services/summarization.ts threads bodies through SummarizeOptions
  → generateSummary → runApiChain → tryApiProvider; cache key now includes
  bodies (via U6's buildSummaryCacheKey signature).

MCP world-brief:
- api/mcp.ts pairs headlines with their RSS snippets and POSTs `bodies`
  to /api/news/v1/summarize-article so the MCP tool surface is no longer
  starved.

Email digest:
- scripts/seed-digest-notifications.mjs plain-text formatDigest appends
  a ~200-char truncated snippet line under each story; HTML formatDigestHtml
  renders a dim-grey description div between title and meta. Both gated
  on non-empty description (R6 — empty → today's behavior).

Real-time alerts:
- src/services/breaking-news-alerts.ts BreakingAlert gains optional
  description; checkBatchForBreakingAlerts reads item.snippet; dispatchAlert
  includes `description` in the /api/notify payload when present.

Notification relay:
- scripts/notification-relay.cjs formatMessage gated on
  NOTIFY_RELAY_INCLUDE_SNIPPET=1 (default off). When on, RSS-origin
  payloads render a `> <snippet>` context line under the title. When off
  or payload.description absent, output is byte-identical to pre-U7.

Audit (RSS vs domain):
- tests/notification-relay-payload-audit.test.mjs enforces file-level
  @notification-source tags on every producer, rejects `description:` in
  domain-origin payload blocks, and verifies the relay codepath gates
  snippet rendering under the flag.
- Tag added to ais-relay.cjs (domain), seed-aviation.mjs (domain),
  alert-emitter.mjs (domain), breaking-news-alerts.ts (rss).

Deferred (plan explicitly flags): InsightsPanel + cluster-producer
plumbing (bodies default to [] — will unlock gradually once news:insights:v1
producer also carries primarySnippet).

Covers R5, R6.

* docs+test: grounding-path note + bump pinned CACHE_VERSION v5→v6 (U8)

Final verification for the RSS-description-end-to-end fix:

- docs/architecture.mdx — one-paragraph "News Grounding Pipeline"
  subsection tracing parser → story:track:v1.description → NewsItem.snippet
  → brief / SummarizeArticle / dashboard / email / relay / MCP, with the
  empty-description R6 fallback rule called out explicitly.
- tests/summarize-reasoning.test.mjs — Fix-4 static-analysis pin updated
  to match the v6 bump from U6. Without this the summary cache bump silently
  regressed CI's pinned-version assertion.

Final sweep (2026-04-24):
- grep -rn 'brief:llm:description:v1' → only in the U5 legacy-row test
  simulation (by design: proves the v2 bump forces cold-start).
- grep -rn 'brief:llm:whymatters:v2/v6/shadow:v4' → no live references.
- grep -rn 'summary:v5' → no references.
- CACHE_VERSION = 'v6' in src/utils/summary-cache-key.ts.
- Full tsx --test sweep across all tests/*.test.{mjs,mts}: 6747/6747 pass.
- npm run typecheck + typecheck:api: both clean.

Covers R4, R6, R7.

* fix(rss-description): address /ce:review findings before merge

14 fixes from structured code review across 13 reviewer personas.

Correctness-critical (P1 — fixes that prevent R6/U7 contract violations):
- NewsPanel signature covers currentBodies so view-mode toggles that leave
  headlines identical but bodies different now invalidate in-flight summaries.
  Without this, switching renderItems → renderClusters mid-summary let a
  grounded response arrive under a stale (now-orphaned) cache key.
- summarize-article.ts re-pairs bodies with headlines BEFORE dedup via a
  single zip-sanitize-filter-dedup pass. Previously bodies[] was indexed by
  position in light-sanitized headlines while findIndex looked up the
  full-sanitized array — any headline that sanitizeHeadlines emptied
  mispaired every subsequent body, grounding the LLM on the wrong story.
- Client skips the pre-chain cache lookup when bodies are present, since
  client builds keys from RAW bodies while server sanitizes first. The
  keys diverge on injection content, which would silently miss the
  server's authoritative cache every call.

Test + audit hardening:
- Legacy v1 eviction test now uses the real hashBriefStory(story()) suffix
  instead of a literal "somehash", so a bug where the reader still queried
  the v1 prefix at the real key would actually be caught.
- tests/summary-cache-key.test.mts adds 400-char clip identity coverage so
  the canonicalizer's clip and any downstream clip can't silently drift.
- tests/news-rss-description-extract.test.mts renames the well-formed
  CDATA test and adds a new test documenting the malformed-]]> fallback
  behavior (plain regex captures, article content survives).

Safe_auto cleanups:
- Deleted dead SNIPPET_PUSH_MAX constant in notification-relay.cjs.
- BETA-mode groq warm call now passes bodies, warming the right cache slot.
- seed-digest shares a local normalize-equality helper for description !=
  headline comparison, matching the parser's contract.
- Pair-wise sort in summary-cache-key tie-breaks on body so duplicate
  headlines produce stable order across runs.
- buildSummaryCacheKey gained JSDoc documenting the client/server contract
  and the bodies parameter semantics.
- MCP get_world_brief tool description now mentions RSS article-body
  grounding so calling agents see the current contract.
- _shared.ts `opts.bodies![i]!` double-bang replaced with `?? ''`.
- extractRawTagBody regexes cached in module-level Map, mirroring the
  existing TAG_REGEX_CACHE pattern.

Deferred to follow-up (tracked for PR description / separate issue):
- Promote shared MAX_BODY constant across the 5 clip sites
- Promote shared truncateForDisplay helper across 4 render sites
- Collapse NewsPanel.{currentHeadlines, currentBodies} → Array<{title, snippet}>
- Promote sanitizeStoryForPrompt to shared/brief-llm-core.js
- Split list-feed-digest.ts parser helpers into sibling -utils.ts
- Strengthen audit test: forward-sweep + behavioral gate test

Tests: 6749/6749 pass. Typecheck clean on both configs.

* fix(summarization): thread bodies through browser T5 path (Codex #2)

Addresses the second of two Codex-raised findings on PR koala73#3370:

The PR threaded bodies through the server-side API provider chain
(Ollama → Groq → OpenRouter → /api/news/v1/summarize-article) but the
local browser T5 path at tryBrowserT5 was still summarising from
headlines alone. In BETA_MODE that ungrounded path runs BEFORE the
grounded server providers; in normal mode it remains the last
fallback. Whenever T5-small won, the dashboard summary surface
regressed to the headline-only path — the exact hallucination class
this PR exists to eliminate.

Fix: tryBrowserT5 accepts an optional `bodies` parameter and
interleaves each body with its paired headline via a `headline —
body` separator in the combined text (clipped to 200 chars per body
to stay within T5-small's ~512-token context window). All three call
sites (BETA warm, BETA cold, normal-mode fallback) now pass the
bodies threaded down from generateSummary options.bodies.

When bodies is empty/omitted, the combined text is byte-identical to
pre-fix (R6 preserved).

On Codex finding #1 (story:track:v1 additive-only HSET keeps a body
from an earlier mention of the same normalized title), declining to
change. The current rule — "if this mention has a body, overwrite;
otherwise leave the prior body alone" — is defensible: a body from
mention A is not falsified by mention B being body-less (a wire
reprint doesn't invalidate the original source's body). A feed that
publishes a corrected headline creates a new normalized-title hash,
so no stale body carries forward. The failure window is narrow (live
story evolving while keeping the same title through hours of
body-less wire reprints) and the 7-day STORY_TTL is the backstop.
Opening a follow-up issue to revisit semantics if real-world evidence
surfaces a stale-grounding case.

* fix(story-track): description always-written to overwrite stale bodies (Codex #1)

Revisiting Codex finding #1 on PR koala73#3370 after re-review. The previous
response declined the fix with reasoning; on reflection the argument
was over-defending the current behavior.

Problem: buildStoryTrackHsetFields previously wrote `description` only
when non-empty. Because story:track:v1 rows are collapsed by
normalized-title hash, an earlier mention's body would persist for up
to STORY_TTL (7 days) on subsequent body-less mentions of the same
story. Consumers reading `track.description` via HGETALL could not
distinguish "this mention's body" from "some mention's body from the
last week," silently grounding brief / whyMatters / SummarizeArticle
LLMs on text the current mention never supplied. That violates the
grounding contract advertised to every downstream surface in this PR.

Fix: HSET `description` unconditionally on every mention — empty
string when the current item has no body, real body when it does. An
empty value overwrites any prior mention's body so the row is always
authoritative for the current cycle. Consumers continue to treat
empty description as "fall back to cleaned headline" (R6 preserved).
The 7-day STORY_TTL and normalized-title hash semantics are unchanged.

Trade-off accepted: a valid body from Feed A (NYT) is wiped when Feed
B (AP body-less wire reprint) arrives for the same normalized title,
even though Feed A's body is factually correct. Rationale: the
alternative — keeping Feed A's body indefinitely — means the user
sees Feed A's body attributed (by proximity) to an AP mention at a
later timestamp, which is at minimum misleading and at worst carries
retracted/corrected details. Honest absence beats unlabeled presence.

Tests: new stale-body overwrite sequence test (T0 body → T1 empty →
T2 new body), existing "writes description when non-empty" preserved,
existing "omits when empty" inverted to "writes empty, overwriting."
cache-keys.ts contract comment updated to mark description as
always-written rather than optional.

…read (koala73#3385)

* feat(seed): BUNDLE_RUN_STARTED_AT_MS env + runSeed SIGTERM cleanup

Prereq for the re-export-share Comtrade seeder (plan 2026-04-24-003),
usable by any cohort seeder whose consumer needs bundle-level freshness.

Two coupled changes:

1. `_bundle-runner.mjs` injects `BUNDLE_RUN_STARTED_AT_MS` into every
   spawned child. All siblings in a single bundle run share one value
   (captured at `runBundle` start, not spawn time). Consumers use this
   to detect stale peer keys — if a peer's seed-meta predates the
   current bundle run, fall back to a hard default rather than read
   a cohort-peer's last-week output.

2. `_seed-utils.mjs::runSeed` registers a `process.once('SIGTERM')`
   handler that releases the acquired lock and extends existing-data
   TTL before exiting 143. `_bundle-runner.mjs` sends SIGTERM on
   section timeout, then SIGKILL after KILL_GRACE_MS (5s). Without
   this handler the `finally` path never runs on SIGKILL, leaving
   the 30-min acquireLock reservation in place until its own TTL
   expires — the next cron tick silently skips the resource.

Regression guard memory: `bundle-runner-sigkill-leaks-child-lock` (PR
koala73#3128 root cause).

Tests added:
- bundle-runner env injection (value within run bounds)
- sibling sections share the same timestamp (critical for the
  consumer freshness guard)
- runSeed SIGTERM path: exit 143 + cleanup log
- process.once contract: second SIGTERM does not re-enter handler

* fix(seed): address P1/P2 review findings on SIGTERM + bundle contracts

Addresses PR koala73#3384 review findings (todos 256, 257, 259, 260):

koala73#256 (P1) — SIGTERM handler narrowed to fetch phase only. Was installed
at runSeed entry and armed through every `process.exit` path; could
race `emptyDataIsFailure: true` strict-floor exits (IMF-External,
WB-bulk) and extend seed-meta TTL when the contract forbids it —
silently re-masking 30-day outages. Now the handler is attached
immediately before `withRetry(fetchFn)` and removed in a try/finally
that covers all fetch-phase exit branches.

koala73#257 (P1) — `BUNDLE_RUN_STARTED_AT_MS` now has a first-class helper.
Exported `getBundleRunStartedAtMs()` from `_seed-utils.mjs` with JSDoc
describing the bundle-freshness contract. Fleet-wide helper so the
next consumer seeder imports instead of rediscovering the idiom.

koala73#259 (P2) — SIGTERM cleanup runs `Promise.allSettled` on disjoint-key
ops (`releaseLock` + `extendExistingTtl`). Serialising compounded
Upstash latency during the exact failure mode (Redis degraded) this
handler exists to handle, risking breach of the 5s SIGKILL grace.

koala73#260 (P2) — `_bundle-runner.mjs` asserts topological order on
optional `dependsOn` section field. Throws on unknown-label refs and
on deps appearing at a later index. Fleet-wide contract replacing
the previous prose-comment ordering guarantee.

Tests added/updated:
- New: SIGTERM handler removed after fetchFn completes (narrowed-scope
  contract — post-fetch SIGTERM must NOT trigger TTL extension)
- New: dependsOn unknown-label + out-of-order + happy-path (3 tests)

Full test suite: 6,866 tests pass (+4 net).

* fix(seed): getBundleRunStartedAtMs returns null outside a bundle run

Review follow-up: the earlier `Math.floor(Date.now()/1000)*1000` fallback
regressed standalone (non-bundle) runs. A consumer seeder invoked
manually just after its peer wrote `fetchedAt = (now - 5s)` would see
`bundleStartMs = Date.now()`, reject the perfectly-fresh peer envelope
as "stale", and fall back to defaults — defeating the point of the
peer-read path outside the bundle.

Returning null when `BUNDLE_RUN_STARTED_AT_MS` is unset/invalid keeps
the freshness gate scoped to its real purpose (across-bundle-tick
staleness) and lets standalone runs skip the gate entirely. Consumers
check `bundleStartMs != null` before applying the comparison; see the
companion `seed-sovereign-wealth.mjs` change on the stacked PR.

* test(seed): SIGTERM cleanup test now verifies Redis DEL + EXPIRE calls

Greptile review P2 on PR koala73#3384: the existing test only asserted exit
code + log line, not that the Redis ops were actually issued. The
log claim was ahead of the test.

Fixture now logs every Upstash fetch call's shape (EVAL / pipeline-
EXPIRE / other) to stderr. Test asserts:

- >=1 EVAL op was issued during SIGTERM cleanup (releaseLock Lua
  script on the lock key)
- >=1 pipeline-EXPIRE op was issued (extendExistingTtl on canonical
  + seed-meta keys)
- The EVAL body carries the runSeed-generated runId (proves it's
  THIS run's release, not a phantom op)
- The EXPIRE pipeline touches both the canonicalKey AND the
  seed-meta key (proves the keys[] array was built correctly
  including the extraKeys merge path)

Full test suite: 6,866 tests pass, typecheck clean.

* feat(resilience): Comtrade-backed re-export-share seeder + SWF Redis read

Plan ref: docs/plans/2026-04-24-003-feat-reexport-share-comtrade-seeder-plan.md

Motivating case. Before this PR, the SWF `rawMonths` denominator for
the `sovereignFiscalBuffer` dimension used GROSS annual imports for
every country. For re-export hubs (goods transiting without domestic
settlement), this structurally under-reports resilience: UAE's 2023
$941B of imports include $334B of transit flow that never represents
domestic consumption. Net imports = gross × (1 − reexport_share).

The previous (PR 3A) design flattened a hand-curated YAML into Redis;
the YAML shipped empty and never populated, so the correction never
applied and the cohort audit showed no movement.

Gap #2 (this PR). Two coupled changes to make the correction actually
apply:

1. Comtrade-backed seeder (`scripts/seed-recovery-reexport-share.mjs`).
   Rewritten to fetch UN Comtrade `flowCode=RX` (re-exports) and
   `flowCode=M` (imports) per cohort member, compute share = RX/M at
   the latest co-populated year, clamp to [0.05, 0.95], publish the
   envelope. Header auth (`Ocp-Apim-Subscription-Key`) — subscription
   key never reaches URL/logs/Redis. `maxRecords=250000` cap with
   truncation detection. Sequential + retry-on-429 with backoff.

   Hub cohort resolved by Phase 0 empirical probe (plan §Phase 0):
   ['AE', 'PA']. Six candidates (SG/HK/NL/BE/MY/LT) return HTTP 200
   with zero RX rows — Comtrade doesn't expose RX for those reporters.

2. SWF seeder reads from Redis (`scripts/seed-sovereign-wealth.mjs`).
   Swaps `loadReexportShareByCountry()` (YAML) for
   `loadReexportShareFromRedis()` (Redis key written by #1). Guarded
   by bundle-run freshness: if the sibling Reexport-Share seeder's
   `seed-meta` predates `BUNDLE_RUN_STARTED_AT_MS` (set by the
   prereq PR's `_bundle-runner.mjs` env-injection), HARD fallback
   to gross imports rather than apply last-month's stale share.

Health registries. Both new keys registered in BOTH `api/health.js`
SEED_META (60-day alert threshold) and `api/seed-health.js`
SEED_DOMAINS (43200min interval). feedback_two_health_endpoints_must_match.

Bundle wiring. `seed-bundle-resilience-recovery` Reexport-Share
timeout bumped 60s → 300s (Comtrade + retry can take 2-3 min
worst-case). Ordering preserved: Reexport-Share before Sovereign-
Wealth so the SWF seeder reads a freshly-written key in the same
cron tick.

Deletions. YAML + loader + 7 obsolete loader tests removed; single
source of truth is now Comtrade → Redis.

Prereq. Stacks on PR koala73#3384 (feat/bundle-runner-env-sigterm)
which adds BUNDLE_RUN_STARTED_AT_MS env injection + runSeed
SIGTERM cleanup. This PR's bundle-freshness guard depends on
that env variable.

Tests (19 new, 7 deleted, +12 net):
- Pure math: parseComtradeFlowResponse, computeShareFromFlows,
  clampShare, declareRecords + credential-leak source scan (15)
- Integration (Gap #2 regression guards): SWF seeder loadReexport
  ShareFromRedis — fresh/absent/malformed/stale-meta/missing-meta (5)
- Health registry dual-registry drift guard — scoped to this PR's
  keys, respecting pre-existing asymmetry (4)
- Bundle-ordering + timeout assertions (2)

Phase 0 cohort validation committed to plan. Full test suite
passes: 6,881 tests.

* fix(resilience): address P1/P2 review findings — adopt shared helpers, pin freshness boundary

Addresses PR koala73#3385 review findings:

koala73#257 (P1) consumer — `seed-sovereign-wealth.mjs` imports the shared
`getBundleRunStartedAtMs` helper from `_seed-utils.mjs` (added in the
prereq commit) instead of its own `getBundleStartMs`. Single source of
truth for the bundle-freshness contract.

koala73#258 (P2) — `seed-recovery-reexport-share.mjs` isMain guard uses the
canonical `pathToFileURL(process.argv[1]).href === import.meta.url`
form instead of basename-suffix matching. Handles symlinks, case-
different paths on macOS HFS+, and Windows path separators without
string munging.

koala73#260 (P2) consumer — Sovereign-Wealth declares `dependsOn:
['Reexport-Share']` in the bundle spec. `_bundle-runner.mjs` (prereq
commit) now enforces topological order on load and throws on
violation — replaces the previous prose-comment ordering contract.

koala73#261 (P2) — added a test to `tests/seed-sovereign-wealth-reads-redis-
reexport-share.test.mts` pinning the inclusive-boundary semantic:
`fetchedAtMs === bundleStartMs` must be treated as FRESH. Guards
against a future refactor to `<=` that would silently reject peers
writing at the very first millisecond of the bundle run.

Rebased onto updated prereq. Full test suite: 6,886 tests pass (+5 net).

* fix(resilience): freshness gate skipped in standalone mode; meta still required

Review catch: the previous `bundleStartMs = Date.now()` fallback made
standalone/manual `seed-sovereign-wealth.mjs` runs ALWAYS reject any
previously-seeded re-export-share meta as "stale" — even when the
operator ran the Reexport seeder milliseconds beforehand. Defeated
the point of the peer-read path outside the bundle.

With `getBundleRunStartedAtMs()` now returning null outside a bundle
(companion commit on the prereq branch), the consumer only applies
the freshness gate when `bundleStartMs != null`. Standalone runs
accept any `fetchedAt` — the operator is responsible for ordering.

Two guards survive the change:
- Meta MUST exist (absence = peer-outage fail-safe, both modes)
- In-bundle: meta MUST be at or after `BUNDLE_RUN_STARTED_AT_MS`

Two new tests pin both modes:
- standalone: accepts meta written 10 min before this process started
- standalone: still rejects missing meta (peer-outage fail-safe
  survives gate bypass)

Rebased onto updated prereq. Full test suite: 6,888 tests (+2 net).

* fix(resilience): filter world-aggregate Comtrade rows + skip final-retry sleep

Greptile review of PR koala73#3385 flagged two P2s in the Comtrade seeder.

Finding #3 (parseComtradeFlowResponse double-count risk):
`cmdCode=TOTAL` without a partner filter currently returns only
world-aggregate rows in practice — but `parseComtradeFlowResponse`
summed every row unconditionally. A future refactor adding per-
partner querying would silently double-count (world-aggregate row +
partner-level rows for the same year), cutting the derived share in
half with no test signal.

Fix: explicit `partnerCode ∈ {'0', 0, null/undefined}` filter. Matches
current empirical behavior (aggregate-only responses) and makes the
construct robust to a future partner-level query.

Finding #4 (wasted backoff on final retry):
429 and 5xx branches slept `backoffMs` before `continue`, but on
`attempt === RETRY_MAX_ATTEMPTS` the loop condition fails immediately
after — the sleep was pure waste. Added early-return (parallel to the
existing pattern in the network-error catch branch) so the final
attempt exits the retry loop at the first non-success response
without extra latency.

Tests:
- 3 new `parseComtradeFlowResponse` variants: world-only filter,
  numeric-0 partnerCode shape, rows without partnerCode field
- Existing tests updated: the double-count assertion replaced with
  a "per-partner rows must NOT sum into the world-aggregate total"
  assertion that pins the new contract

Rebased onto updated prereq. Full test suite: 6,890 tests (+2 net).

…lead divergence (koala73#3396)

* feat(brief-llm): canonical synthesis prompt + v3 cache key

Extends generateDigestProse to be the single source of truth for
brief executive-summary synthesis (canonicalises what was previously
split between brief-llm's generateDigestProse and seed-digest-
notifications.mjs's generateAISummary). Ports Brain B's prompt
features into buildDigestPrompt:

- ctx={profile, greeting, isPublic} parameter (back-compat: 4-arg
  callers behave like today)
- per-story severity uppercased + short-hash prefix [h:XXXX] so the
  model can emit rankedStoryHashes for stable re-ranking
- profile lines + greeting opener appear only when ctx.isPublic !== true

validateDigestProseShape gains optional rankedStoryHashes (≥4-char
strings, capped to MAX_STORIES_PER_USER × 2). v2-shaped rows still
pass — field defaults to [].

hashDigestInput v3:
- material includes profile-SHA, greeting bucket, isPublic flag,
  per-story hash
- isPublic=true substitutes literal 'public' for userId in the cache
  key so all share-URL readers of the same (date, sensitivity, pool)
  hit ONE cache row (no PII in public cache key)

Adds generateDigestProsePublic(stories, sensitivity, deps) wrapper —
no userId param by design — for the share-URL surface.

Cache prefix bumped brief:llm:digest:v2 → v3. v2 rows expire on TTL.
Per the v1→v2 precedent (see hashDigestInput comment), one-tick cost
on rollout is acceptable for cache-key correctness.

Tests: 72/72 passing in tests/brief-llm.test.mjs (8 new for the v3
behaviors), full data suite 6952/6952.

Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md
Step 1, Codex-approved (5 rounds).

* feat(brief): envelope v3 — adds digest.publicLead for share-URL surface

Bumps BRIEF_ENVELOPE_VERSION 2 → 3. Adds optional
BriefDigest.publicLead — non-personalised executive lead generated
by generateDigestProsePublic (already in this branch from the
previous commit) for the public share-URL surface. Personalised
`lead` is the canonical synthesis for authenticated channels;
publicLead is its profile-stripped sibling so api/brief/public/*
never serves user-specific content (watched assets/regions).

SUPPORTED_ENVELOPE_VERSIONS = [1, 2, 3] keeps v1 + v2 envelopes
in the 7-day TTL window readable through the rollout — the
composer only ever writes the current version, but readers must
tolerate older shapes that haven't expired yet. Same rollout
pattern used at the v1 → v2 bump.

Renderer changes (server/_shared/brief-render.js):
- ALLOWED_DIGEST_KEYS gains 'publicLead' (closed-key-set still
  enforced; v2 envelopes pass because publicLead === undefined is
  the v2 shape).
- assertBriefEnvelope: new isNonEmptyString check on publicLead
  when present. Type contract enforced; absence is OK.

Tests (tests/brief-magazine-render.test.mjs):
- New describe block "v3 publicLead field": v3 envelope renders;
  malformed publicLead rejected; v2 envelope still passes; ad-hoc
  digest keys (e.g. synthesisLevel) still rejected — confirming
  the closed-key-set defense holds for the cron-local-only fields
  the orchestrator must NOT persist.
- BRIEF_ENVELOPE_VERSION pin updated 2 → 3 with rollout-rationale
  comment.

Test results: 182 brief-related tests pass; full data suite
6956/6956.

Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md
Step 2, Codex Round-3 Medium #2.

* feat(brief): synthesis splice + rankedStoryHashes pre-cap re-order

Plumbs the canonical synthesis output (lead, threads, signals,
publicLead, rankedStoryHashes from generateDigestProse) through the
pure composer so the orchestration layer can hand pre-resolved data
into envelope.digest. Composer stays sync / no I/O — Codex Round-2
High #2 honored.

Changes:

scripts/lib/brief-compose.mjs:
- digestStoryToUpstreamTopStory now emits `hash` (the digest story's
  stable identifier, falls back to titleHash when absent). Without
  this, rankedStoryHashes from the LLM has nothing to match against.
- composeBriefFromDigestStories accepts opts.synthesis = {lead,
  threads, signals, rankedStoryHashes?, publicLead?}. When passed,
  splices into envelope.digest after the stub is built. Partial
  synthesis (e.g. only `lead` populated) keeps stub defaults for the
  other fields — graceful degradation when L2 fallback fires.

shared/brief-filter.js:
- filterTopStories accepts optional rankedStoryHashes. New helper
  applyRankedOrder re-orders stories by short-hash prefix match
  BEFORE the cap is applied, so the model's editorial judgment of
  importance survives MAX_STORIES_PER_USER. Stable for ties; stories
  not in the ranking come after in original order. Empty/missing
  ranking is a no-op (legacy callers unchanged).

shared/brief-filter.d.ts:
- filterTopStories signature gains rankedStoryHashes?: string[].
- UpstreamTopStory gains hash?: unknown (carried through from
  digestStoryToUpstreamTopStory).

Tests added (tests/brief-from-digest-stories.test.mjs):
- synthesis substitutes lead/threads/signals/publicLead.
- legacy 4-arg callers (no synthesis) keep stub lead.
- partial synthesis (only lead) keeps stub threads/signals.
- rankedStoryHashes re-orders pool before cap.
- short-hash prefix match (model emits 8 chars; story carries full).
- unranked stories go after in original order.

Test results: 33/33 in brief-from-digest-stories; 182/182 across all
brief tests; full data suite 6956/6956.

Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md
Step 3, Codex Round-2 Low + Round-2 High #2.

* feat(brief): single canonical synthesis per user; rewire all channels

Restructures the digest cron's per-user compose + send loops to
produce ONE canonical synthesis per user per issueSlot — the lead
text every channel (email HTML, plain-text, Telegram, Slack,
Discord, webhook) and the magazine show is byte-identical. This
eliminates the "two-brain" divergence that was producing different
exec summaries on different surfaces (observed 2026-04-25 0802).

Architecture:

composeBriefsForRun (orchestration):
- Pre-annotates every eligible rule with lastSentAt + isDue once,
  before the per-user pass. Same getLastSentAt helper the send loop
  uses so compose + send agree on lastSentAt for every rule.

composeAndStoreBriefForUser (per-user):
- Two-pass winner walk: try DUE rules first (sortedDue), fall back
  to ALL eligible rules (sortedAll) for compose-only ticks.
  Preserves today's dashboard refresh contract for weekly /
  twice_daily users on non-due ticks (Codex Round-4 High #1).
- Within each pass, walk by compareRules priority and pick the
  FIRST candidate with a non-empty pool — mirrors today's behavior
  at scripts/seed-digest-notifications.mjs:1044 and prevents the
  "highest-priority but empty pool" edge case (Codex Round-4
  Medium #2).
- Three-level synthesis fallback chain:
    L1: generateDigestProse(fullPool, ctx={profile,greeting,!public})
    L2: generateDigestProse(envelope-sized slice, ctx={})
    L3: stub from assembleStubbedBriefEnvelope
  Distinct log lines per fallback level so ops can quantify
  failure-mode distribution.
- Generates publicLead in parallel via generateDigestProsePublic
  (no userId param; cache-shared across all share-URL readers).
- Splices synthesis into envelope via composer's optional
  `synthesis` arg (Step 3); rankedStoryHashes re-orders the pool
  BEFORE the cap so editorial importance survives MAX_STORIES.
- synthesisLevel stored in the cron-local briefByUser entry — NOT
  persisted in the envelope (renderer's assertNoExtraKeys would
  reject; Codex Round-2 Medium #5).

Send loop:
- Reads lastSentAt via shared getLastSentAt helper (single source
  of truth with compose flow).
- briefLead = brief?.envelope?.data?.digest?.lead — the canonical
  lead. Passed to buildChannelBodies (text/Telegram/Slack/Discord),
  injectEmailSummary (HTML email), and sendWebhook (webhook
  payload's `summary` field). All-channel parity (Codex Round-1
  Medium #6).
- Subject ternary reads cron-local synthesisLevel: 1 or 2 →
  "Intelligence Brief", 3 → "Digest" (preserves today's UX for
  fallback paths; Codex Round-1 Missing #5).

Removed:
- generateAISummary() — the second LLM call that produced the
  divergent email lead. ~85 lines.
- AI_SUMMARY_CACHE_TTL constant — no longer referenced. The
  digest:ai-summary:v1:* cache rows expire on their existing 1h
  TTL (no cleanup pass).

Helpers added:
- getLastSentAt(rule) — extracted Upstash GET for digest:last-sent
  so compose + send both call one source of truth.
- buildSynthesisCtx(rule, nowMs) — formats profile + greeting for
  the canonical synthesis call. Preserves all today's prefs-fetch
  failure-mode behavior.

Composer:
- compareRules now exported from scripts/lib/brief-compose.mjs so
  the cron can sort each pass identically to groupEligibleRulesByUser.

Test results: full data suite 6962/6962 (was 6956 pre-Step 4; +6
new compose-synthesis tests from Step 3).

Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md
Steps 4 + 4b. Codex-approved (5 rounds).

* fix(brief-render): public-share lead fail-safe — never leak personalised lead

Public-share render path (api/brief/public/[hash].ts → renderer
publicMode=true) MUST NEVER serve the personalised digest.lead
because that string can carry profile context — watched assets,
saved-region names, etc. — written by generateDigestProse with
ctx.profile populated.

Previously: redactForPublic redacted user.name and stories.whyMatters
but passed digest.lead through unchanged. Codex Round-2 High
(security finding).

Now (v3 envelope contract):
- redactForPublic substitutes digest.lead = digest.publicLead when
  the v3 envelope carries one (generated by generateDigestProsePublic
  with profile=null, cache-shared across all public readers).
- When publicLead is absent (v2 envelope still in TTL window OR v3
  envelope where publicLead generation failed), redactForPublic sets
  digest.lead to empty string.
- renderDigestGreeting: when lead is empty, OMIT the <blockquote>
  pull-quote entirely. Page still renders complete (greeting +
  horizontal rule), just without the italic lead block.
- NEVER falls back to the original personalised lead.

assertBriefEnvelope still validates publicLead's contract (when
present, must be a non-empty string) BEFORE redactForPublic runs,
so a malformed publicLead throws before any leak risk.

Tests added (tests/brief-magazine-render.test.mjs):
- v3 envelope renders publicLead in pull-quote, personalised lead
  text never appears.
- v2 envelope (no publicLead) omits pull-quote; rest of page
  intact.
- empty-string publicLead rejected by validator (defensive).
- private render still uses personalised lead.

Test results: 68 brief-magazine-render tests pass; full data suite
remains green from prior commit.

Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md
Step 5, Codex Round-2 High (security).

* feat(digest): brief lead parity log + extra acceptance tests

Adds the parity-contract observability line and supplementary
acceptance tests for the canonical synthesis path.

Parity log (per send, after successful delivery):
  [digest] brief lead parity user=<id> rule=<v>:<s>:<lang>
    synthesis_level=<1|2|3> exec_len=<n> brief_lead_len=<n>
    channels_equal=<bool> public_lead_len=<n>

When channels_equal=false an extra WARN line fires —
"PARITY REGRESSION user=… — email lead != envelope lead." Sentry's
existing console-breadcrumb hook lifts this without an explicit
captureMessage call. Plan acceptance criterion A5.

Tests added (tests/brief-llm.test.mjs, +9):
- generateDigestProsePublic: two distinct callers with identical
  (sensitivity, story-pool) hit the SAME cache row (per Codex
  Round-2 Medium #4 — "no PII in public cache key").
- public + private writes never collide on cache key (defensive).
- greeting bucket change re-keys the personalised cache (Brain B
  parity).
- profile change re-keys the personalised cache.
- v3 cache prefix used (no v2 writes).

Test results: 77/77 in brief-llm; full data suite 6971/6971
(was 6962 pre-Step-7; +9 new public-cache tests).

Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md
Steps 6 (partial) + 7. Acceptance A5, A6.g, A6.f.

* test(digest): backfill A6.h/i/l/m acceptance tests via helper extraction

* fix(brief): close two correctness regressions on multi-rule + public surface

Two findings from human review of the canonical-synthesis PR:

1. Public-share redaction leaked personalised signals + threads.
   The new prompt explicitly personalises both `lead` and `signals`
   ("personalise lead and signals"), but redactForPublic only
   substituted `lead` — leaving `signals` and `threads` intact.
   Public renderer's hasSignals gate would emit the signals page
   whenever `digest.signals.length > 0`, exposing watched-asset /
   region phrasing to anonymous readers. Same privacy bug class
   the original PR was meant to close, just on different fields.

2. Multi-rule users got cross-pool lead/storyList mismatch.
   composeAndStoreBriefForUser picks ONE winning rule for the
   canonical envelope. The send loop then injected that ONE
   `briefLead` into every due rule's channel body — even though
   each rule's storyList came from its own (per-rule) digest pool.
   Multi-rule users (e.g. `full` + `finance`) ended up with email
   bodies leading on geopolitics while listing finance stories.
   Cross-rule editorial mismatch reintroduced after the cross-
   surface fix.

Fix 1 — public signals + threads:
- Envelope shape: BriefDigest gains `publicSignals?: string[]` +
  `publicThreads?: BriefThread[]` (sibling fields to publicLead).
  Renderer's ALLOWED_DIGEST_KEYS extended; assertBriefEnvelope
  validates them when present.
- generateDigestProsePublic already returned a full prose object
  (lead + signals + threads) — orchestration now captures all
  three instead of just `.lead`. Composer splices each into its
  envelope slot.
- redactForPublic substitutes:
    digest.lead    ← publicLead (or empty → omits pull-quote)
    digest.signals ← publicSignals (or empty → omits signals page)
    digest.threads ← publicThreads (or category-derived stub via
                     new derivePublicThreadsStub helper — never
                     falls back to the personalised threads)
- New tests cover all three substitutions + their fail-safes.

Fix 2 — per-rule synthesis in send loop:
- Each due rule independently calls runSynthesisWithFallback over
  ITS OWN pool + ctx. Channel body lead is internally consistent
  with the storyList (both from the same pool).
- Cache absorbs the cost: when this is the winner rule, the
  synthesis hits the cache row written during the compose pass
  (same userId/sensitivity/pool/ctx) — no extra LLM call. Only
  multi-rule users with non-overlapping pools incur additional
  LLM calls.
- magazineUrl still points at the winner's envelope (single brief
  per user per slot — `(userId, issueSlot)` URL contract). Channel
  lead vs magazine lead may differ for non-winner rule sends;
  documented as acceptable trade-off (URL/key shape change to
  support per-rule magazines is out of scope for this PR).
- Parity log refined: adds `winner_match=<bool>` field. The
  PARITY REGRESSION warning now fires only when winner_match=true
  AND the channel lead differs from the envelope lead (the actual
  contract regression). Non-winner sends with legitimately
  different leads no longer spam the alert.

Test results:
- tests/brief-magazine-render.test.mjs: 75/75 (+7 new for public
  signals/threads + validator + private-mode-ignores-public-fields)
- Full data suite: 6995/6995 (was 6988; +7 net)
- typecheck + typecheck:api: clean

Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md
Addresses 2 review findings on PR koala73#3396 not anticipated in the
5-round Codex review.

* fix(brief): unify compose+send window, fall through filter-rejection

Address two residual risks in PR koala73#3396 (single-canonical-brain refactor):

Risk 1 — canonical lead synthesized from a fixed 24h pool while the
send loop ships stories from `lastSentAt ?? 24h`. For weekly users
that meant a 24h-pool lead bolted onto a 7d email body — the same
cross-surface divergence the refactor was meant to eliminate, just in
a different shape. Twice-daily users hit a 12h-vs-24h variant.

Fix: extract the window formula to `digestWindowStartMs(lastSentAt,
nowMs, defaultLookbackMs)` in digest-orchestration-helpers.mjs and
call it from BOTH the compose path's digestFor closure AND the send
loop. The compose path now derives windowStart per-candidate from
`cand.lastSentAt`, identical to what the send loop will use for that
rule. Removed the now-unused BRIEF_STORY_WINDOW_MS constant.

Side-effect: digestFor now receives the full annotated candidate
(`cand`) instead of just the rule, so it can reach `cand.lastSentAt`.
Backwards-compatible at the helper level — pickWinningCandidateWithPool
forwards `cand` instead of `cand.rule`.

Cache memo hit rate drops since lastSentAt varies per-rule, but
correctness > a few extra Upstash GETs.

Risk 2 — pickWinningCandidateWithPool returned the first candidate
with a non-empty raw pool as winner. If composeBriefFromDigestStories
then dropped every story (URL/headline/shape filters), the caller
bailed without trying lower-priority candidates. Pre-PR behaviour was
to keep walking. This regressed multi-rule users whose top-priority
rule's pool happens to be entirely filter-rejected.

Fix: optional `tryCompose(cand, stories)` callback on
pickWinningCandidateWithPool. When provided, the helper calls it after
the non-empty pool check; falsy return → log filter-rejected and walk
to the next candidate; truthy → returns `{winner, stories,
composeResult}` so the caller can reuse the result. Without the
callback, legacy semantics preserved (existing tests + callers
unaffected).

Caller composeAndStoreBriefForUser passes a no-synthesis compose call
as tryCompose — cheap pure-JS, no I/O. Synthesis only runs once after
the winner is locked in, so the perf cost is one extra compose per
filter-rejected candidate, no extra LLM round-trips.

Tests:
- 10 new cases in tests/digest-orchestration-helpers.test.mjs
  covering: digestFor receiving full candidate; tryCompose
  fall-through to lower-priority; all-rejected returns null;
  composeResult forwarded; legacy semantics without tryCompose;
  digestWindowStartMs lastSentAt-vs-default branches; weekly +
  twice-daily window parity assertions; epoch-zero ?? guard.
- Updated tests/digest-cache-key-sensitivity.test.mjs static-shape
  regex to match the new `cand.rule.sensitivity` cache-key shape
  (intent unchanged: cache key MUST include sensitivity).

Stacked on PR koala73#3396 — targets feat/brief-two-brain-divergence.

…plan U1-U4) (koala73#3397)

* feat(energy-atlas): GEM pipeline import infrastructure (PR 1, plan U1-U4)

Lands the parser, dedup helper, validator extensions, and operator runbook
for the Global Energy Monitor (CC-BY 4.0) pipeline-data refresh — closing
~3.6× of the Energy Atlas pipeline-scale gap once the operator runs the
import.

Per docs/plans/2026-04-25-003-feat-energy-parity-pushup-plan.md PR 1.

U1 — Validator + schema extensions:
- Add `'gem'` to VALID_SOURCES in scripts/_pipeline-registry.mjs and to the
  evidence-bearing-source whitelist in derivePipelinePublicBadge so GEM-
  sourced offline rows derive a `disputed` badge via the external-signal
  rule (parity with `press`/`satellite`/`ais-relay`).
- Export VALID_SOURCES so tests assert against the same source-of-truth
  the validator uses (matches the VALID_OIL_PRODUCT_CLASSES pattern from
  PR koala73#3383).
- Floor bump (MIN_PIPELINES_PER_REGISTRY 8→200) intentionally DEFERRED
  to the follow-up data PR — bumping it now would gate the existing 75+75
  hand-curated rows below the new floor and break seeder publishes
  before the GEM data lands.

U2 — GEM parser (test-first):
- scripts/import-gem-pipelines.mjs reads a local JSON file (operator pre-
  converts GEM Excel externally — no `xlsx` dependency added). Schema-
  drift sentinel throws on missing columns. Status mapping covers
  Operating/Construction/Cancelled/Mothballed/Idle/Shut-in. ProductClass
  mapping covers Crude Oil / Refined Products / mixed-flow notes.
  Capacity-unit conversion handles bcm/y, bbl/d, Mbd, kbd.
- 22 tests in tests/import-gem-pipelines.test.mjs cover schema sentinel,
  fuel split, status mapping, productClass mapping, capacity conversion,
  minimum-viable-evidence shape, registry-shape conformance, and bad-
  coordinate rejection.

U3 — Deduplication (pure deterministic):
- scripts/_pipeline-dedup.mjs: dedupePipelines(existing, candidates) →
  { toAdd, skippedDuplicates }. Match rule: haversine ≤5km AND name
  Jaccard ≥0.6 (BOTH required). Reverse-direction-pair-aware.
- 19 tests cover internal helpers, match logic, id collision, determinism,
  and empty inputs.

U4 — Operator runbook (data import deferred):
- docs/methodology/pipelines.mdx: 7-step runbook for the operator to
  download GEM, pre-convert Excel→JSON, dry-run with --print-candidates,
  merge with --merge, bump the registry floor, and commit with
  provenance metadata.
- The actual data import is intentionally OUT OF SCOPE for this agent-
  authored PR because GEM downloads are registration-gated. A follow-up
  PR will commit the imported scripts/data/pipelines-{gas,oil}.json +
  bump MIN_PIPELINES_PER_REGISTRY → 200 + record the GEM release SHA256.

Tests: typecheck clean; 67 tests pass across the three test files.

Codex-approved through 8 review rounds against origin/main @ 0500733.

* fix(energy-atlas): wire --merge to dedupePipelines + within-batch dedup (PR1 review)

P1 — --merge was a TODO no-op (import-gem-pipelines.mjs:291):
- Previously exited with code 2 + a "TODO: wire dedup once U3 lands"
  message. The PR body and the methodology runbook both advertised
  --merge as the operator path.
- Add mergeIntoRegistry(filename, candidates) helper that loads the
  existing envelope, runs dedupePipelines() against the candidate
  list, sorts new entries alphabetically by id (stable diff on rerun),
  validates the merged registry via validateRegistry(), and writes
  to disk only after validation passes. CLI --merge now invokes it
  for both gas and oil + prints a per-fuel summary.
- Source attribution: the registry envelope's `source` field is
  upgraded to mention GEM (CC-BY 4.0) on first merge so the data file
  itself documents provenance.

P2 — dedup transitive-match bug (_pipeline-dedup.mjs:120):
- Pre-fix loop checked each candidate ONLY against the original
  `existing` array. Two GEM rows that match each other but not anything
  in `existing` would BOTH be added, defeating the dedup contract for
  same-batch duplicates (real example: a primary GEM entry plus a
  duplicate row from a regional supplemental sheet).
- Now compares against existing FIRST (existing wins on cross-set
  match — preserves richer hand-curated evidence), then falls back to
  the already-accepted toAdd set. Within-batch matches retain the FIRST
  accepted candidate (deterministic by candidate-list order).

Tests: 22 in tests/pipeline-dedup.test.mjs (3 new) cover the
within-batch dedup, transitive collapse, and existing-wins-over-
already-accepted scenarios. typecheck clean.

* fix(energy-atlas): cross-file-atomic --merge (PR1 review #2)

P1 — partial-import on disk if oil validation fails after gas writes
(import-gem-pipelines.mjs:329 / :350):
- Previous flow ran `mergeIntoRegistry('pipelines-gas.json', gas)` which
  wrote to disk, then `mergeIntoRegistry('pipelines-oil.json', oil)`. If
  oil validation failed, the operator was left with a half-imported
  state: gas had GEM rows committed to disk but oil didn't.
- Refactor into a two-phase API:
  1. prepareMerge(filename, candidates) — pure, no disk I/O. Builds the
     merged envelope, validates it, throws on validation failure.
  2. mergeBothRegistries(gasCandidates, oilCandidates) — calls
     prepareMerge for BOTH fuels first; only writes to disk after BOTH
     pass validation. If oil's prepareMerge throws, gas was never
     touched on disk.
- CLI --merge now invokes mergeBothRegistries. The atomicity guarantee
  is documented inline in the helper.

typecheck clean. No new tests because the existing dedup + validate
suites cover the underlying logic; the change is purely about call
ordering for atomicity.

* fix(energy-atlas): deterministic lastEvidenceUpdate + clarify test comment (PR1 review #3)

P2 — lastEvidenceUpdate was non-deterministic (Greptile P2):
- Previous code used new Date().toISOString() per parser run, so two runs
  of parseGemPipelines on the same input on different days produced
  byte-different output. Quarterly re-imports would produce noisy
  full-row diffs even when the upstream GEM data hadn't changed.
- New: resolveEvidenceTimestamp(envelope) derives the timestamp from
  envelope.downloadedAt (the operator-recorded date) or sourceVersion
  if it parses as ISO. Falls back to 1970-01-01 sentinel when neither
  is set — deliberately ugly so reviewers spot the missing field in
  the data file diff rather than getting silent today's date.
- Computed once per parse run so every emitted candidate gets the
  same timestamp.

P2 — misleading test comment (Greptile P2):
- Comment in tests/import-gem-pipelines.test.mjs:136 said "400_000 bbl/d
  ÷ 1000 = 400 Mbd" while the assertion correctly expects 0.4 (because
  the convention is millions, not thousands). Rewrote the comment to
  state the actual rule + arithmetic clearly.

3 new tests for determinism: (a) two parser runs produce identical
output, (b) timestamp derives from downloadedAt, (c) missing date
yields the epoch sentinel (loud failure mode).

…an U7-U8) (koala73#3402)

* feat(energy-atlas): live tanker map layer + contract (PR 3, plan U7-U8)

Lands the third and final parity-push surface — per-vessel tanker positions
inside chokepoint bounding boxes, refreshed every 60s. Closes the visual
gap with peer reference energy-intel sites for the live AIS tanker view.

Per docs/plans/2026-04-25-003-feat-energy-parity-pushup-plan.md PR 3.
Codex-approved through 8 review rounds against origin/main @ 0500733.

U7 — Contract changes (relay + handler + proto + gateway + rate-limit + test):

- scripts/ais-relay.cjs: parallel `tankerReports` Map populated for AIS
  ship type 80-89 (tanker class) per ITU-R M.1371. SEPARATE from the
  existing `candidateReports` Map (military-only) so the existing
  military-detection consumer's contract stays unchanged. Snapshot
  endpoint extended to accept `bbox=swLat,swLon,neLat,neLon` + `tankers=true`
  query params, with bbox-filtering applied server-side. Tanker reports
  cleaned up on the same retention window as candidate reports; capped
  at 200 per response (10× headroom for global storage).
- proto/worldmonitor/maritime/v1/{get_,}vessel_snapshot.proto:
  - new `bool include_tankers = 6` request field
  - new `repeated SnapshotCandidateReport tanker_reports = 7` response
    field (reuses existing message shape; parallel to candidate_reports)
- server/worldmonitor/maritime/v1/get-vessel-snapshot.ts: REPLACES the
  prior 5-minute `with|without` cache with a request-keyed cache —
  (includeCandidates, includeTankers, quantizedBbox) — at 60s TTL for
  the live-tanker path and 5min TTL for the existing density/disruption
  consumers. Also adds 1° bbox quantization for cache-key reuse and a
  10° max-bbox guard (BboxTooLargeError) to prevent malicious clients
  from pulling all tankers through one query.
- server/gateway.ts: NEW `'live'` cache tier. CacheTier union extended;
  TIER_HEADERS + TIER_CDN_CACHE both gain entries with `s-maxage=60,
  stale-while-revalidate=60`. RPC_CACHE_TIER maps the maritime endpoint
  from `'no-store'` to `'live'` so the CDN absorbs concurrent identical
  requests across all viewers (without this, N viewers × 6 chokepoints
  hit AISStream upstream linearly).
- server/_shared/rate-limit.ts: ENDPOINT_RATE_POLICIES entry for the
  maritime endpoint at 60 req/min/IP — enough headroom for one user's
  6-chokepoint tab plus refreshes; flags only true scrape-class traffic.
- tests/route-cache-tier.test.mjs: regex extended to include `live` so
  the every-route-has-an-explicit-tier check still recognises the new
  mapping. Without this, the new tier would silently drop the maritime
  route from the validator's route map.

U8 — LiveTankersLayer consumer:

- src/services/live-tankers.ts: per-chokepoint fetcher with 60s in-memory
  cache. Promise.allSettled — never .all — so one chokepoint failing
  doesn't blank the whole layer (failed zones serve last-known data).
  Sources bbox centroids from src/config/chokepoint-registry.ts
  (CORRECT location — server/.../​_chokepoint-ids.ts strips lat/lon).
  Default chokepoint set: hormuz_strait, suez, bab_el_mandeb,
  malacca_strait, panama, bosphorus.
- src/components/DeckGLMap.ts: new `createLiveTankersLayer()` ScatterplotLayer
  styled by speed (anchored amber when speed < 0.5 kn, underway cyan,
  unknown gray); new `loadLiveTankers()` async loader with abort-controller
  cancellation. Layer instantiated when `mapLayers.liveTankers && this.liveTankers.length > 0`.
- src/config/map-layer-definitions.ts: `LayerDefinition` for `liveTankers`
  with `renderers: ['flat'], deckGLOnly: true` (matches existing
  storageFacilities/fuelShortages pattern). Added to `VARIANT_LAYER_ORDER.energy`
  near `ais` so getLayersForVariant() and sanitizeLayersForVariant()
  include it on the energy variant — without this addition the layer
  would be silently stripped even when toggled on.
- src/types/index.ts: `liveTankers?: boolean` on the MapLayers union.
- src/config/panels.ts: ENERGY_MAP_LAYERS + ENERGY_MOBILE_MAP_LAYERS
  both gain `liveTankers: true`. Default `false` everywhere else.
- src/services/maritime/index.ts: existing snapshot consumer pinned to
  `includeTankers: false` to satisfy the proto's new required field;
  preserves identical behavior for the AIS-density / military-detection
  surfaces.

Tests:
- npm run typecheck clean.
- 5 unit tests in tests/live-tankers-service.test.mjs cover the default
  chokepoint set (rejects ids that aren't in CHOKEPOINT_REGISTRY), the
  60s cache TTL pin (must match gateway 'live' tier s-maxage), and bbox
  derivation (±2° padding, total span under the 10° handler guard).
- tests/route-cache-tier.test.mjs continues to pass after the regex
  extension; the new maritime tier is correctly extracted.

Defense in depth:
- THREE-layer cache (CDN 'live' tier → handler bbox-keyed 60s → service
  in-memory 60s) means concurrent users hit the relay sub-linearly.
- Server-side 200-vessel cap on tanker_reports + client-side cap;
  protects layer render perf even on a runaway relay payload.
- Bbox-size guard (10° max) prevents a single global-bbox query from
  exfiltrating every tanker.
- Per-IP rate limit at 60/min covers normal use; flags scrape-class only.
- Existing military-detection contract preserved: `candidate_reports`
  field semantics unchanged; consumers self-select via include_tankers
  vs include_candidates rather than the response field changing meaning.

* fix(energy-atlas): wire LiveTankers loop + 400 bbox-range guard (PR3 review)

Three findings from review of koala73#3402:

P1 — loadLiveTankers() was never called (DeckGLMap.ts:2999):
- Add ensureLiveTankersLoop() / stopLiveTankersLoop() helpers paired with
  the layer-enabled / layer-disabled branches in updateLayers(). The
  ensure helper kicks an immediate load + a 60s setInterval; idempotent
  so calling it on every layers update is safe.
- Wire stopLiveTankersLoop() into destroy() and into the layer-disabled
  branch so we don't hammer the relay when the layer is off.
- Layer factory now runs only when liveTankers.length > 0; ensureLoop
  fires on every observed-enabled tick so first-paint kicks the load
  even before the first tanker arrives.

P1 — bbox lat/lon range guard (get-vessel-snapshot.ts:253):
- Out-of-range bboxes (e.g. ne_lat=200) previously passed the size
  guard (200-195=5° < 10°) but failed at the relay, which silently
  drops the bbox param and returns a global capped subset — making
  the layer appear to "work" with stale phantom data.
- Add isValidLatLon() check inside extractAndValidateBbox(): every
  corner must satisfy [-90, 90] / [-180, 180] before the size guard
  runs. Failure throws BboxValidationError.

P2 — BboxTooLargeError surfaced as 500 instead of 400:
- server/error-mapper.ts maps errors to HTTP status by checking
  `'statusCode' in error`. The previous BboxTooLargeError extended
  Error without that property, so the mapper fell through to
  "unhandled error" → 500.
- Rename to BboxValidationError, add `readonly statusCode = 400`.
  Mapper now surfaces it as HTTP 400 with a descriptive reason.
- Keep BboxTooLargeError as a backwards-compat alias so existing
  imports / tests don't break.

Tests:
- Updated tests/server-handlers.test.mjs structural test to pin the
  new class name + statusCode + lat/lon range checks. 24 tests pass.
- typecheck (src + api) clean.

* fix(energy-atlas): thread AbortSignal through fetchLiveTankers (PR3 review #2)

P2 — AbortController was created + aborted but signal was never passed
into the actual fetch path (DeckGLMap.ts:3048 / live-tankers.ts:100):
- Toggling the layer off, destroying the map, or starting a new refresh
  did not actually cancel in-flight network work. A slow older refresh
  could complete after a newer one and overwrite this.liveTankers with
  stale data.

Threading:
- fetchLiveTankers() now accepts `options.signal: AbortSignal`. Signal
  is passed through to client.getVesselSnapshot() per chokepoint via
  the Connect-RPC client's standard `{ signal }` option.
- Per-zone abort handling: bail early if signal is already aborted
  before the fetch starts (saves a wasted RPC + cache write); re-check
  after the fetch resolves so a slow resolver can't clobber cache
  after the caller cancelled.

Stale-result race guard in DeckGLMap.loadLiveTankers:
- Capture controller in a local before storing on this.liveTankersAbort.
- After fetchLiveTankers resolves, drop the result if EITHER:
  - controller.signal is now aborted (newer load cancelled this one)
  - this.liveTankersAbort points to a different controller (a newer
    load already started + replaced us in the field)
- Without these guards, an older fetch that completed despite
  signal.aborted could still write to this.liveTankers and call
  updateLayers, racing with the newer load.

Tests: 1 new signature-pin test in tests/live-tankers-service.test.mts
verifies fetchLiveTankers accepts options.signal — guards against future
edits silently dropping the parameter and re-introducing the race.
6 tests pass. typecheck clean.

* fix(energy-atlas): bound vessel-snapshot cache via LRU eviction (PR3 review)

Greptile P2 finding: the in-process cache Map grows unbounded across the
serverless instance lifetime. Each distinct (includeCandidates,
includeTankers, quantizedBbox) triple creates a slot that's never evicted.
With 1° quantization and a misbehaving client the keyspace is ~64,000
entries — realistic load is ~12, so a 128-slot cap leaves 10x headroom
while making OOM impossible.

Implementation:
- SNAPSHOT_CACHE_MAX_SLOTS = 128.
- evictIfNeeded() walks insertion order and evicts the first slot whose
  inFlight is null. Slots with active fetches are skipped to avoid
  orphaning awaiting callers; we accept brief over-cap growth until
  in-flight settles.
- touchSlot() re-inserts a slot at the end of Map insertion order on
  hit / in-flight join / fresh write so it counts as most-recently-used.

…2 — flag-gated dark) (koala73#3407)

* feat(resilience): financialSystemExposure dim scaffold (Phase 2 Ship 2 — flag-gated dark)

Adds the new `financialSystemExposure` resilience dimension introduced in
plan 2026-04-25-004 Phase 2, behind the
`RESILIENCE_FIN_SYS_EXPOSURE_ENABLED` env flag. The flag defaults OFF so
the dim ships dark — the scorer returns the empty-data shape (score=0,
coverage=0, imputationClass=null) until the 3 component seeders
(seed-bis-lbs, seed-fatf-listing, seed-wb-external-debt) are populating
in production. Rollout pattern matches energy v2 (plan
2026-04-24-001).

Components (weights total 1.0 inside the dim):
  short_term_external_debt_pct_gni    0.35  (WB IDS, lowerBetter, 15-0)
  bis_lbs_xborder_us_eu_uk_pct_gdp    0.30  (BIS LBS by-parent, U-shape)
  fatf_listing_status                  0.20  (FATF, discrete: black=0, gray=30, compliant=100)
  financial_center_redundancy          0.15  (BIS LBS by-parent count, higherBetter, 1-10)

When the flag flips ON, the scorer preflights all 3 required seed-meta
envelopes (the BIS LBS seed serves both Component 2 and Component 4 —
no separate Component 4 seeder). Missing envelopes throw
`ResilienceConfigurationError(message, missingKeys)` (two-arg form per
Codex R3 P1 #2) which `scoreAllDimensions` catches and routes to
`imputationClass='source-failure'`. Per-country data gaps inside an
otherwise-published envelope are distinct: per-component reads return
null, the slot drops out of the weighted blend.

Cache prefixes bumped in lockstep with the new dim:
  resilience:score:v13:    → v14:
  resilience:ranking:v13   → v14
  resilience:history:v8:   → v9:

The scaffold also reweights `tradePolicy` 1.0 → 0.5 in
RESILIENCE_DIMENSION_WEIGHTS (the new dim shares the economic-domain
weight). This shifts the headline overallScore by ~0.46 points in the
flag-off baseline because the half-weighted tradePolicy contributes
proportionally less to the coverage-weighted economic-domain mean.
When the flag flips on with seeders populated, the new dim's actual
signal will rebalance the headline score.

What this PR ships:
  - new `scoreFinancialSystemExposure` + `normalizeBandLowerBetter` U-shape helper
  - 4 new INDICATOR_REGISTRY entries (BIS-derived tagged non-commercial/enrichment per Codex R1 #8)
  - api/health.js + api/seed-health.js dual-registry entries for the 3 new seed keys
  - frontend label map + ResilienceWidget "20 dimensions" copy
  - methodology doc heading "Financial System Exposure" + indicator table
  - tests/resilience-financial-system-exposure.test.mts: 13 tests pinning the
    formula contract + fail-closed preflight + flag-off rollout posture
  - parity test extension for v14/v9 prefixes
  - 22-dim count update across release-gate + handlers + indicator-registry tests
  - flag-gated-dark exemption in release-gate's coverage check

What this PR does NOT ship (deliberately deferred):
  - The 3 component seeders themselves (seed-bis-lbs, seed-fatf-listing,
    seed-wb-external-debt). These need real-API integration with BIS SDMX,
    FATF HTML scraping, and WB IDS — best landed as a separate PR with
    fixture-recorded tests.
  - Methodology doc `docs/methodology/financial-system-exposure.md` with
    full alternatives + license attribution.
  - Bundle registration in scripts/seed-bundle-macro.mjs.
  - Cohort sanity-check anchor test (RU/IR/KP < 20 on
    financialSystemExposure) — needs real seeder data.

Once those land and seeders populate Redis, flip
`RESILIENCE_FIN_SYS_EXPOSURE_ENABLED=true` in Vercel + Railway env
config to activate the dim.

Stacked on PR koala73#3405 (Phase 1 Ship 1).

🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0

Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>

* feat(resilience): financialSystemExposure component seeders + methodology doc

Completes Phase 2 of plan 2026-04-25-004 by shipping the 3 component
seeders, registering them in the macro bundle, adding 34 new fixture-
based tests, and writing the full methodology doc with license attribution.

Seeders (per plan §Files (Phase 2)):
  scripts/seed-wb-external-debt.mjs    — WB IDS short-term external debt as % GNI
                                         (DT.DOD.DSTC.IR.ZS × DT.DOD.DECT.GN.ZS,
                                          mrv=5 + pickLatestPerCountry per memory
                                          `feedback_wb_bulk_mrv1_null_coverage_trap`)
  scripts/seed-bis-lbs.mjs              — BIS LBS by-parent SDMX integration
                                         (12-dim key with L_POS_TYPE=N per Codex
                                          R3 P1 #1; 16 enumerated parent ISO2 codes
                                          per Codex R4 P1 #2 — no `4F` aggregate;
                                          ISO2 direct, no M49 mapping per CL_BIS_IF_REF_AREA)
  scripts/seed-fatf-listing.mjs         — FATF entry-page parser with dynamic
                                         publication-URL follow + sanity-check
                                         band gates + monthly cadence + 90d cache
                                         TTL fallback

Bundle registration:
  scripts/seed-bundle-macro.mjs — Option A per Codex R1 #5 (less operational
                                  overhead than provisioning a new bundle).

Tests (34 new):
  tests/seed-wb-external-debt.test.mjs  — combineExternalDebt formula pinning,
                                          IMF Article IV 15% GNI anchor,
                                          conservative-year selection, validate floor
  tests/seed-bis-lbs.test.mjs           — combineLbsByCounterparty Brazil 2024Q4
                                          ground-truth anchor, 1% GDP threshold
                                          for parentCount, BIS aggregate-code
                                          allow-list (5J/5A skipped), SDMX-JSON
                                          shape parsing, parser-regression guard
  tests/seed-fatf-listing.test.mjs      — findPublicationLink entry-page anchor
                                          extraction (case-insensitive), country-
                                          name lookup with apostrophe handling
                                          (`People's` → `peoples`), pub-date
                                          inference from URL slug + header,
                                          DPRK-on-call-for-action invariant

Methodology doc:
  docs/methodology/financial-system-exposure.md — full construct definition,
                                                   per-component formulas + score
                                                   shapes + coverage matrices,
                                                   fail-closed preflight,
                                                   methodology invariants,
                                                   sanctions-isolated-jurisdiction
                                                   sanity-check anchors (RU/IR/KP/...
                                                   < 20), bounded-movement gate
                                                   (60%+ |Δ|<3pt), data-source
                                                   licensing (BIS terms of use),
                                                   alternatives considered (5),
                                                   future considerations (Phase 3-5)

Quality:
  - npm run typecheck + typecheck:api: clean
  - npm run test:data: 7149/7149 pass (was 7115; +34 new seeder tests)
  - npm run lint + lint:md + version:check: clean
  - Edge function bundle check: clean

Activation runbook (when ready to flip the dim live):
  1. Merge this PR
  2. Run seed-wb-external-debt + seed-bis-lbs + seed-fatf-listing manually
     via `railway run --service seed-bundle-macro -- node scripts/<seeder>.mjs`
  3. Verify all 3 seed-meta envelopes published:
       redis-cli GET 'seed-meta:economic:wb-external-debt'
       redis-cli GET 'seed-meta:economic:bis-lbs'
       redis-cli GET 'seed-meta:economic:fatf-listing'
  4. Set RESILIENCE_FIN_SYS_EXPOSURE_ENABLED=true in Vercel + Railway env config
  5. Flush v14 caches: bulk DEL resilience:score:v14:* + DEL resilience:ranking:v14
  6. Run seed-resilience-scores.mjs to bulk-warm v14 with the new dim's signal
  7. Cohort audit: snapshot resilience:score:v14:* for all 222 countries;
     RU/IR/KP must score < 20 on financialSystemExposure (gate the construct
     before stable-rollout)
  8. Bounded-movement gate: 60% of countries |Δ|<3pt; outliers > 12pt must
     be in the explicitly-predicted RU/IR/KP/CU/VE/BY/LY/MM set
  9. Remove FLAG_GATED_DARK_DIMENSIONS allow-list entry in
     tests/resilience-release-gate.test.mts in the same commit that flips
     the flag

🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0

Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>

* chore(resilience): address self-review hardening (P1 + 4×P2 + 4×P3)

P1 — BIS LBS sequential-fetch timeout math:
  Sequential 16 parents × 60s timeout = 960s worst-case, exceeding the
  bundle's 600s timeoutMs. SIGTERM mid-flight would have leaked the
  child-lock + produced a covertly-degraded payload (per memory
  `bundle-runner-sigkill-leaks-child-lock`). Fix: parallelize parent
  fetches with concurrency=4 via a bounded-concurrency runner. Caps wall
  time at ~4 × 60s = 240s on the slow path while staying polite to BIS.

P2 — BIS LBS parent-success gate:
  Previous "all 16 failed" short-circuit was too permissive. If 15 of 16
  parent fetches failed, a single-parent payload would pass validate
  (>100 counterparty floor) and skew Component 4 (financialCenterRedundancy)
  low for every counterparty until the next successful run. Added
  MIN_SUCCESSFUL_PARENTS=12 gate; below that, throw → seed-meta unchanged
  → previous valid payload stays alive under cache TTL.

P2 — FATF findPublicationLink prefers highest-year anchor:
  Previous "first anchor matching label" was vulnerable to FATF page
  layouts where a sidebar links to historical publications using the
  same wording. Fix: collect all candidates, sort by year (URL slug or
  anchor text), return highest. Logs all candidates at WARN when more
  than one matches so ops can spot drift.

P2 — FATF unmatched country-name surfacing:
  Previous parser silently dropped country names not in
  shared/country-names.json. If FATF introduced a new spelling
  ("Mauretania", "Türkiye"), the country fell out of the listing and
  defaulted to "compliant" (score 100) — materially shifting its
  financialSystemExposure score under a fresh seed-meta. Fix:
  extractListedCountries now returns { listed, unmatchedCandidates }.
  Seeder logs unmatched at WARN; throws if > 2 candidates unmatched
  (indicates parser drift / new spellings the lookup needs to learn).

P2 — WB external-debt yearMismatch metadata:
  Composition can mix vintages of the two source indicators (different
  WB IDS lag patterns). Previous output silently used min(year) as the
  conservative anchor. Fix: emit `yearMismatch: boolean` +
  `shortTermPctOfTotalDebtYear` + `totalDebtPctOfGniYear` on each
  country record so the dashboard / scorer / audit can flag countries
  with cross-year composition. Pinning test asserts min(year) selection
  + correct flag.

P3 — BIS LBS upper-bound corruption guard:
  Added `latestVal > 1e8` skip in extractClaimsByCounterparty. 1e8
  millions = $100T (>half of global GDP), well above any plausible
  bilateral claim. Drops corrupt SDMX values silently rather than
  letting them inflate totalXborderPctGdp downstream.

P3 — BIS LBS validation floor 100 → 150:
  BIS LBS counterparty coverage is ~200+ jurisdictions. Floor of 100
  would have accepted a payload with massive coverage regression.
  Tightened to 150.

P3 — FATF grey-list floor 8 → 12 + band 8-40 → 12-40:
  Historical FATF grey-list size has been 15+ since 2020. Floor of 8
  was too lenient. Tightened to 12 with comment explaining the
  historical band.

P3 — FATF parse-failure WARN logging:
  All sanity-check throw paths in seed-fatf-listing now emit a
  console.warn explaining the failure and noting "previous valid
  payload remains under cache TTL" before throwing. Plan called for
  "warn loudly"; the implicit fall-back-to-cache pattern is now
  diagnostic-friendly.

Suggestion — BIS LBS droppedForMissingGdp provenance:
  Counterparties seen in BIS LBS but dropped because no WB GDP record
  was available are now collected into a `droppedForMissingGdp` array
  on the seed payload. Surfaces silent coverage gaps for ops triage
  without polluting the main `countries` map.

Suggestion — methodology doc operational footguns + smoke test:
  New §"Common operational footguns" section in
  docs/methodology/financial-system-exposure.md surfacing the BIS LBS
  4F-aggregate-rejection lesson + ISO 3166-1 (not M49) clarification +
  pre-flag-flip smoke test commands.

Quality gates:
  - npm run typecheck + typecheck:api: clean
  - npm run lint + lint:md + version:check: clean
  - npm run test:data: 7153/7153 pass (was 7149; +4 hardening tests)

🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0

Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>

* fix(resilience): scoreFinancialSystemExposure preflights UNVERSIONED seed-meta keys

P1 reviewer catch: the preflight in scoreFinancialSystemExposure was
reading `seed-meta:economic:<key>:v1` while runSeed
(scripts/_seed-utils.mjs) writes the freshness record at
`seed-meta:${dataKey.replace(/:v\d+$/, '')}` — i.e. with the trailing
:v\d+ stripped. Once `RESILIENCE_FIN_SYS_EXPOSURE_ENABLED=true` was
flipped, every /api/resilience/* request would have hit the missing-
seed-meta path indefinitely, throwing ResilienceConfigurationError and
stamping every country's financialSystemExposure as
imputationClass='source-failure' even with healthy seeders running.

The same unversioned shape is already used by api/health.js +
api/seed-health.js + every other in-tree scorer that walks
seed-meta keys via _dimension-freshness.ts.

Fix: route the preflight through `resolveSeedMetaKey` from
_dimension-freshness.ts. That helper already strips the trailing
:v\d+ AND applies the SOURCE_KEY_META_OVERRIDES table — the canonical
in-tree pattern for "given a registry data-key, return the seed-meta
key that runSeed actually writes." Inlining the regex would have
re-introduced the same writer/reader drift this helper exists to
prevent.

Tests:
  - All formula + preflight tests (which previously mocked the
    incorrect versioned form) updated to the unversioned key shape so
    they actually exercise the production read path.
  - New regression-guard test "preflight reads UNVERSIONED seed-meta
    keys (matches runSeed write-key shape)" pins the exact key shape
    the scorer probes. Asserts both presence of the unversioned form
    AND absence of the versioned form. A future refactor that
    accidentally re-versions the preflight will fail loudly here
    instead of silently routing every country to source-failure.

Quality gates:
  - npm run typecheck:api: clean
  - npm run test:data: 7154/7154 pass (was 7153; +1 contract guard)

🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0

Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>

* fix(resilience): address Greptile P1 + P2 catches on PR koala73#3407 (4 issues)

P1 — 30-point scoring cliff at 25% boundary in normalizeBandLowerBetter:
  Original draft had piecewise-linear segments with mismatched endpoints:
    - At value=25:    sweet spot ended at 100, over-exposed started at 70
                      → 30-point cliff × 0.30 weight ≈ 9-pt headline swing
    - At value=5:     low-int ended at 70, sweet started at 75 → 5-pt jump
  Cliffs in piecewise-linear scorers cause ranking instability for
  countries near band edges (24.9% scores ~99.9, 25.1% scores ~70).
  Re-anchored adjacent segments to share endpoints — function is now
  piecewise-CONTINUOUS at 5%, 25%, and 60% transitions:
    0%-5%:    60 → 75   (slope +3/pct, was +2)
    5%-25%:   75 → 100  (unchanged)
    25%-60%:  100 → 30  (slope −2/pct, was 70 → 30 / slope −1.14)
    60%+:     30 → 0    (unchanged)
  New regression test pins continuity at all three boundaries (samples
  values immediately above/below each transition, asserts |Δ| ≤ 1pt).

P2 — readFatfStatus defaults empty listings dict to "compliant":
  An empty `listings: {}` payload that bypassed the seeder's validate()
  would silently score every country at 100 (compliant default), masking
  a parser regression. Added defense-in-depth guard: empty dict → null
  component score → slot drops out of weighted blend → coverage shrinks
  visibly rather than the dim looking healthy. Seeder validate already
  enforces ≥1 black + ≥12 grey so this can't reach production through
  the normal write path; the guard costs nothing and catches malformed
  payloads that bypass validation. New regression test pins.

P2 — bisLbsXborderPctGdp goalposts mismatch U-shape peak:
  Registry entry had `goalposts: { worst: 60, best: 15 }` implying a
  linear lowerBetter scale peaking at 15%. The U-shape function actually
  peaks at 25% (value=15% scores 87.5, not 100). Updated to
  `{ worst: 60, best: 25 }` (over-exposed branch, peak anchor) with an
  explicit comment that goalposts here are documentation-only — the
  actual scorer uses normalizeBandLowerBetter, not a generic linear
  normalizer. Tooling reading the registry should consult the scorer
  helper directly.

P2 (resolved differently than originally suggested) —
api/bootstrap.js missing 3 new data keys:
  Greptile flagged that AGENTS.md requires bootstrap hydration for new
  data sources. Verified the bootstrap-hydration-coverage test enforces
  this via a `getHydratedData` consumer requirement in src/.
  The 3 new keys (economic:wb-external-debt:v1, economic:bis-lbs:v1,
  economic:fatf-listing:v1) are SERVER-ONLY — they feed
  scoreFinancialSystemExposure inside /api/resilience/* handlers; no
  client panel consumes them directly. Adding them to bootstrap without
  a consumer would fail the parity tests. Documented in api/bootstrap.js
  as a deferred entry: "if a future PR adds a client panel that displays
  raw BIS LBS / FATF / WB external-debt data, register the keys here
  AND add the corresponding consumer + cache-keys.ts entries in the
  same PR."

Note on Greptile's P1 #1 (preflight `:v1` suffix mismatch): already
resolved in commit d904103 (uses `resolveSeedMetaKey` from
_dimension-freshness.ts). Greptile reviewed an earlier commit.

Quality gates:
  - npm run typecheck:api: clean
  - npm run lint + lint:md + version:check: clean
  - npm run test:data: 7156/7156 pass (was 7154; +2 regression guards)

🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0

Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>

…populates (koala73#3410)

* fix(ais-relay): subscribe to ShipStaticData so tanker layer actually populates

User-reported on energy.worldmonitor.app: Live Tanker Positions layer
renders zero vessels despite PR koala73#3402 wiring being correct end-to-end.

Root cause:
  AISStream's PositionReport.MetaData does NOT carry `ShipType` per their
  schema — that field only arrives in ShipStaticData (Type 5) frames.
  PR koala73#3402 shipped tanker capture predicated on `meta.ShipType`:

      const shipType = Number(meta.ShipType);  // always NaN
      if (Number.isFinite(shipType) && shipType >= 80 && shipType <= 89) {
        tankerReports.set(mmsi, ...);  // never executes
      }

  The relay subscribed to AISStream with `FilterMessageTypes:
  ['PositionReport']` only — ShipStaticData never reached the relay, so
  the predicate always failed (NaN), `tankerReports` stayed permanently
  empty, and getVesselSnapshot returned `tankerReports: []` to every
  caller. The frontend layer correctly rendered zero vessels.

  Military detection (isLikelyMilitaryCandidate) survived because it has
  fallback paths (NAVAL_PREFIX_RE on ShipName + MMSI-suffix 00/99). Tanker
  detection had zero fallback because tanker MMSIs and names don't follow
  a single regex pattern.

Fix:
  - Add 'ShipStaticData' to AISStream's FilterMessageTypes — Type 5
    frames are broadcast every ~6 min per vessel (vs every 2-10s for
    PositionReport while underway), so the volume add is small.
  - Dispatch ShipStaticData → new processShipStaticDataForMeta handler
    that caches { shipType, shipName, lastSeen } by MMSI in a vesselMeta
    Map.
  - Tanker capture in processPositionReportForSnapshot now falls back to
    vesselMeta.get(mmsi).shipType when meta.ShipType is missing.
  - vesselMeta gets TTL eviction (24h) in cleanupAggregates so a long-
    running relay doesn't accumulate metadata for vessels that have left
    all tracked regions. 24h covers vessels with intermittent visibility
    while bounding memory growth.

Tests:
  - tests/relay-tanker-shipstatic.test.mjs (new, 5/5 pass) — pins the
    fix shape so a regression can't flip FilterMessageTypes back to
    PositionReport-only.
  - `node -c` clean.

Deploy:
  Requires a Railway redeploy of the AIS-relay service to pick up the
  new subscription filter. After redeploy, vesselMeta needs ~5-10 min
  to populate from ShipStaticData broadcasts before tanker classification
  reaches steady state for vessels in the 6 chokepoint bboxes.

* fix(ais-relay): tighten ShipStaticData parsing — UserID fallback + field-name pinning (PR3410 review)

Three review findings on PR koala73#3410:

P2 — Static-analysis tests don't pin field names (correctness/testing):
  Pre-fix tests asserted `vesselMeta.set(... { shipType` which only checks
  the property name, not the source field. A typo regression like
  Number(sd.Typ) or Number(sd.shipType) (lowercase) would still pass —
  silently re-emptying the tanker layer.
  Added two new tests:
    - "reads ShipType from sd.Type (NOT meta.ShipType)" — pins the exact
      Number(sd.Type) substring; also pins sd.Name for shipName fallback.
    - "accepts MMSI from meta.MMSI OR sd.UserID" — pins the new fallback
      shape (see #2 below).

P2 — MMSI-extraction lacks UserID fallback (correctness):
  AISStream's ShipStaticData payload sample mirrors MMSI as `UserID` on
  the message body, while PositionReport puts it under MetaData.MMSI.
  If a wrapper-schema variant ever ships a Type 5 frame without
  MetaData.MMSI, processShipStaticDataForMeta early-returned and
  vesselMeta stayed empty — silent re-empty of the tanker layer.
  Defensive: `String(meta.MMSI || sd.UserID || '')`.

P3 — Test #4 order check used file-wide indexOf (testing):
  RELAY.indexOf('vesselMeta.get(mmsi)') finds the FIRST occurrence in
  the file. A future earlier vesselMeta.get(...) elsewhere would let
  the in-tanker-path lookup be removed without the test failing.
  Scoped the substring search to the body of
  processPositionReportForSnapshot (slice from fn declaration to the
  next top-level `function ` line).

Tests: 7/7 pass (was 5). `node -c` clean.

* fix(ais-relay): broaden shipType fix — chokepoint transit + military candidate paths also use vesselMeta

User-reported review finding on PR koala73#3410: the previous commit fixed only
the tanker snapshot path. Two more sites in processPositionReportForSnapshot
read the same broken meta.ShipType field:

  Site 1 (line ~6651, vessels record):
    vessels.set(mmsi, { ..., shipType: meta.ShipType, ... });
  Consumer: classifyVesselType(vessel?.shipType) at the chokepoint
  transit logging site (line ~6545). With shipType permanently undefined,
  classifyVesselType always returned 'other' — silently breaking per-type
  transit counts in /seedChokepointTransits and any downstream consumer
  using transit-by-type breakdowns.

  Site 2 (line ~6692, military candidate record):
    candidateReports.set(mmsi, { ..., shipType: meta.ShipType, ... });
    isLikelyMilitaryCandidate(meta) — read meta.ShipType for the
    35/55/50-59 type-based arms.
  Consequence: the military classifier survived in production only via
  the NAVAL_PREFIX_RE + MMSI-suffix fallbacks. The type-based arm never
  fired. candidateReports also exposed shipType: undefined to callers.

Fix:
  - Compute `effectiveShipType` ONCE at the top of processPositionReportForSnapshot,
    preferring meta.ShipType when present (defense for any future schema
    enrichment) and falling back to vesselMeta cache from ShipStaticData.
  - Pass effectiveShipType to vessels.set, isLikelyMilitaryCandidate, and
    the tanker-capture predicate. All three sites now classify correctly.
  - Refactor isLikelyMilitaryCandidate(meta) → isLikelyMilitaryCandidate(meta, resolvedShipType)
    with backward-compat fallback to meta.ShipType when no override is passed.

Same root cause, same vesselMeta cache solves all three sites.

Tests: 10/10 (was 7) — added 3 new regression tests pinning:
  - vessels.set shipType field comes from `effectiveShipType`
  - isLikelyMilitaryCandidate signature accepts resolvedShipType param
  - candidateReports.set shipType field comes from `effectiveShipType`
node -c clean.

* fix(ais-relay): hard size cap + Type=0 guard on vesselMeta (PR3410 Greptile review)

Two findings on PR koala73#3410:

P1 — vesselMeta has TTL but no hard size cap:
  Every peer Map in cleanupAggregates (tankerReports, candidateReports,
  densityGrid, vesselHistory) follows its TTL loop with evictMapByTimestamp.
  vesselMeta did not — leaving memory growth unbounded against a hostile
  or buggy upstream flooding unique MMSIs faster than TTL drains them.
  Added MAX_VESSEL_META=50000 (covers ~50-70k active global AIS fleet
  with headroom) + evictMapByTimestamp call after the TTL loop.

P2 — Number(null) === 0 could overwrite valid cache entries:
  processShipStaticDataForMeta's `if (!Number.isFinite(shipType)) return`
  guard accepts shipType=0 (AIS code 0 = "Not available" per ITU-R M.1371).
  A vessel that broadcasts {Type: 85} then later {Type: null} would have
  its cached tanker classification overwritten with shipType=0, downgrading
  it to non-tanker on the next PositionReport.
  Tightened to `|| shipType <= 0`.

Tests: 11/11 (was 10) — added 2 new pins:
  - vesselMeta has TTL AND hard size cap (asserts MAX_VESSEL_META +
    evictMapByTimestamp(vesselMeta, ...))
  - processShipStaticDataForMeta rejects shipType <= 0 (asserts the
    `<= 0` guard so a refactor can't silently drop it)
node -c clean.

…flow (financialSystemExposure activation) (koala73#3412)

* fix(resilience): correct WB indicator + switch BIS LBS → BIS CBS dataflow

Two activation-time bugs caught when running the financialSystemExposure
seeders against production Redis (PR koala73#3407 follow-up).

== Fix #1 — WB IDS short-term external debt indicator ==

Original code used `DT.DOD.DSTC.IR.ZS × DT.DOD.DECT.GN.ZS / 100`,
intended to compose "short-term external debt as % of GNI." But
`DT.DOD.DSTC.IR.ZS` is "% of total RESERVES" (NOT "% of total
external debt"). Argentina (164%), Turkey (114%), and Sri Lanka all
had intermediate ratios > 100% because their short-term debt exceeds
reserves — caught by Redis audit on the first live seed run.

Fix: divide absolute USD values directly.
  shortTermDebtPctGni = (DT.DOD.DSTC.CD / NY.GNP.MKTP.CD) × 100

Live verification: BR=4%, AR=7.77%, TR=13.26%, LK=6.03%, NG=9.05% —
all in plausible 0-15% range matching real-world short-term debt ratios.

== Fix #2 — BIS LBS → BIS CBS dataflow ==

Plan called for "BIS LBS by-parent" data, but `WS_LBS_D_PUB` only
exposes counterparty as the aggregate `5J`. Per-counterparty
breakdowns require `WS_CBS_PUB` (Consolidated Banking Statistics).

CBS: 11 dimensions (LBS had 12), parent is L_REP_CTY, CBS_BASIS=F
selects foreign-claims ultimate-risk basis. SDMX key:
  Q.S.<PARENT>.4B.F.C.A.A.TO1.A.

Live verification: 201 records, all 16 parents fetched, BR=19.6%,
CN=3.1%, AE=46.7%, SG=233%, CH=278% — values match real-world bank
exposure including U-shape penalty for hub jurisdictions.

Filename + Redis key (`economic:bis-lbs:v1`) retained for historical
continuity; scorer-side contract unchanged.

What's pending for full activation: FATF seeder still blocked by
HTTP 403 on fatf-gafi.org. Separate PR.

Quality gates:
  - typecheck:api / lint / lint:md: clean
  - npm run test:data: 7189/7189 pass

🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0

Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>

* fix(resilience): address Greptile P2s on PR koala73#3412 + hub-self-exclusion

Three fixes on top of the BIS LBS → CBS migration.

== Greptile #1 — BIS_AGGREGATE_CODES Set was dead code ==

The filter chain was:
  cpCode.length !== 2 || !/^[A-Z]{2}$/.test(cpCode) || BIS_AGGREGATE_CODES.has(cpCode)

Every entry in BIS_AGGREGATE_CODES had a digit (5J, 1C, A2, 4F, ...),
so all of them already failed the `/^[A-Z]{2}$/` regex BEFORE the Set
was consulted. The Set never caught anything in production.

Real risk the Set was supposed to mitigate: BIS introduces a 2-letter
ALL-ALPHA aggregate (e.g. `EU`) that the regex would pass. Replaced
with `ALPHA_AGGREGATE_CODES` — empty by default, audited each CBS
quarterly publish. Verified against the live `WS_CBS_PUB`
L_CP_COUNTRY codelist (252 values as of 2026-04-25): no current
2-letter all-alpha aggregates exist.

== Greptile #2 — combineLbsByCounterparty alias removed ==

`export { combineCbsByCounterparty as combineLbsByCounterparty }` made
the old LBS name a first-class export of the module. Any future code
importing `combineLbsByCounterparty` would silently get CBS data while
the name implied LBS semantics. Only the test file consumed it.

Removed the alias, renamed test imports to `combineCbsByCounterparty`.

== Self-noted finding — hub-country self-counting in Component 4 ==

Caught during the 2026-04-25 production activation audit. For
Singapore and Switzerland (both in PARENT_COUNTRIES AND on the
counterparty list), the BIS CBS payload included their own domestic
banking claims:
  SG-on-SG:  $584B
  CH-on-CH:  $2.2T

These are domestic loan books, NOT foreign-bank cross-border exposure.
Component 4 (`parentCount`) is supposed to measure "how many INDEPENDENT
FOREIGN USD-clearing routes remain if a major counterparty pulls" —
domestic banks aren't a foreign-fallback route.

Filter: drop entries where `cp === parent` when building
claimsByCpByParent. Live verification of the impact:

  Country  OLD pct      NEW pct      OLD parentCount  NEW parentCount
  SG       232.98%      126.11%      12               11
  CH       277.89%       43.69%      10                9
  US        ...          32.2%       (US-on-US excluded)
  GB        ...          65.96%      (GB-on-GB excluded)
  BR        19.62%       19.62%      4                 4 (unchanged; BR not in PARENT_COUNTRIES)
  AE        46.69%       46.69%      10               10 (unchanged; AE not in PARENT_COUNTRIES)

CH dropped from "Iceland-2008 territory" to "over-exposed" — the old
number was a measurement artifact. The new numbers reflect actual
foreign-bank exposure as the construct intends.

Tests: new "excludes self-claims" regression guard pins the SG/CH
behavior. Methodology doc gets a §"Self-exclusion rule" callout under
Component 4.

Quality gates:
  - typecheck:api / lint / lint:md: clean
  - npm run test:data: 7190/7190 pass (was 7189; +1 self-exclusion guard)
  - Live production seed run: 201 records, hub-self-exclusion verified

🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0

Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>

…a73#3431)

* feat(convex/broadcast): audience-export pipeline for PRO launch

Stacked on feat/customers-normalized-email-index — needs that branch's
customers.normalizedEmail field + index + backfill before this action
can produce a clean dedup.

Adds:
- convex/broadcast/audienceExport.ts: paginated internalAction plus
  three internalQueries that build the deduped audience and push
  contacts to a Resend Audience via the Contacts API.

Dedup formula: registrations − emailSuppressions − customers. Any
user who's been through Dodo checkout (active, cancelled, expired)
is excluded — pitching paid users a "buy PRO!" email is the worst-
case failure mode the launch is trying to avoid.

Idempotent on re-run: Resend's 422 already_exists response is counted
as `alreadyExists`, not `failed`. Cursor-paginated for safe resume
mid-export. Dry-run mode (counts only, no Resend calls) for verifying
exclusion math before the first real send.

Codegen catch-up:
- _generated/api.d.ts updated to declare broadcast/audienceExport
  (this file cross-references its own siblings via internal.* and
  needs the typed reference).
- Also catches up payments/backfillCustomerNormalizedEmail from the
  base PR — that file didn't cross-reference so its CI passed
  without the codegen update; adding now so future calls into it
  resolve correctly.
- Convex regenerates identically on deploy — no drift risk.

Operational:
  npx convex run broadcast/audienceExport:exportProLaunchAudience \
    '{"audienceId":"aud_xxx"}'
  # Loop, passing continueCursor from each response, until isDone:true

* fix(audienceExport): address PR review (P1×3)

All three findings valid; verified line numbers match commit 7a440c1
and the Resend API claim against current docs at
https://resend.com/docs/api-reference/contacts/create-contact.

P1 #1 — Paid-customer exclusion fails open when normalizedEmail missing:
`getPaidEmails` now derives the join key from `customers.email` (the
existing `email.trim().toLowerCase()` convention) when `normalizedEmail`
is unset. The backfill becomes a perf optimization — a missed/incomplete
run can no longer silently leak paid users into the launch audience.

P1 #2 — Wrong Resend endpoint:
Switched from `POST /audiences/{id}/contacts` (legacy, may still resolve
but no longer the canonical 2026 path) to `POST /contacts` with
`segments: [{ id: segmentId }]` in the body, per current Resend docs.
Renamed the action arg from `audienceId` to `segmentId` to match
Resend's renaming of Audiences → Segments. Updated JSDoc usage examples.

P1 #3 — 409/422 catch-all masking validation errors:
Added `isDuplicateContactError()` helper that parses the 422 body and
matches on `name` (`*_already_exists` / `*_duplicate`) and `message`
(`/already (exists|in)|duplicate/`). Only duplicate-shaped responses
increment `alreadyExists`; everything else (missing segment, invalid
email, unauthorized field, etc.) increments `failed` and logs the
parsed body. Also dropped the speculative 409 branch — Resend doesn't
use 409 for contacts; 422 is the real status.

* fix(audienceExport): two-step upsert guarantees segment membership (P1)

Round-2 review finding: previous fix counted POST /contacts 422 duplicates
as alreadyExists without verifying the contact ended up in OUR segment.
Resend's global-Contacts model means a 422 duplicate could mean the
contact exists in a different segment or no segment at all — and the
`segments` field on the duplicate path is not applied — so existing
global contacts could be silently OMITTED from the launch segment while
the export reports success.

New `upsertContactToSegment(apiKey, email, segmentId)` helper does a
deterministic two-step:

  1. POST /contacts with `segments: [{ id }]`. Brand-new globally →
     creates and assigns in one call → outcome=created.
  2. If step 1 returns a duplicate-shaped 422, the contact exists
     globally. Disambiguate with POST /contacts/{email}/segments/{id}:
       - 2xx → was global-only or in another segment, now linked here
         → outcome=linkedExisting
       - 422 duplicate-shaped → was already in this segment
         → outcome=alreadyInSegment
       - anything else → outcome=failed

Stats updated:
- `upserted` now counts BOTH `created` AND `linkedExisting` (anything
  that ended up in the segment via this call).
- `linkedExisting` added as a separate diagnostic counter — operator
  can compare to verify how many were pre-existing global contacts.
- `alreadyExists` now strictly means "was already in this segment
  before this call" — never inferred from a global-duplicate response
  without segment-membership verification.

* fix(audienceExport): address PR review round 3 (P2×3)

Stale: P1 "422 conflates duplicate with validation errors" was cited
on commit 7a440c1; addressed in ebf2bf8 — replied as resolved.

Three valid P2s on current code, all fixed:

P2 — Missing User-Agent header on Resend fetches:
AGENTS.md:185 requires User-Agent on every server-side fetch. Added
USER_AGENT constant and applied to both /contacts and
/contacts/{email}/segments/{id} POSTs.

P2 — Raw email PII written to Convex dashboard logs:
The action's failure-path console.error interpolated `${email}` into
log lines that anyone with project access can read. Added maskEmail()
helper (`[email protected]` → `jo******@example.com`) and applied
to the failure-path log.

P2 — `upserted` semantics differ between dry-run and live:
The previous draft incremented stats.upserted in BOTH dry-run (every
email passing dedup) and live (only successful Resend calls). Operators
comparing dry-run to live totals to validate dedup math would see a
spurious discrepancy on any failure / linkedExisting / alreadyInSegment.

Reshaped ExportStats:
- Dry-run-only: `wouldUpsertAfterDedup` — count of emails that pass
  the dedup filters and would be attempted on a live run.
- Live-only: `upserted`, `linkedExisting`, `alreadyExists`, `failed`
  — strictly result of Resend interactions; all zero in dry-run mode.
- Shared (dedup-only): `suppressedSkipped`, `paidSkipped`, `emptyEmail`.

Operator can now compare dry-run `wouldUpsertAfterDedup` to live
`upserted + alreadyExists` and any divergence is a real Resend issue,
not a counter-semantics artefact.

…#3473)

* feat(broadcast): cron-driven ramp runner with kill-gate halt

Replaces the manual three-command ritual (assignAndExportWave →
createProLaunchBroadcast → sendProLaunchBroadcast) with a daily
cron at 13:00 UTC that:

  1. Fetches the prior wave's getBroadcastStats
  2. Halts (sets killGateTripped=true, deactivates ramp) if bounce
     rate > 4% or complaint rate > 0.08% — operator must clear
     before resume
  3. Otherwise runs assignAndExportWave + create + send for the
     next tier in `rampCurve`

Singleton config table `broadcastRampConfig` (keyed by literal
"current") holds the curve, current tier, kill-gate state, and last-
wave tracking. Admin mutations: initRamp / pauseRamp / resumeRamp /
clearKillGate / abortRamp / getRampStatus.

Safety rails:
- `MIN_DELIVERED_FOR_KILLGATE = 100`: kill-gate ignored until prior
  wave has enough delivered events for stable rate calc (avoids
  trip on sample-size noise: 1 bounce / 10 delivered = 10%)
- `MIN_HOURS_BETWEEN_WAVES = 18`: cron defers if prior wave is
  fresher than 18h (bounces / complaints take time to flow back via
  Resend webhook)
- `UNDERFILL_RATIO = 0.5`: deactivates ramp when assignAndExportWave
  returns < 50% of requested count (pool drained signal)
- Kill-gate latch is one-way — never auto-clears. Operator runs
  `clearKillGate '{"reason":"..."}'` after investigating, which
  stamps the cleared reason into lastRunStatus for audit
- Partial-failure recovery: if assignAndExportWave / create /
  send throws mid-flight, status records as "partial-failure" with
  the offending error and the cron blocks until cleared. Throws
  bubble to Convex auto-Sentry for paging
- `_recordWaveSent` mutation does an `expectedCurrentTier` check
  before patching — two concurrent cron firings can't both advance
  the same tier (defence-in-depth; cron isn't supposed to overlap
  but Convex doesn't guarantee at-most-once on retried cron runs)

Wave-label naming: `${prefix}-${tier + offset}`. Default offset 3
means tier 0 → wave-3, tier 1 → wave-4, etc. — picks up cleanly
after manually-sent canary-250 + wave-2.

Daily-cron timing 13:00 UTC: late enough that overnight bounces /
complaints from the prior 24h have flowed back via webhook, early
enough (9am ET / 6am PT / 3pm CET) that a tripped kill-gate hits
US business hours for triage.

Files:
- convex/schema.ts: new `broadcastRampConfig` table + by_key index
- convex/broadcast/rampRunner.ts: runDailyRamp action + admin
  mutations + the two recording mutations
- convex/crons.ts: wires runDailyRamp to crons.daily
- convex/_generated/api.d.ts: regenerated

Operator setup (run once after deploy):

  npx convex run broadcast/rampRunner:initRamp '{
    "rampCurve": [1500, 5000, 15000, 25000],
    "waveLabelPrefix": "wave",
    "waveLabelOffset": 3
  }'

After that, the cron handles everything until either kill-gate trips
or the pool drains. Status check anytime via:

  npx convex run broadcast/rampRunner:getRampStatus '{}'

* fix(broadcast): seed prior wave + halt on partial export failures

PR koala73#3473 review:

P1 #1 — first automated wave skipped kill-gate for the last manually
sent wave because `initRamp` had no way to seed prior-wave metadata.
With currentTier=-1 and lastWaveBroadcastId=undefined, the kill-gate
block at runDailyRamp's Step 1 was unreachable on the first tick after
init. Add `seedLastWave*` optional args; require them as a pair when
`waveLabelOffset > 0` (operational signal that this is a resumption
after manual waves, not a fresh ramp).

P1 #2 — runner narrowed `assignAndExportWave`'s return type to only
`{segmentId, assigned, underfilled}`, dropping `failed` and
`stampFailed`. A wave that requested 500 with 250 push failures + 250
successes would have proceeded to create + send the broadcast, marking
the tier as cleanly advanced. `stampFailed > 0` is worse: contacts are
in the Resend segment (will be emailed) but unstamped (re-eligible for
the next pick → guaranteed duplicate-email). Now: widen the local type
to the full `WaveExportStats`, export it from audienceWaveExport.ts,
and abort the run with `partial-failure` status if either failure
counter is non-zero. Operator clears via the existing
`lastRunStatus === partial-failure` gate.

* fix(broadcast): add clearPartialFailure recovery mutation

PR koala73#3473 review (third P1):

The partial-failure block I added in 35091b5 (treat any non-zero
export failure counter as halt-don't-proceed) had no recovery path.
`runDailyRamp` refuses to advance while
`lastRunStatus === "partial-failure"`, but `clearKillGate` no-ops when
`killGateTripped` is false — so a partial-failure would block the cron
forever short of `abortRamp` or hand-patching the DB.

Add `clearPartialFailure(reason: string)` matching the `clearKillGate`
shape: requires partial-failure status (else no-op), records audit
reason in `lastRunStatus`, clears `lastRunError`. Kept separate from
`clearKillGate` deliberately — kill-gate is an email-reputation
investigation (bounce/complaint thresholds), partial-failure is a
mechanical export/send investigation (Resend logs, Convex stamp
errors). Different recovery requirements; conflating them would
encourage operators to clear without reading the right log.

Updated operator-usage docstring with the new command.

…eads (koala73#3667)

* fix(brief): grounding gate on digest synthesis — block hallucinated leads

The 2026-05-12 0801 send shipped a "President Biden announced a new
executive order targeting cryptocurrency mixers and privacy coins" lead
to users whose actual story pool was Trump-era geopolitics (Iran
ceasefire, Israeli strikes in Lebanon, Sudan drones, Cuba rhetoric,
Russia/Ukraine, EU sanctions). The magazine envelope rendered the
correct grounded lead; the email Executive Summary block rendered a
fully fabricated narrative with four threads all about the imaginary
Biden EO. Shape was valid — content was a hallucination from training-
data priors.

Root cause: validateDigestProseShape only checks shape (lengths,
types, array presence). Any well-formed JSON whose lead names entities
absent from the input pool was happily cached for 4h and re-served on
every cache hit until the row TTL'd out. The canonical-brain promise
from PR koala73#3396 ("compose-phase synthesis is reused on send-phase cache
read") only guarantees same-output-per-cache-hit; it does not guard
against the LLM committing to a fabricated row in the first place.

This adds checkLeadGrounding(synthesis, stories): extract proper-noun
tokens (capitalised, length ≥4) from story headlines; require ≥2
distinct hits in the lead + thread teasers (relaxed to 1 when the
corpus has <4 anchor tokens). Length cap of 4 deliberately filters
short-form acronyms (US/EU/UN/RSF) which are too generic to
discriminate. Plumbed through validateDigestProseShape (new optional
stories arg, back-compat preserved) → parseDigestProse →
generateDigestProse cache-hit path. Cache key bumped v4 → v5 so
existing v4 rows (which may carry shape-valid hallucinations) are
evicted on rollout; 4h paid-for-once cost matching the established
pattern at this function.

When a cached row fails the grounding gate, the cron's existing 3-level
fallback chain falls through to L2 (capped pool, no profile/greeting)
and ultimately L3 (stub with "Digest" subject). A user gets either a
re-rolled grounded lead or a degraded subject line — never a
hallucinated headline.

Test coverage: the verbatim May 12 hallucinated lead + the verbatim
2026-05-12 story headlines as fixtures, asserting the validator
rejects. Plus the actual magazine lead from the same day as positive
control, plus skip-on-empty-stories back-compat, plus single-story
threshold relaxation, plus short-acronym filtering, plus a
generateDigestProse cache-hit re-LLM regression test.

86/86 brief-llm tests pass. 179/179 brief-related tests pass.
typecheck clean. biome lint clean.

* fix(brief): address PR koala73#3667 review — lead must ground independently + token-set matching

Two real bugs caught on review of the grounding validator.

#1 — Combined haystack let a hallucinated lead pass when teasers
happened to mention real entities.

  Before: lead + threads[].teaser were joined into one string and
  any ≥2 anchor hits across the combination passed. So a synthesis
  with a fabricated "President Biden announced..." lead and grounded
  teasers like "Trump rejected Tehran's response" would accept —
  even though the visible top-of-email lead stayed fabricated.

  After: lead must independently hit ≥1 anchor. Combined check
  (≥2 hits, or ≥1 for sparse corpora) still applies on top of that.
  A hallucinated lead with grounded teasers now correctly rejects.

#2 — haystack.includes(tok) accepted unrelated entities via
substring match.

  Before: anchor "iran" hit on "tirana", "oman" on "romania",
  "india" on "indiana". Any anchor that happens to appear as a
  substring of an unrelated word in the synthesis prose counted.

  After: both sides tokenise on the same delimiter regex into Sets
  and match by membership. Story-side keeps the proper-noun
  capitalisation filter; synthesis-side does not (so possessive
  forms / sentence-medial mentions still count as anchor hits).

Both regressions covered by new golden tests:
  - hallucinated-lead-grounded-threads: rejects with the new lead
    independence requirement, accepts when the lead is also grounded
  - substring-trap-corpus (Iran/Oman/India + Tirana/Romania/Indiana):
    rejects under token-set matching, accepts under real word matches

Tests: 88/88 brief-llm, 8604/8604 full suite, typecheck clean,
biome clean.

* fix(brief): address PR koala73#3667 review round 2 — stopword filter + Unicode delimiters

Two more bugs in the grounding validator caught on second review.

#1 (HIGH) — Generic capitalised words leaked into the anchor set.

  Before: any capitalised word ≥4 chars from a headline became an
  anchor. So "President Trump signed Iran sanctions" added
  "president" to storyTokens. A hallucinated lead like "President
  Biden announced..." passed the lead-anchor check via the shared
  "president" token, and a teaser mentioning Iran satisfied the
  combined threshold — recreating the exact failure mode this PR
  is trying to block.

  After: GROUNDING_ANCHOR_STOPWORDS filters honorifics ("President",
  "Senator", "Minister"), generic role labels ("Officials",
  "Members", "Forces"), quasi-adjectives ("Senior", "Federal",
  "Former"), sentence-start filler ("After", "Following",
  "Despite"), and calendar words. Specific entity names (Iran,
  Trump, Israel, EU) are deliberately NOT on the list. "May" is
  also omitted (Theresa May, May Day, May the month all collide).

#2 (MEDIUM) — Unicode apostrophes weren't delimiters.

  Before: GROUNDING_TOKEN_DELIMS only included ASCII apostrophe.
  Reuters/AP/Guardian headlines use U+2019 ("China's", "Iran's",
  "DPRK's"). The regex didn't split on U+2019, so "China's" became
  a single token "china's" that no normal lead saying "China"
  could ever match — a false negative that would reject genuinely
  grounded leads.

  After: U+2018, U+2019, U+201C, U+201D, U+00B4 added alongside
  ASCII counterparts.

Both regressions covered by new golden tests:
  - "President Trump signed..." headlines + "President Biden
    announced..." hallucinated lead must REJECT
  - U+2019 headlines + ASCII-quote grounded lead must ACCEPT

Tests: 90/90 brief-llm, 8606/8606 full suite, typecheck clean,
biome clean.

* fix(brief): address PR koala73#3667 review round 3 — bigram-leading title stopwords

Round 2 added "President" to the anchor stopword list. Round 3 review
caught that other common bigram titles still leak through.

  Before: "Prime Minister Netanyahu says Iran threats continue" added
  "prime" to anchors. A hallucinated "Prime Minister Trudeau announced
  cryptocurrency restrictions..." passed the lead-anchor check via the
  shared "prime" token. A teaser mentioning Iran satisfied the combined
  threshold. Same shape worked for Chief Justice / Cardinal X /
  Chancellor X / Speaker X / Ambassador X.

  After: GROUNDING_ANCHOR_STOPWORDS extended with bigram-leading
  titles: prime, chief, premier, chancellor, speaker, ambassador,
  envoy, commissioner, attorney, cardinal, archbishop, monsignor,
  reverend, pastor, bishop, lord, lady, dame, congressman/woman/person,
  representative, delegate, baron(ess).

Regression test covers Prime Minister / Chief Justice / Cardinal
ride-along leads, plus a counter-control naming the actual entity
(Netanyahu) which still passes.

Tests: 91/91 brief-llm, 8607/8607 full suite, typecheck clean,
biome clean.

* fix(brief): observability for digest synthesis failures (PR koala73#3667 review #3)

Greptile review caught: when generateDigestProse returns null, ops
can't tell whether the LLM threw, returned no content, returned
malformed JSON, or returned a shape-valid hallucination that the
grounding gate rejected. All four classes look identical in logs —
just "L1 returned null" — so a sustained model regression looks
identical to an infra blip during on-call triage.

Add two distinct console.warn lines:
  - "LLM call threw" — provider/network failure (catches the actual
    error message for triage)
  - "ungrounded or malformed output" — provider returned but parsing
    or grounding rejected (logs text length so 0 vs ~900-char
    distinguishes "no content" from "model drift/hallucination")

Cost note documented inline: we deliberately do NOT cache the
failure with a sentinel under the v5 key. At temperature 0.4 the
next cron tick may roll a grounded output for the same prompt;
caching the failure would block legitimate retries. The L1→L2→L3
fallback in runSynthesisWithFallback handles user-visible
degradation; these logs handle ops visibility.

Tests: 91/91 brief-llm pass, typecheck clean, biome clean.

* fix(brief): PR koala73#3667 review round 4 cleanup — extract grounding helpers + fixture comment

Two reviewer-flagged cleanups:

- P2: extract `extractAnchors` / `tokensetOf` from inside
  checkLeadGrounding to file-level `extractAnchorTokens` /
  `groundingTokenSet`. Avoids closure re-instantiation per call,
  separates the two helpers cleanly, and makes them individually
  inspectable / unit-testable. Behaviour unchanged — same callers,
  same inputs, same outputs.

- P3: add a load-bearing comment on the `story()` test factory
  documenting that the default headline ("Iran threatens to close
  Strait of Hormuz...") is what grounds every `validJson` lead used
  by `generateDigestProse` / `generateDigestProsePublic` cache-shape
  tests. If a future contributor changes the default headline so it
  no longer mentions Iran/Hormuz, those tests would silently reject
  via the v5 grounding gate with cascading "expected truthy, got
  null" failures whose root cause is invisible. Comment names the
  invariant + the escape hatch (override headline + update
  validJson).

Tests: 91/91 brief-llm pass, typecheck clean, biome clean.

* fix(brief): PR koala73#3667 review round 5 — JSDoc attachment, blind-spot warn, stopword heuristic

Three reviewer findings:

#1 — JSDoc block was orphaned by the round-4 helper extraction.
  The big "Cheap content-grounding check..." doc sat at line 519,
  BEFORE extractAnchorTokens and groundingTokenSet — JSDoc parsers
  attach a doc block to the NEXT declaration, so the rich docs were
  describing the wrong function. Moved directly above
  checkLeadGrounding where they belong.

#2 — Lowercase-headline blind spot. If a feed ever produces all-
  lowercase or all-≤3-char headlines, extractAnchorTokens returns
  empty for every story, storyTokens.size === 0, and the gate
  silently returns true (skip). Added a console.warn gated on
  stories.length >= 3 so synthetic single-headline test corpora
  don't spam logs but real production feed regressions surface in
  cron output.

#3 — Stopword maintenance heuristic. Added a comment to
  GROUNDING_ANCHOR_STOPWORDS describing the detection rule: dump a
  week of real headlines, tokenise with stopwords disabled, count
  frequencies, inspect any token appearing in >~10% of headlines
  that isn't a known proper noun. The Prime/Chief/Cardinal gaps
  caught on rounds 2-3 would have surfaced from such a frequency
  audit. Captures the maintenance burden as an actionable signal
  rather than guesswork.

Tests: 91/91 brief-llm pass, typecheck clean, biome clean. The
new console.warn calls are intentionally surface in test output
when triggered (reviewer round 5 #4 — informational, no action).

… cleanup (koala73#3673)

* fix(brief): strip "Video:"/"Watch:" prefixes + match wire-name vs feed-name in suffix

Two real headline-noise issues observed in production briefs.

#1 — Editorial-format prefixes shipped to the magazine.

  May 12 brief, page 16/18:
    "Video: Philippine senator flees ICC arrest over role in drug war"

  The "Video: " tells the user nothing the magazine card body
  doesn't already convey (every card has its own source line and
  body block). Same shape applies to Watch / Live / Photos / Photo /
  Gallery / Listen / Podcast / Breaking / Exclusive / Opinion /
  Analysis / Update.

  New stripHeadlinePrefix helper handles all of the above with a
  REQUIRED trailing colon — preserves legitimate headlines that use
  one of these words as a noun ("Video game regulator fines...",
  "Watch list updated by sanctions office...", "Live broadcasts
  paused...").

#2 — Wire-name suffix not stripped when feed-name is the longer form.

  May 13 brief, story 5:
    "Putin says Russia will deploy new Sarmat nuclear missile this
     year - Reuters"
  primarySource: "Reuters World"

  stripHeadlineSuffix used strict equality: "reuters" !== "reuters
  world", so the suffix shipped. Fix is asymmetric word-boundary
  prefix-match: tail must be a SHORTER prefix of publisher (with a
  trailing space delimiter), e.g. tail "Reuters" against publisher
  "Reuters World" → strip. The inverse direction (publisher prefix
  of tail) is rejected ON PURPOSE — "Story - AP News analysis" with
  publisher "AP News" must NOT strip because "analysis" is editorial
  content extending the publisher name, not a desk-name suffix.

Both helpers wired into digestStoryToUpstreamTopStory in two-stage
order: prefix first, then suffix (handles "Video: Story - Al Jazeera"
correctly).

Test coverage:
  - REGRESSION (May 13 brief) for the Sarmat/Reuters/Reuters World case
  - REGRESSION (May 12 brief) for the Video: Philippine senator case
  - REJECTS-the-inverse coverage for AP News / AP News analysis
  - Word-boundary stem rejection ("reuter" / "Reuters", "iran" /
    "iranian press")
  - All 12 prefix variants individually tested

Tests: 69/69 brief-from-digest-stories pass (was 56), 8669/8669 full
unit suite pass, typecheck clean, biome clean.

* docs(brief): fix stale 'either direction' comment to reflect asymmetric one-directional match (PR koala73#3673 review)

The inline comment in stripHeadlineSuffix said 'word-boundary prefix
in either direction' but the actual implementation (and the
isPublisherWordPrefix docstring) is one-directional only: tail must be
a SHORTER prefix of publisher, never the reverse. The asymmetry is
intentional and load-bearing — it's what blocks editorial suffixes
like 'AP News analysis' from stripping when publisher is 'AP News'.

The 'either direction' wording was left over from round-1 of the same
PR's internal iteration. Updated comment names the asymmetry
explicitly and cross-references the full rationale on
isPublisherWordPrefix.

Comment-only change. Behaviour unchanged: 69/69 brief-from-digest
tests still pass, biome clean.

* fix(brief): prioritize critical topic clusters

* test(brief): lock ordering tie breakers

…ackoff + temp gate (koala73#3694)

* fix(seed-portwatch): rate-limit recovery — concurrency 12→6 + batch backoff + temporary gate

Option E from the 2026-05-14 incident triage: combined response to the
ongoing ArcGIS degradation where both direct AND proxy paths are
throttled.

Background (post-koala73#3676 + koala73#3681 deployed):
  Run #1 (19:55 UTC May 13): 24/30 proxy successes, gate fails 24<50
  Run #2 (00:03 UTC May 14):  5/30 proxy successes, gate fails 5<50

Both koala73#3676 (cold-fetch cap) and koala73#3681 (proxy fallback on timeout) are
working as designed. The remaining problem: Decodo proxy is now also
being rate-limited — successive runs degrade as our usage spikes the
proxy's bucket. ArcGIS itself appears degraded for this dataset.

Three coordinated changes:

(B) CONCURRENCY 12 → 6

Halve in-flight fetches so neither ArcGIS-direct nor Decodo-proxy
sees a 12-concurrent burst. Math at cold-fetch cap 30:
  5 batches × ~60s realistic + 4×5s backoff ≈ 320s
  5 batches × ~90s worst case + 4×5s backoff ≈ 470s
Both fit the 570s bundle budget.

(C) BATCH_BACKOFF_MS = 5_000 (new)

Sleep 5s between batches (skip on last, skip on signal-abort to
preserve SIGTERM responsiveness). Spaces out per-batch bursts so
neither service hits its rate-limit window from our run alone.
20s total added — negligible against the 570s budget.

(Temporary) MIN_VALID_COUNTRIES 50 → 25

Coverage gate lowered so partial-success runs (5-25 successful
fetches + stale-served from cache) can advance seed-meta. Pre-fix,
seed-meta was frozen at 2026-05-12 00:03 UTC for 60+ hours because
no run reached 50. The 25 floor is sized so a 5-success run still
fails (real outage) but a 25+ partial-success run advances meta and
clears the operator-facing WARNING.

Marked TEMPORARY in the inline comment with a 2026-05-20 review
target — must revert to 50 once ArcGIS recovers and runs reliably
exceed 50 again. Test enforces the comment shape so a future
reviewer can't silently normalise 25 as the new permanent floor.

Tests: 87/87 pass (was 85, +2 for BATCH_BACKOFF + MIN_VALID_COUNTRIES
invariants; +1 modification for CONCURRENCY value).

This week's portwatch series:
  koala73#3676  Structural cold-fetch cap (merged)
  koala73#3681  Proxy retry on timeout (merged)
  This   Rate-limit recovery (concurrency + backoff + gate)

* review: break loop on signal-abort + fix wrong PR number in test (P2×2)

Greptile PR koala73#3694 review:

P2 — Signal-abort skipped the sleep but kept the loop running.

Pre-fix: `if (batchIdx < batches && !signal?.aborted) await sleep(...)`
correctly skipped the 5s sleep on abort, but the for-loop continued
to the next iteration and started a fresh batch of up to 6 concurrent
fetches via withPerCountryTimeout (which creates its own AbortController
not linked to the parent signal). Net effect: the actual SIGTERM
backstop was onSigterm → process.exit(1), not the abort guard.

Fix: `if (signal?.aborted) break;` before the sleep — exits the loop
immediately so SIGTERM doesn't start additional in-flight work.

P2 — Test comment cited wrong PR number (koala73#3683 vs koala73#3694).

The "Halved from 12 → 6 on 2026-05-14 (PR koala73#3683)" comment in the
CONCURRENCY test referenced a non-existent PR in this series. Fixed
to koala73#3694 so a future reviewer following the comment lands on the
right diff.

Tests: 87/87 still pass. Updated the BATCH_BACKOFF_MS test to assert
both the new `break` shape and the simplified sleep condition; the
previous combined `batchIdx < batches && !signal?.aborted` pattern is
now split across the two checks.

* review: bypass 80% degradation guard in cap-mode (P1)

Greptile PR koala73#3694 round 3 P1: with the temp coverage gate lowered to 25,
a cap-mode partial-success run would CLEAR the coverage gate
(countryData.size ≥ 25) but STILL fail the unchanged degradation guard
(countryData.size < prevCount × 0.8 ≈ 139 in the incident state) —
seed-meta never advances, WARNING persists, PR's main recovery claim
broken.

Math at the current incident state:
  prevCount               174
  degradation threshold   0.8 × 174 = 139
  cap-mode realistic      30 cold-fetch + ~24 stale-served = ~54
  → 54 << 139 → DEGRADATION GUARD FAILS → extendTtl + return
  → seed-meta stays frozen, WARNING persists.

Fix shape: signal cap-mode from fetchAll() back to main() via a new
`capTriggered: bool` field plus `servedStaleCount` / `droppedTooOldCount`
/ `droppedNoCacheCount` counters. main() bypasses the 80% guard for
cap-mode runs (intentional partial coverage by design — see koala73#3676's
rotation contract) and logs a `PARTIAL PUBLISH (cap-mode)` line with
the fresh/stale split.

The bypass is scoped: non-cap runs (where needsFetch ≤ 30 and we
fetched all misses normally) STILL apply the 80% guard so silent
data-loss scenarios (ArcGIS regression silently drops 100 → 50
countries) remain caught.

Logic shape:
  if (!validateFn) → extendTtl + return                    [coverage gate]
  if (capTriggered) → log PARTIAL PUBLISH + fall through   [bypass]
  else if (countryData.size < prevCount × 0.8) → extendTtl + return [silent-loss guard]
  → publish + advance seed-meta

Tests (3 new): fetchAll-shape (returns the 4 new fields), bypass
shape (else-if structure ensures cap-mode skips the guard), bypass
exclusivity (non-cap runs still enforce the guard with exactly 2
references to `prevCount × 0.8` — one in condition, one in error msg).
90/90 pass (was 87).

…e retries (koala73#3701)

* fix(seed-portwatch): surface degraded ArcGIS 400 body + cap degenerate retries

ArcGIS Daily_Ports_Data, during degradation episodes, returns HTTP 200
with a 400 error body (`Cannot perform query. Invalid query parameters.`)
after 30-56s of server processing. Our 45s FETCH_TIMEOUT fires before
the body lands, callers see AbortError, the existing circuit-breaker
that pattern-matches the error message never fires, and the seeder
treats every degraded country as a generic timeout (then a doomed proxy
retry, then another doomed first-direct-retry from the one-shot
fetchWithRetryOnInvalidParams).

Two surgical fixes:

1. **Diagnostic body-capture on timeout.** When the direct fetch
   hits its FETCH_TIMEOUT (not a caller-signal abort), make ONE extra
   fetch with +20s budget purely to read the response body. If it
   contains `body.error`, throw with the real ArcGIS message so the
   upstream circuit-breaker can act on the upstream-degradation signal.
   Gated to fire once per process — the proxy budget Greptile P2'd on
   PR koala73#3681 stays intact for subsequent timeouts.

2. **Bail-don't-retry threshold on Invalid-query-parameters.** Preserve
   the legit one-shot retry for transient single-call flakes (2026-04-20
   BRA/IDN/NGA pattern), but cap total retries-on-this-error per
   process at 5. Beyond that, throw a clear `ArcGIS degraded — N errors`
   message so the operator sees the regression class instead of N
   identical retries × 45s burning the 540s container budget.

Investigation that drove this: direct curl probes against
services9.arcgis.com revealed (a) HTTP 200 with a 400 error body, (b)
non-deterministic results across attempts within minutes (same query,
ERR → OK → ERR → 504), (c) WHERE-clause reshaping helps unevenly but
never reliably. The right frame is "upstream is degraded, surface it
and protect the container budget" not "we're being rate-limited."

The existing cap-mode + stale-serve + temp-gate-25 logic (PRs
koala73#3676/koala73#3681/koala73#3694) already rides through partial coverage — these
changes just make sure we don't waste the container budget while
ArcGIS recovers, and that the next diagnostic round has the real error
message instead of generic AbortError.

Tests:
- 3 new structural pattern-match tests covering ERROR_BODY_CAPTURE_EXTRA_MS,
  _captureErrorBodyAfterTimeout helper, once-per-run gating,
  INVALID_PARAMS_RETRY_THRESHOLD, counter increment + threshold check,
  reset helpers.
- 96/96 tests pass (was 93 before).

* review: gate cap-mode bypass on fresh upstream contact + abort-aware backoff sleep (P1+P2)

P1 — Cap-mode bypass could publish a stale-only canonical list as healthy.

Pre-fix the capTriggered bypass of the 80% degradation guard was
unconditional. With the lowered MIN_VALID_COUNTRIES=25 + cap-mode +
servedStale entries seeding countryData, a run with all-stale entries
(freshFetched=0, cacheHits=0, servedStale=27, dropped=147) could:
  - Pass validateFn (countryData.size >= 25 — all from stale-cache)
  - Bypass the 80% guard (capTriggered=true)
  - Shrink the canonical list from ~174 → 27 stale-only entries
  - Advance seed-meta, clearing the operator-facing WARNING

Fix: gate the bypass on freshFetchedCount + cacheHitCount ≥
MIN_FRESH_FETCH_FOR_CAP_BYPASS (5). Below that floor, cap-mode is NOT
rotational steady-state — it's an ArcGIS-completely-down scenario, and
the run falls through to the 80% guard (which extendExistingTtl-only,
preserves canonical list, keeps WARNING visible). New log line
"CAP-MODE BYPASS REFUSED: only N fresh upstream contacts" makes the
refusal observable to operators.

This composes with the INVALID_PARAMS_RETRY_THRESHOLD=5 fix earlier in
this PR — that change increases the rate at which cold fetches return
errors during upstream degradation, making the stale-only scenario
this P1 guards against MORE likely, not less.

P2 — Inter-batch backoff sleep is now abort-aware.

The 5s BATCH_BACKOFF_MS wait used a plain setTimeout that ignored the
caller signal mid-sleep, so a SIGTERM during the wait still forced the
full 5s before observing the abort. Now races the setTimeout against
signal's abort event so the loop exits immediately on a real
cancellation (the signal?.aborted check at the top of the next
iteration then `break`s the loop). Cosmetic change but matches the
PR-body intent that the loop is SIGTERM-responsive.

Tests:
- 2 new structural pattern-match tests covering MIN_FRESH_FETCH_FOR_CAP_BYPASS,
  freshFetchedCount counter, upstream-contact gate, CAP-MODE BYPASS REFUSED
  branch, and the abort-aware sleep shape.
- 4 existing tests updated to match the new destructure shape + new prevCount
  × 0.8 reference count (now 4: 2 conditions + 2 Math.ceil sites, one set per
  guard branch).
- 95/95 portwatch tests pass; 8804/8805 npm run test:data (1 flake is the
  pre-existing scripts/_bundle-fixture-env-*.mjs race between
  bundle-runner.test.mjs and news-classify-cache-prefix-audit.test.mjs,
  unrelated to this change).

* review: gate body-capture on success not attempt + short-circuit proxy-confirmed errors (P2×2)

Two Greptile P2 comments on PR koala73#3701 review round 2.

P2 #1 — Body-capture gate fired on first ATTEMPT, not first SUCCESS.

During consistent degradation, the 20s ERROR_BODY_CAPTURE_EXTRA_MS
window isn't long enough — ArcGIS needs 30-56s of server processing
per query from connection start. A failed first capture (capture also
timing out) flipped the once-per-run flag, locking out every
subsequent timing-out country from ever capturing the body. The
diagnostic value could be lost for an entire run, defeating the whole
point of the capture path.

Fix: track SUCCESS count (cap at 1 — we only need the body shape once)
AND ATTEMPT count (cap at 3 to bound total cost). Null captures consume
an attempt but not a success, so subsequent timing-out countries keep
trying. Worst-case cost: 3 × 20s wall-clock per run, still within
PER_COUNTRY_TIMEOUT_MS=90s caps.

P2 #2 — Proxy-confirmed "Invalid query parameters" still triggered retry.

When arcgisProxyRetry returns an error body, the thrown message has
the `(via proxy after ${reason})` prefix and still matches
`/Invalid query parameters/i`. fetchWithRetryOnInvalidParams counted
it, then retried via fetchWithTimeout — which would timeout → proxy →
hit the same authoritative error, burning a full direct+proxy cycle
for nothing. The proxy response IS the definitive upstream signal;
retrying is meaningless.

Fix: counter still increments (proxy-confirmed errors are valid
degradation signals for the threshold), but short-circuit the retry
with `if (/via proxy after/i.test(msg)) throw err`.

Tests: 2 new structural pattern-match tests + 1 existing test relaxed
to accept the new `if (captured?.error) { successCount++; throw ... }`
shape. 96/96 portwatch tests pass; 8806/8806 npm run test:data
(unrelated bundle-fixture race cleared this run).

* review: threshold-before-short-circuit + realistic capture budget (P1+P2 round 3)

P1 — Proxy-confirmed Invalid Params never reached the degradation threshold.

Pre-fix the proxy-confirmed short-circuit (`/via proxy after/i` → throw)
ran BEFORE the threshold check. So during the actual incident — where
nearly every error arrives via the proxy fallback path — every error
threw the per-country proxy message and the clean
`ArcGIS degraded — N 'Invalid query parameters' errors` message was
unreachable.

Fix: reorder so threshold check fires first, then proxy short-circuit.
Counter still increments for proxy-confirmed errors (they contribute to
the threshold), and once the threshold is exceeded the degraded
message surfaces regardless of whether the error came from direct or
proxy.

P2 — 20s capture budget didn't realistically catch 30-56s responses.

The 3-attempts-× 20s fix was bounded but each attempt was a FRESH
request starting from t=0 (the original 45s direct fetch was already
aborted). For the observed degraded response class (30-56s server-side
from request start), each 20s capture aborts before the body lands.

Fix: bump ERROR_BODY_CAPTURE_EXTRA_MS 20s → 40s. Math under
PER_COUNTRY_TIMEOUT_MS=90s:
  direct (45s timeout) + capture (40s budget) = 85s, leaving 5s before
  the per-country wrap fires. Proxy retry effectively dies for
  timing-out countries in this mode — accepted tradeoff because the
  proxy is itself degraded (Decodo throttled per koala73#3694 history), so it
  wasn't helping anyway. Attempt cap stays at 3 (across the run, not
  per country) so the diagnostic value is captured by 3rd timing-out
  country; subsequent countries go straight to proxy.

Tests: 1 new ordering test using src.search() index comparison to
assert threshold-throw-before-proxy-short-circuit; 2 existing tests
updated for the bumped constant and the longer span between
increment and short-circuit. 97/97 portwatch tests pass; 8806/8807
npm run test:data (1 flake = pre-existing _bundle-fixture-* race
between bundle-runner and news-classify-cache-prefix-audit).

…News placeholders (koala73#3717)

* fix(feeds): address koala73#3715 review — server parity, 0-item placeholders, parity guard

Reviewer P1 #1: I only updated the CLIENT feed config in koala73#3715; server-side
_feeds.ts:263 still pointed Blockworks at the dead blockworks.co/feed, so
the digest path (loadNewsCategory → /api/news/v1/list-feed-digest) kept
hitting the Cloudflare-blocked upstream and caching zero items. Same
mirror-drift class as PR koala73#3712's allowlist miss; recorded in
feedback_implement_every_site_my_own_diagnosis_enumerated.

Reviewer P1 #2: I validated with `grep -c '<item>'` (counts LINES, not
occurrences) and called the URLs "fine". Re-counted with
`grep -o '<item>' | wc -l`:

- Kitco News (50), Kitco Gold (50), Mining Weekly (100), FX Empire Gold
  (76), Mining Journal (19) — all real volume, fine.
- Blockworks: 0 items across every probe (site: + time-window variants).
  Google doesn't index blockworks.co for News (Cloudflare likely blocks
  Googlebot on the same wholesale tier).
- Commodity Trade Mantra: 0 items via site:, 2 items via "" / 30d.
  Effectively unindexed.

Fix: REMOVE both feeds rather than ship silent placeholders.

- Blockworks (client + server): The Block already exists on both sides
  and covers the same institutional-crypto territory.
- Commodity Trade Mantra (client): commodity-news still has Mining.com /
  Bloomberg / Reuters / S&P Global / CNBC — coverage isn't lost.

New parity test (tests/feeds-client-server-parity.test.mjs) fails when a
feed NAME appears on both sides with inconsistent routing (one side
Google News, the other direct). It exposed 12 PRE-EXISTING drifts (each
its own per-feed judgment) — grandfathered as KNOWN_DRIFTS so this PR
doesn't block on unrelated tech debt. New drift fails the test. Locked-in
regression: neither file may re-add `https://blockworks.co/feed`.

* fix(feeds): remove server-side Commodity Trade Mantra + extend parity test

Reviewer caught a P1 on koala73#3717: I removed Commodity Trade Mantra from the
CLIENT in the first commit of this PR but missed the SERVER copy at
server/worldmonitor/news/v1/_feeds.ts:341. The failure mode the reviewer
described is real:

  1. Server fetches 7 feeds for commodity-news, including CTM
  2. Server ranks by importanceScore, then .slice(0, MAX_ITEMS_PER_CATEGORY)
     (list-feed-digest.ts:1076-1082) — CTM items COUNT toward the cap
  3. Client filters by `enabledNames` (data-loader.ts:908-914) — CTM not in
     client config → CTM items DROPPED
  4. Net: invisible items crowd out visible ones, shrinking the panel

Two changes:

1. Server entry removed with a comment naming the exact failure mode.

2. tests/feeds-client-server-parity.test.mjs extended with:
   - Fixed extractor: the previous single-line regex missed ~46 multiline
     locale-keyed entries on the client side (France 24, EuroNews, DW News,
     etc.), which would have made the new orphan check fire false positives.
     New extractor scans up to 600 chars forward from each `name:` for the
     matching `url:`, capturing locale-object URLs intact.
   - New test: `no NEW server-only feed entries`. Server-only orphans are
     the exact crowd-out failure mode the reviewer caught. Five existing
     orphans grandfathered (Trump-Truth-Social, White House Actions, First
     Round Review, YC News, YC Blog) — each should be reviewed and either
     mirrored on the client OR removed from server. Set should SHRINK.
   - New regression: CTM specifically must not reappear on the server.
   - DW News added to KNOWN_DRIFTS: client uses mixed direct/Google-News
     per locale, server uses pure direct; classifier treats any-locale-GN
     as GN. Worth reconciling separately, not in this PR.

All 5 parity tests pass; typecheck + biome clean.

…he panel can't hang forever (koala73#3718)

* fix(daily-market-brief): cap LLM summarizer + total build budget so the panel can't hang forever

## Symptom
PRO users were reporting the Daily Market Brief panel stuck on "Building
daily market brief..." indefinitely after a cache miss. The try/catch
around the summarizer was decorative — it only handles REJECTIONS, and a
hung upstream (LLM provider slow, Vercel cold-start + slow network) leaves
the awaited promise pending forever, never reaching catch.

## Verified root cause (no extrapolation this time)

Traced the chain top-to-bottom for an enforced timeout:

  loadDailyMarketBrief                   data-loader.ts:1635
   → buildDailyMarketBrief               daily-market-brief.ts:381
     → generateSummary                   summarization.ts:197
       → summaryResultBreaker.execute    no timeoutMs option (line 20)
         → tryApiProvider                summarization.ts:76
           → summaryBreaker.execute      no timeoutMs option
             → newsClient.summarizeArticle(...)   accepts options.signal
                                                   but no caller passes one

`vercel.json` has no `maxDuration` override; the default function timeout
is whatever Vercel applies, but the CLIENT awaits TCP keepalive, so from
the browser's perspective the fetch can be effectively forever.

## Fix

`src/utils/with-timeout.ts` — new utility. Races a promise with a
deadline; the source promise is not cancelled (JS has no general cancel)
but the await unblocks via TimeoutError so caller fallback paths fire.

`src/services/daily-market-brief.ts:432` — wrap the
`summaryProvider(...)` call in `withTimeout(..., 45_000, 'daily-brief-
summary')`. On timeout the EXISTING catch (line 444) falls back to the
rules-based summary (`buildRuleSummary`) — already pre-computed at the
top of the function, so this costs nothing extra. Added
`summarizerTimeoutMs?` to `BuildDailyMarketBriefOptions` so tests can
exercise the fallback in ~30ms without waiting 45s.

`src/app/data-loader.ts:1644` — wrap `getCachedDailyMarketBrief(...)` in
a 3s `withTimeout` + `.catch(() => null)`. A hung persistent-cache layer
can't keep the panel on its default Loading state — fall through to
"build from scratch" instead.

`src/app/data-loader.ts:1667` — wrap `buildDailyMarketBrief(...)` in a
60s `withTimeout` (outer budget). Inner summarizer has its own 45s cap;
this catches the dynamic-import path (`getDefaultSummarizer()` import
hanging on a slow chunk fetch), `_collectRegimeContext` /
`_collectYieldCurveContext` / `_collectSectorContext` exotic hangs
(though they're already in `Promise.allSettled`), etc. The existing
catch (line 1693) serves the cached version or shows an error.

## Test plan
- `tests/with-timeout.test.mjs` — 7 unit tests: resolve-first, reject-
  first, hang→TimeoutError, timer cleanup (proves a 5s-budget test
  exits at 5ms not 5s), onTimeout-once, onTimeout-not-called-on-success,
  onTimeout-throw-doesn't-hijack.
- `tests/daily-market-brief.test.mts` — new regression test
  `a hanging summarizer must not stall the brief` reproduces the prod
  shape with an injected `() => new Promise(() => {})` (pending forever)
  and asserts the brief returns within the timeout with
  `provider: 'rules'` and `fallback: true`. Test elapsed ~33ms.

Local gate: typecheck PASS, biome on touched files PASS (4 pre-existing
warnings on data-loader.ts unrelated complexity scores, present on main
too), lint:api-contract PASS.

* fix(daily-market-brief): close Greptile koala73#3718 review gaps

## P2 (Greptile, valid)

The original PR wrapped the upfront cache read in withTimeout but I
missed the recovery cache read inside the catch block at
`data-loader.ts:1715` — same hang risk in the exact path the fix
creates. If the outer 60s build budget fires AND IndexedDB is also
degraded, the recovery read keeps the panel stuck.

Wrap the recovery `getCachedDailyMarketBrief(...)` in the same 3s
`withTimeout` (label `'daily-brief-cache-read-recovery'` so logs
distinguish the two cache reads) + the existing `.catch(() => null)`
absorbs both the TimeoutError and any persistent-cache failure into
the null-result branch that the showError fallback already handles.

## Nits

- Renamed `tests/with-timeout.test.mjs` → `.mts` to match the project
  convention for tests that directly import `.ts` sources (see
  `daily-market-brief.test.mts`). Functionally identical; lines up
  with the rest of the test directory.

- Added the missing test case Greptile flagged: assert `onTimeout` is
  NOT invoked when the source REJECTS first (the prior set only covered
  the resolve-first path). Without `.finally(clearTimeout)` the timer
  would still fire after `Promise.race` already settled — onTimeout
  would run as a phantom side-effect after the caller had already moved
  on. The test waits 80ms past the 50ms budget to prove the timer was
  cleared.

All 8 `withTimeout` cases pass; the daily-market-brief regression test
still falls back within budget; typecheck + biome clean.

* fix(daily-market-brief): close two more hang paths (koala73#3718 review round 2)

Both findings valid — and embarrassing, because they're the same hang
class this PR was opened to fix. The earlier passes wrapped the LLM
summarizer call (the obvious suspect) but missed three more unbounded
awaits in the same function.

## P1 #1 — Context collectors hung before the 60s envelope started

`_collectRegimeContext` calls `client.getFearGreedIndex({})` and
`_collectYieldCurveContext` calls `client.getFredSeriesBatch(...)` —
both unbounded RPCs. `Promise.allSettled` only converts REJECTIONS into
status:rejected; it waits forever for pending-forever promises. My
withTimeout envelope was wrapped around `buildDailyMarketBrief(...)`
AFTER the allSettled line, so a hung context collector kept the panel
on "Building daily market brief..." forever — exactly the symptom this
PR was supposed to fix.

Wrap each RPC-using collector in `withTimeout(..., 8_000)` with its own
label. 8s is generous for an RPC and leaves >36s of the outer 60s
budget for the actual LLM call. `_collectSectorContext` is sync (only
reads hydrated data) so it needs no wrap; allSettled accepts non-
promises directly.

## P1 #2 — Cache write blocked render

`await cacheDailyMarketBrief(brief)` ran BEFORE the render call, so a
hung IndexedDB / Tauri-Store write meant the user never saw the
finished brief even though it was sitting in memory ready to display.
The build budget proved nothing by itself.

Render first, persist after. Cache write is now fire-and-forget with
its own 5s budget (`void withTimeout(...).catch(...)`) — a hung backend
becomes "no warmup for tomorrow's load" instead of "panel stuck on
Building forever."

## On the catch-path cache read

The reviewer also flagged the catch-block `getCachedDailyMarketBrief`
call as unbounded; my previous commit (Greptile P2 fix) already wrapped
that one with `withTimeout(..., 3_000, 'daily-brief-cache-read-recovery')`.
Verified still present in this round.

Local gate: typecheck PASS, biome on data-loader.ts PASS (4 pre-
existing complexity warnings unrelated to this diff), with-timeout
suite 8/8, daily-market-brief regression still passes.

…diness refresh (koala73#3833)

* fix(map): swallow deck.gl/maplibre interleaved-mode render race

Module-scoped installDeckInterleavedRaceFilter() adds a one-shot
capture-phase window error listener that suppresses
  TypeError: Cannot read properties of null (reading 'id')
when the throwing file matches /deck-stack-…\.js/. Called from
initDeck() — idempotent across recreateWithFallback / HMR.

Trigger is the deck.gl 9.x + maplibre-gl 5.x interleaved-mode race:
setProps({layers}) finalizes a layer while maplibre's painter is
mid-render, so the painter callback iterates a list that just had
a slot nulled. MapboxOverlay.onError can't see this — maplibre, not
deck, owns the throwing callstack.

Sentry's beforeSend in main.ts:313-315 already drops this pattern
for telemetry, so impact is purely console-noise removal. First-
party .id crashes still surface because the filter requires BOTH
the message shape AND the deck-stack chunk filename.

* fix(panels): gate tech-readiness refresh by viewport on non-happy variants

Replace `if (SITE_VARIANT !== 'happy')` with `… && shouldLoad('tech-readiness')`,
matching the thermal-escalation gate one line below.

Why this fixes the energy/finance/commodity-variant 5s timeout:
App.ts:576-583 merges ALL_PANELS into panelSettings on every variant
for cross-variant pref carryover, so shouldCreatePanel('tech-readiness')
returns true everywhere — but the bootstrap seed key only exists on
full + tech variants, so the unconditional refresh() at
services/economic/index.ts:694 times out on every other variant's
data-loader cycle. Viewport-gating prevents the refresh fan-out from
firing on variants whose panel will never be visible.

* fix(panels): close two boot-path bypasses on tech-readiness variant gate

Reviewer found that PR koala73#3833's first cut was incomplete — the
`shouldLoad('tech-readiness')` gate I added in data-loader was
silently bypassed on initial boot, and a parallel auto-refresh path
in panel-layout was never gated at all.

P1 #1 — data-loader.ts:602
  `shouldLoad(id)` returns `forceAll || isPanelNearViewport(id)` at
  data-loader.ts:444. App.ts:1226 calls `loadAllData(true)` on startup,
  forcing `shouldLoad` to true on every variant — so commodity/finance/
  energy were still enqueueing `TechReadinessPanel.refresh()` at boot.

P1 #2 — panel-layout.ts:1278
  The `lazyPanel('tech-readiness', ...)` factory calls `void p.refresh()`
  unconditionally inside the dynamic import. Because App.ts:577-583
  merges ALL_PANELS into panelSettings on every variant for cross-variant
  pref carryover, `shouldCreatePanel('tech-readiness')` (just a key-
  existence check at panel-layout.ts:854) is true everywhere — and the
  hide-disabled path at panel-layout.ts:2028-2030 runs AFTER the import's
  refresh has already fired the 5s `/api/bootstrap?keys=techReadiness`
  fetch. So the variant gate has to live inside the factory, not after.

Fix — introduce `isPanelInVariantDefaults(key)` helper in src/config/panels.ts
that returns `VARIANT_DEFAULTS[SITE_VARIANT].includes(key)`. Use it in
both auto-refresh paths:

  data-loader.ts:602
    if (isPanelInVariantDefaults('tech-readiness') && shouldLoad('tech-readiness'))

  panel-layout.ts:1278 (factory body)
    if (isPanelInVariantDefaults('tech-readiness')) {
      void p.refresh();
    }

The panel is still CREATED on all variants so users who opt-in via
settings can still see and use it — but the eager fetch only fires
where the bootstrap seeder actually populates the key (full + tech).

Regression test — tests/tech-readiness-variant-gate.test.mts uses a
line-walker (not a strict regex) to assert:
  1. data-loader's techReadiness task is gated by isPanelInVariantDefaults
  2. panel-layout's lazyPanel factory wraps p.refresh() in the same gate
  3. the helper is exported from the @/config barrel

Mutation-tested locally: reverting either gate makes the test fail
with a clear diagnostic naming the bypassed location.

Note — national-debt at panel-layout.ts:1286 has the same factory
shape (eager `void p.refresh()` on a variant-restricted panel that's
only in FULL_PANELS). Left for a follow-up so this PR stays scoped
to the reviewer's findings.

…+ lock down (koala73#3832)

* fix(route-explorer): close Asia→DE lane gap for TW/KR/JP/VN/TH/PH/IN + lock down

PR koala73#3828 fixed HK→Germany by adding `china-europe-suez` and `asia-europe-cape`
to HK's `nearestRouteIds` in `scripts/shared/country-port-clusters.json`, and
explicitly deferred TW/KR/JP/VN/TH/PH because each needed review against actual
shipping data. While triaging a reporter's HK→DE / Premium Stock / WM Analyst
empty-state screenshots (pr-3718 session), I confirmed those six countries plus
IN still fail with `noModeledLane: true` against DE — server reports the gap
honestly, UI shows "No modeled lane for this pair."

Root cause for each is identical to HK's: the country has only
Pacific/intra-Asia/Gulf-oil route IDs, none of which appear in DE's cluster
(`china-europe-suez`, `asia-europe-cape`, `transatlantic`). Server intersection
at server/worldmonitor/supply-chain/v1/get-route-explorer-lane.ts:233-234
returns zero shared routes → noModeledLane=true.

Fix: add `china-europe-suez` and `asia-europe-cape` (the trunk Asia↔Europe
container routes terminating at Rotterdam) to TW, KR, JP, VN, TH, PH, IN.
Both routes are tagged `category: 'container', status: 'active'` in
src/config/trade-routes.ts; they are factually correct for every major Asian
export hub shipping to Northern Europe.

Lock-down: tests/country-port-clusters-asia-europe-lane.test.mts (3 cases)

  1. Data invariant — every Asian port country in {CN,HK,TW,JP,KR,SG,MY,ID,
     TH,VN,PH,IN} must have a non-empty `nearestRouteIds` intersection with
     DE in the cluster JSON. Reverting any of the seven new entries flips
     this test red with a clear "add china-europe-suez and/or asia-europe-
     cape to <ISO2>'s nearestRouteIds" error.
  2. Computed lane — `computeLane('HK','DE','85','container')` must return
     `noModeledLane: false` and a non-empty `primaryRouteId`. This is the
     EXACT shape of the reporter's screenshot (HK→Germany, Electrical &
     Electronics HS2=85, Container, AUTO badge); the original PR koala73#3828 had
     no test pinning this case, so a future contributor reverting the HK
     entry would not be caught.
  3. Full Asian-port matrix — same `noModeledLane: false` assertion for
     all 12 Asian export hubs against DE, defending the algorithm boundary
     (computeLane) on top of the data invariant.

Bite-test: reverting just the data change (git stash on the JSON) flips
tests #1 and #3 red, naming the exact 7 offenders (TW/JP/KR/TH/VN/PH/IN);
test #2 stays green because HK was already fixed by koala73#3828. Confirms each
assertion bites its intended regression.

The 30-query smoke matrix in tests/route-explorer-lane.test.mts (38 cases)
stays 38/38 green. tsc clean.

This complements PR koala73#3828's HK fix and clears the deferred-follow-up note
without touching unrelated areas. The reporter's third screenshot (HK→DE
"No modeled lane") will resolve on next deploy regardless of this PR
(PR koala73#3828 already fixed HK); this PR additionally prevents TW/KR/JP/VN/TH/
PH/IN from showing the same dead-end to other Asia-export-hub users, and
makes both fixes regression-proof.

* fix(route-explorer): address PR koala73#3832 review P1 — IN→DE picks india-europe, not china-europe-suez

Reviewer (correct): adding `china-europe-suez` + `asia-europe-cape` to IN
satisfied `noModeledLane === false` for IN→DE but pushed the resolver to
pick `china-europe-suez` (Shanghai → Rotterdam) as the primary route. The
Route Explorer UI highlights the route's from→to ports, so an Indian
shipment to Germany rendered a Shanghai origin port. The test was vacuous
on this dimension — it only asserted `noModeledLane === false`, never
which route resolved.

Root cause of my original choice: DE's `nearestRouteIds` didn't list
`india-europe`, so no intersection existed for IN→DE via that route. I
papered over the missing intersection by polluting IN's profile with
China-origin trunk routes instead of fixing DE's profile.

Correct fix (follows existing convention): every European country that
already has `china-europe-suez` (the Asia-Europe trunk to Rotterdam) also
gets `india-europe` (the India-Europe trunk to Rotterdam). Both routes
terminate at the same Northern-European hub; the data model treats trunk
routes as broadly serving any European port country. Applied uniformly to
DE, GB, FR, IT, ES, NL, BE, PL, SE, DK, FI, GR, PT (the 13 European
countries with `china-europe-suez`).

IN's `nearestRouteIds` reverts to `["india-europe", "india-se-asia",
"gulf-asia-oil"]` — no more China-origin pollution. IN→DE now intersects
on `india-europe` (only shared route), so the resolver picks it
unambiguously.

Test tightening (the actual lock the reviewer asked for):
`tests/country-port-clusters-asia-europe-lane.test.mts` adds an
`EXPECTED_PRIMARY_ROUTE_TO_DE` map and asserts `computeLane(<iso2>, 'DE',
'85', 'container').primaryRouteId === expected` for all 12 Asian-port
countries. IN→DE must equal `'india-europe'`; all others must equal
`'china-europe-suez'`. Removed the weaker "noModeledLane: false for the
matrix" test — the stricter primaryRouteId assertion subsumes it (a
`noModeledLane=true` response fails the equality check with a clear
message naming the offender).

Bite-test of the new assertion:
  - Revert `india-europe` from DE only → both tests #1 and #3 flip red
    naming IN: data invariant says "IN → DE: shared routes = []", lane
    assertion says "IN → DE: noModeledLane=true (expected
    primaryRouteId='india-europe')". Confirms the assertion bites the
    exact reviewer-flagged regression.
  - HK→DE remains green (china-europe-suez is correct for an HK origin).
  - 38/38 of the existing `tests/route-explorer-lane.test.mts` smoke
    matrix stays green. tsc clean.

The other Asian origins (TW/JP/KR/VN/TH/PH/SG/MY/ID) already correctly
intersect DE on `china-europe-suez` (added in this PR's first commit) —
that's the trunk route their containers actually take to Europe, so
their primaryRouteId assertion was already correct.

* test(route-explorer): drop redundant HK-specific case (PR koala73#3832 review P2)

Greptile P2: the standalone HK → DE test was fully subsumed by the matrix
test, which iterates over ASIAN_PORT_COUNTRIES (includes HK at index 1)
and asserts `primaryRouteId === EXPECTED_PRIMARY_ROUTE_TO_DE[iso2]`.
HK's expected value is 'china-europe-suez'; a regression on HK fails the
equality check with a clear message, just like every other origin.

Renamed the surviving matrix test to call out HK→DE explicitly so the
provenance trail to the pr-3718 reporter symptom isn't lost.

…9 regression) (koala73#3835)

The 2026-05-19 Pro brief shipped "How nuclear war would impact the
global food system" from Bulletin of Atomic Scientists as CRITICAL
story #6, sitting alongside breaking news about the Iran-Israel war
and the WHO Ebola declaration. The brief promises event-driven
intelligence — a Bulletin analysis essay is not an event.

The existing classifier missed it on all three current signals:
- STRONG #1 (URL section /opinion/): Bulletin URLs have no opinion-
  style path because the WHOLE SITE is commentary, no hard-news
  section to distinguish from.
- STRONG #2 (headline prefix "Opinion:"): hard-news-shaped title.
- CORROBORATING (quote-wrap + columnist framing): description reads
  like a news lede.

The signal IS the publisher. Add STRONG #3: a hand-curated allowlist
of commentary-only publishers, matched by hostname (suffix-anchored to
permit `newsletter.<host>` / `m.<host>` while rejecting typo-domains
like `evilthebulletin.org`).

Initial list (per docs/plans/2026-05-19-001 U1):
- thebulletin.org      — Bulletin of Atomic Scientists
- project-syndicate.org — op-eds from world leaders / academics
- foreignaffairs.com   — CFR's analysis quarterly
- foreignpolicy.com    — Foreign Policy magazine
- warontherocks.com    — defense analysis blog

Maintenance commitment: quarterly review against `droppedOpinion`
telemetry to catch (a) new commentary publishers to add, (b) listed
publishers that launched a hard-news section. Rollback path is
remove-from-Set, never per-URL exceptions (cruft).

Pattern mirrors the existing safePathname/STRONG_URL_SEGMENTS shape;
new safeHostname helper parses URL().hostname so the match is on
the parsed host (not raw .includes() on the link string — closes the
tracking-param injection vector documented in PR koala73#3748).

7 new test cases cover the May 19 regression, the 4 other publishers,
subdomain matching, typo-domain rejection, tracking-param / fragment
spoofing, malformed URLs, and the "allowlisted host PLUS hard-news
content → still opinion" rule.

This is PR-1 of the 4-PR Phase 1 wave from
docs/plans/2026-05-19-001-fix-brief-content-quality-regressions-plan.md.