…ing-for-stocks

Batch market and RSS fetching with progressive updates

… freshness

Assessment & Documentation:
- Add docs/GEOPOLITICAL_ASSESSMENT.md with full platform analysis
- Strategic improvement roadmap with prioritized recommendations

Quick Win #1 - Data Freshness (intelligence gaps):
- Add getIntelligenceGaps() and getIntelligenceGapSummary() to data-freshness.ts
- Add human-readable messages explaining what analysts CAN'T see
- Add hasCriticalGaps() for alert integration

Quick Win #2 - Escalation Scores:
- Add escalationScore (1-5), escalationTrend, escalationIndicators to Hotspot type
- Update 11 major hotspots with scores (Sahel, Haiti, Horn of Africa, Moscow,
  Beijing, Kyiv, Taipei, Tehran, Tel Aviv, Pyongyang, Sana'a)

Quick Win #3 - Signal Context ("Why It Matters"):
- Add SIGNAL_CONTEXT with whyItMatters, actionableInsight, confidenceNote
- Add getSignalContext() helper for all 10 signal types
- Explains analytical significance of each signal type

Quick Win #4 - Historical Context:
- Add HistoricalContext interface with lastMajorEvent, precedentCount,
  cyclicalRisk fields
- Add whyItMatters field to Hotspot type
- Update major hotspots with historical precedents and geopolitical significance

Quick Win #5 - Propaganda Risk Flags:
- Add PropagandaRisk type and SourceRiskProfile interface
- Add SOURCE_PROPAGANDA_RISK mapping for state media (Xinhua, TASS, RT, CGTN)
- Add getSourcePropagandaRisk() and isStateAffiliatedSource() helpers
- Flag medium-risk state-affiliated sources (Al Jazeera, France 24, DW, etc.)

…koala73#2976)

* feat(intelligence): Mobility v1 population (Phase 2 PR2)

Phase 2 PR2 of the Regional Intelligence Model. Replaces the Phase 0
empty `mobility` stub with a real MobilityState built from existing
Redis inputs — no new seeders, no new API keys, no new dependencies.

## New module: scripts/regional-snapshot/mobility.mjs

Single public entry point:

  buildMobilityState(regionId, sources) -> MobilityState

Pure function. Never throws. Wrapped in a defensive try/catch that
returns the empty shape on any unexpected builder bug so snapshot
persist can never be blocked by mobility data issues.

## Data sources (all already in the repo)

- aviation:delays:faa:v1       — US airports (FAA ASWS, Vercel seeder)
- aviation:delays:intl:v3      — ~51 non-US airports across Europe/APAC/
                                 MENA/Africa/Americas (AviationStack,
                                 Railway ais-relay, 30min refresh)
- aviation:notam:closures:v2   — global ICAO NOTAM closures (FAA-side
                                 seeder hits ICAO API with monitored
                                 ICAO list spanning all regions)
- intelligence:gpsjam:v2       — global GPS jamming hexes (Wingbits,
                                 fetch-gpsjam.mjs)
- military:flights:v1          — global military ADSB tracks (OpenSky,
                                 seed-military-flights.mjs)

## Per-field output mapping

| Field                 | Source                          | Coverage      |
|-----------------------|--------------------------------|---------------|
| airspace[]            | GPS-jam hexes aggregated 1:region | All regions  |
| flight_corridors[]    | (empty in v1)                   | -             |
| airports[]            | FAA + intl delays, severity ≥ MAJOR | All regions |
| reroute_intensity     | clip(military count / 50, 0, 1) | All regions   |
| notam_closures[]      | NOTAM reasons for region airports | All regions  |

## Region classification helpers

Three pure classifiers handle the three different region conventions
the source data uses:

  airportToSnapshotRegion(alert) — splits AviationStack AIRPORT_REGION_*
    by country so americas → {north-america, latam} and apac →
    {south-asia, east-asia}

  gpsjamRegionToSnapshotRegion(label) — maps the 17 fetch-gpsjam.mjs
    classifyRegion labels (iran-iraq, levant, ukraine-russia, etc.) to
    the 7 snapshot regions. Turkey/Caucasus intentionally lands in MENA
    per the geography.js WB override.

  latLonToSnapshotRegion(lat, lon) — bbox classifier for military
    flights. Coarse but shippable; v2 could use h3/geometry.

## Airspace aggregation (P2 design decision)

v1 emits ONE AirspaceStatus per region instead of one per GPS-jam hex.
Emitting per-hex would flood the UI (current MENA data has 40+ hexes).
The aggregated entry's `reason` lists sub-region names and high/med/low
counts so the analyst can still drill into the underlying data via the
evidence chain.

## Reroute intensity caveat

Military flight count is a CRUDE proxy for civil rerouting pressure.
A dedicated civil ADSB track-diversion pipeline would be more rigorous,
but this gets us a defensible 0-1 scalar today. Documented inline.

## Freshness registry

Added 5 new input keys to FRESHNESS_REGISTRY so the snapshot's
missing_inputs / stale_inputs classification tracks mobility sources.
Per-key maxAgeMin set to 2x the source's cron cadence:

  aviation:delays:faa:v1        — 60  min (30min cron)
  aviation:delays:intl:v3       — 90  min (30min cron + Railway jitter)
  aviation:notam:closures:v2    — 120 min (60min cron)
  intelligence:gpsjam:v2        — 240 min (120min cron)
  military:flights:v1           — 30  min (10min cron)

## seed-regional-snapshots.mjs Step 8

Replaced the hardcoded empty stub with a single call:

  const mobility = buildMobilityState(regionId, sources);

No reordering of the compute flow. Pure data, no async.

## Tests — 35 new unit tests

tests/regional-snapshot-mobility.test.mjs:

  airportToSnapshotRegion (5):
    - US/CA/MX → north-america
    - Other Americas → latam
    - IN/PK/BD → south-asia, rest of APAC → east-asia
    - europe/mena/africa direct
    - lowercase labels accepted
    - null/unknown safety

  gpsjamRegionToSnapshotRegion (4): 17 labels + "other" + undefined

  latLonToSnapshotRegion (3): major cities + oceans + invalid inputs

  buildAirports (5): empty sources, region filter, severity filter,
    severity → status (disrupted vs closed), reason passthrough

  buildAirspace (4): empty, region-scoped aggregation, no-match,
    low-level still restricted

  buildRerouteIntensity (4): zero, region-scoped count, saturation,
    missing lat/lon

  buildNotamClosures (4): empty, region attribution via airport alerts,
    unattributable ICAOs skipped, long-reason truncation

  buildMobilityState (4): full shape, all-empty sources, malformed
    inputs don't throw, region isolation across fields

## Verification

- node --test tests/regional-snapshot-mobility.test.mjs: 35/35 pass
- npm run test:data: 4537/4537 pass
- npm run typecheck: clean
- npm run typecheck:api: clean
- scripts/jsconfig.json typecheck: clean on touched files
- biome lint: clean on touched files

## Deferred to future iterations

- Direct civil flight track diversion counts per corridor
- flight_corridors[] stress level + rerouted_flights_24h
- LLM-assisted NOTAM structured parse
- h3 geometry for smarter hex-to-region attribution
- Mobility-based alert triggers in the diff engine

* fix(intelligence): address 2 review findings on koala73#2976

P2 #1 — Mexico bbox mismatch between airport and flight classifiers

airportToSnapshotRegion() routes Mexico to north-america by country,
but latLonToSnapshotRegion()'s NA bbox started at lat 20°N. Mexico City
(19.43°N), Guadalajara (20.67°N), and most Mexican airspace fell into
latam, so NA's reroute_intensity understated its actual military
pressure and the two classifiers disagreed on the same country.

Fix: lower the NA bbox southern edge to lat 16.0°N. That captures
every major Mexican city and state capital (Tuxtla Gutiérrez at
16.75°N is the southernmost) while still routing Guatemala City
(14.6°N), San Salvador, Belize, and Honduras to latam. Tiny airports
at lat ~14.9°N (Tapachula in southern Chiapas) route to latam —
acceptable because they're not in the monitored set and don't carry
meaningful military traffic.

Added 8 regression tests asserting classifier parity across major
Mexican cities and correct routing of Central American capitals.

P2 #2 — seed-meta freshness path for undated payloads

classifyInputs() was treating undated payloads as "fresh" via its
fallback branch, so FAA/NOTAM/AviationStack/GPS-jam stalls would
never drag down snapshot_confidence. None of those four payloads
carry a top-level fetchedAt field; the seeders track freshness in
companion seed-meta:* keys only.

Fix: added an optional `metaKey` field to FreshnessRegistry entries.
classifyInputs() now prefers the metaKey companion's fetchedAt when
the spec declares one, falling back to the primary payload's
top-level timestamp (existing behavior) and finally to "fresh"
(existing fallback for legacy undated payloads).

Wired through the full pipeline:
  - freshness.mjs: added metaKey type, ALL_META_KEYS export, and
    metaPayloads parameter to classifyInputs()
  - snapshot-meta.mjs: buildPreMeta() forwards metaSources
  - seed-regional-snapshots.mjs: readAllInputs() now fetches both
    ALL_INPUT_KEYS and ALL_META_KEYS in a single pipeline and
    returns { sources, metaSources }; main() destructures and
    passes metaSources through computeSnapshot → buildPreMeta
  - computeSnapshot() signature gained an optional metaSources arg
    (defaults to {} for back-compat with existing tests)

Registered metaKey for each mobility input:
  aviation:delays:faa:v1       → seed-meta:aviation:faa
  aviation:delays:intl:v3      → seed-meta:aviation:intl
  aviation:notam:closures:v2   → seed-meta:aviation:notam
  intelligence:gpsjam:v2       → seed-meta:intelligence:gpsjam

military:flights:v1 does NOT declare a metaKey because its payload
already carries top-level fetchedAt — this is asserted in a test.

Added 11 regression tests covering:
  - every undated mobility input has a metaKey
  - military:flights:v1 does NOT need a metaKey
  - ALL_META_KEYS aggregation
  - undated payload + fresh meta → fresh
  - undated payload + STALE meta → stale (the core bug fix)
  - undated payload + missing meta → falls back to fresh
  - missing payload → missing regardless of meta
  - legacy payloads with top-level timestamp still work
  - meta without fetchedAt falls through to payload timestamp

## Verification

- node --test tests/regional-snapshot-mobility.test.mjs: 54/54 pass
  (35 original + 19 new regression tests)
- npm run test:data: 4556/4556 pass
- npm run typecheck: clean
- biome lint on touched files: clean

* feat(intelligence): regime drift history (Phase 3 PR1)

Phase 3 PR1 of the Regional Intelligence Model. Adds an append-only
regime transition log per region plus a premium-gated RPC to read it.

## What landed

### New writer module: scripts/regional-snapshot/regime-history.mjs

Single public entry point:

  recordRegimeTransition(region, snapshot, diff, opts?)
    -> { recorded, entry, pushed, trimmed }

Pure builder + Redis-ops orchestrator + dependency-injected publisher.

Flow:
  1. buildTransitionEntry() returns null when diff has no regime_changed
     (steady-state snapshots produce no entry — pure transition stream)
  2. publishTransitionWithOps() LPUSHes onto
     intelligence:regime-history:v1:{region}, then LTRIMs to keep the
     most recent REGIME_HISTORY_MAX (100) entries
  3. defaultPublisher binds real Upstash REST calls; tests inject an
     in-memory ops object for offline coverage

LTRIM failure is non-fatal — entry already landed, next cycle will
re-trim. LPUSH failure short-circuits and reports pushed=false. The
recorder NEVER throws and is wrapped in its own try/catch in the seed
loop so snapshot persist is never blocked.

### seed-regional-snapshots.mjs hook

Added a regime-history call alongside the existing alert-emitter call,
right after persistSnapshot success. Same best-effort contract:
unconditional try/catch, log warn on throw, continue main loop.

### Proto + RPC: GetRegimeHistory

  proto/worldmonitor/intelligence/v1/get_regime_history.proto

  - GetRegimeHistoryRequest { region_id, limit (0..100) }
  - GetRegimeHistoryResponse { transitions: RegimeTransition[] }
  - RegimeTransition { region_id, label, previous_label,
                       transitioned_at, transition_driver, snapshot_id }

region_id validated as strict kebab-case (same regex as
get-regional-snapshot). limit capped server-side at MAX_LIMIT=100,
defaulting to 50 when omitted.

Added to IntelligenceService in service.proto. Generated openapi
JSON/YAML committed via `make generate`.

### Server handler: server/worldmonitor/intelligence/v1/get-regime-history.ts

LRANGE-based read (newest-first because the writer LPUSHes). adapter
is a dedicated exported function adaptTransition(raw) for testability.

LRANGE helper is inlined here because server/_shared/redis.ts has no
list helpers yet — this is the first list-reading handler in the
intelligence service. If a second list reader lands, the helper can
be promoted to a shared util.

Empty list / Redis miss / failed JSON parse all return
{ transitions: [] } so the client can distinguish "never changed" from
"upstream broken" via the HTTP status code, not the body.

Registered in handler.ts.

### Premium gating + cache tier

  src/shared/premium-paths.ts:   added /api/intelligence/v1/get-regime-history
  server/gateway.ts RPC_CACHE_TIER: same path with 'slow' tier (matches
                                    route-parity contract enforced by
                                    tests/route-cache-tier.test.mjs)

## Tests — 44 new unit tests

tests/regional-snapshot-regime-history.test.mjs (22 tests):

  buildTransitionEntry (7):
    - null on missing diff/region/snapshot
    - returns entry on regime change
    - first-ever transition (empty previous_label)
    - falls back to generated_at when transitioned_at is missing
    - preserves snapshot_id

  publishTransitionWithOps (8):
    - happy path (LPUSH + LTRIM both succeed)
    - canonical key prefix
    - LTRIM uses REGIME_HISTORY_MAX-1 stop
    - LPUSH failure → not pushed, LTRIM not called
    - LTRIM failure → pushed=true, trimmed=false (non-fatal)
    - LPUSH/LTRIM throwing caught and reported
    - null/empty entry → no-op

  recordRegimeTransition (5):
    - no-op on no regime change
    - records on regime change
    - publisher returning false → recorded=false
    - publisher exceptions swallowed
    - critical escalation labels preserved

  module constants (2): key prefix + max are valid

tests/get-regime-history.test.mts (22 tests):

  adaptTransition (4):
    - all fields snake → camel
    - missing fields → empty/zero defaults
    - first-ever transition shape preserved
    - non-numeric transitioned_at → 0

  handler structural checks (7): canonical key prefix, LRANGE usage,
    adapter export, handler export signature, MAX_LIMIT cap matches
    writer, missing-region short-circuit, malformed-entry filter

  intelligence handler registration (2): import + registration

  security wiring (2): premium path + cache-tier entry

  proto definition (7): RPC method declared, import wired, request
    shape, kebab regex, limit bounds, RegimeTransition fields,
    response shape

## Verification

- node --test tests/regional-snapshot-regime-history.test.mjs: 22/22 pass
- npx tsx --test tests/get-regime-history.test.mts: 22/22 pass
- npm run test:data: 4621/4621 pass
- npm run typecheck: clean
- npm run typecheck:api: clean
- biome lint on touched files: clean

## Deferred to future iterations

- Phase 3 PR2: weekly regional briefs LLM seeder (consumes regime history
  to highlight drift events in the weekly summary)
- Phase 3 PR3: UI block in RegionalIntelligenceBoard for regime drift
  timeline (can ride alongside or after PR2)
- Drift analytics: % of last N days spent in each regime, transition
  frequency rolling window, regime cycle detection
- Alert triggers on drift cycles (e.g., "thrashed between regimes 3 times
  in 7 days")

* fix(intelligence): address 2 review findings on koala73#2981

P2 #1 — transition_driver always empty in the live path

buildRegimeState(balance, previousLabel, '') at Step 11 passed an empty
driver because the diff hasn't been computed yet. The regime-history
recorder reads snapshot.regime.transition_driver which was therefore
always '' in production, despite tests exercising synthetic fixtures
with a populated driver.

Fix: after Step 15 derives triggerReason via inferTriggerReason(diff),
backfill regime.transition_driver = triggerReason when a genuine regime
change occurred. This ensures both the persisted snapshot's regime block
AND the regime-history entry carry the real driver (e.g., 'regime_shift',
'trigger_activation', 'corridor_break').

Added 2 regression tests: populated driver flows through, and pre-fix
empty-driver snapshots remain back-compatible.

P2 #2 — Redis failure returns cached false-empty history

get-regime-history.ts returned 200 {transitions:[]} on LRANGE failure.
The gateway caches 200 GET responses at the slow tier, so a transient
Upstash outage would be pinned as a false-empty history until the cache
TTL expired.

Fix: when redisLrange returns null (Redis unavailable or credentials
missing), the response now includes upstreamUnavailable: true in the
body. The gateway already checks for this flag in the response body
(line 434) and sets Cache-Control: no-store, so transient failures are
not cached.

Added 1 structural test asserting the upstreamUnavailable flag is set.

Verification:
- 24/24 writer tests, 23/23 handler tests, 4624/4624 full suite pass
- npm run typecheck: clean
- biome lint on touched files: clean

* fix(intelligence): correct misleading 'log once per region' comment (Greptile P2)

* feat(intelligence): weekly regional briefs (Phase 3 PR2)

Phase 3 PR2 of the Regional Intelligence Model. Adds LLM-powered
weekly intelligence briefs per region, completing the core feature set.

## New seeder: scripts/seed-regional-briefs.mjs

Standalone weekly cron script (not part of the 6h derived-signals bundle).
For each non-global region:
  1. Read the latest snapshot via two-hop Redis read
  2. Read recent regime transitions from the history log (koala73#2981)
  3. Call the LLM once per region with regime trajectory + balance +
     triggers + narrative context
  4. Write structured brief to intelligence:regional-briefs:v1:weekly:{region}
     with 8-day TTL (survives one missed weekly run)

Reuses the same injectable-callLlm + parse-validation + provider-chain
pattern from narrative.mjs and weekly-brief.mjs.

## New module: scripts/regional-snapshot/weekly-brief.mjs

  generateWeeklyBrief(region, snapshot, transitions, opts?)
    -> { region_id, generated_at, period_start, period_end,
         situation_recap, regime_trajectory, key_developments[],
         risk_outlook, provider, model }

  buildBriefPrompt()    — pure prompt builder
  parseBriefJson()      — JSON parser with prose-extraction fallback
  emptyBrief()          — canonical empty shape

Global region is skipped. Provider chain: Groq -> OpenRouter. Validate
callback ensures only parseable responses pass (narrative.mjs PR koala73#2960
review fix pattern).

## Proto + RPC: GetRegionalBrief

  proto/worldmonitor/intelligence/v1/get_regional_brief.proto

  - GetRegionalBriefRequest { region_id }
  - GetRegionalBriefResponse { brief: RegionalBrief }
  - RegionalBrief { region_id, generated_at, period_start, period_end,
                    situation_recap, regime_trajectory,
                    key_developments[], risk_outlook, provider, model }

## Server handler

  server/worldmonitor/intelligence/v1/get-regional-brief.ts

Simple getCachedJson read + adaptBrief snake->camel adapter.
Returns upstreamUnavailable: true on Redis failure so the gateway
skips caching (matching the get-regime-history pattern from koala73#2981).

## Premium gating + cache tier

  src/shared/premium-paths.ts + server/gateway.ts RPC_CACHE_TIER

## Tests — 27 new unit tests

  buildBriefPrompt (5): region/balance/transitions/narrative rendered,
                        empty transitions handled, missing fields tolerated
  parseBriefJson (5): valid JSON, garbage, all-empty, cap at 5, prose extraction
  generateWeeklyBrief (6): success, global skip, LLM fail, garbage, exception,
                           period_start/end delta
  emptyBrief (2): region_id + empty fields
  handler (4): key prefix, adapter export, upstreamUnavailable, registration
  security (2): premium path + cache tier
  proto (3): RPC declared, import wired, RegionalBrief fields

## Verification

- npm run test:data: 4651/4651 pass
- npm run typecheck + typecheck:api: clean
- biome lint: clean

* fix(intelligence): address 3 review findings on koala73#2989

P2 #1 — no consumer surface for GetRegionalBrief

Acknowledged. The consumer is the RegionalIntelligenceBoard panel,
which will call GetRegionalBrief and render a weekly brief block.
This wiring is Phase 3 PR3 (UI) scope — the RPC + Redis key are the
delivery mechanism, not the end surface. No code change in this commit;
the RPC is ready for the panel to consume.

P2 #2 — readRecentTransitions collapses failure to []

readRecentTransitions returned [] on Redis/network failure, which is
indistinguishable from a genuinely quiet week. The LLM then generates
a brief claiming "no regime transitions" when in reality the upstream
is down — fabricating false input.

Fix: return null on failure. The seeder skips the region with a clear
log message when transitions is null, so the brief is never written
with unreliable input. Empty array [] now only means genuinely no
transitions in the 7-day window.

P2 #3 — parseBriefJson accepts briefs the seeder rejects

parseBriefJson treated non-empty key_developments as valid even if
situation_recap was empty. The seeder gate only writes when
brief.situation_recap is truthy. That mismatch means the validator
pass + provider-fallback logic could accept a response that the seeder
then silently drops.

Fix: require situation_recap in parseBriefJson for valid=true, matching
the seeder gate. Now both checks agree on what constitutes a usable
brief, and the provider-fallback chain correctly falls through when
a provider returns a brief with developments but no recap.

* fix(intelligence): TTL path-segment fix + seed-meta always-write (Greptile P1+P2 on koala73#2989)

P1 — TTL silently not applied (briefs never expire)

Upstash REST ignores query-string SET options (?EX=N). The correct
form is path-segment: /set/{key}/{value}/EX/{seconds}. Without this
fix every brief persists indefinitely and Redis storage grows
unboundedly across weekly runs.

P2 — seed-meta not written when all regions skipped

writeExtraKeyWithMeta was gated on generated > 0. If every region
was skipped (no snapshot yet, or LLM failed), seed-meta was never
written, making the seeder indistinguishable from "never ran" in
health tooling. Now writes seed-meta whenever failed === 0,
carrying regionsSkipped count.

P2 #3 (validate gate) — already fixed in previous commit (parseBriefJson
now requires situation_recap for valid=true).

* fix(intelligence): register regional-briefs in health.js SEED_META + STANDALONE_KEYS (review P2 on koala73#2989)

* fix(intelligence): register regional-briefs in api/seed-health.js (review P2 on koala73#2989)

* fix(intelligence): raise brief TTL to 15 days to cover missed weekly cycle (review P2 on koala73#2989)

* fix(intelligence): distinguish missing-key from Redis-error + coverage-gated health (review P2s on koala73#2989)

P2 #1 — false upstreamUnavailable before first seed

getCachedJson returns null for both "key missing" and "Redis failed",
so the handler was advertising an outage for every region before the
first weekly seed ran. Switched to getRawJson (throws on Redis errors)
so null = genuinely missing key → clean empty 200, and thrown error =
upstream failure → upstreamUnavailable: true for gateway no-store.

P2 #2 — partial run hides coverage loss in health

The seed-meta was written with generated count even if only 1 of 7
regions produced a brief. /api/health treats any positive recordCount
as healthy, so broad regional failure was invisible to operators.

Fix: recordCount is set to 0 when generated < ceil(expectedRegions/2).
This makes /api/health report EMPTY_DATA for severely partial runs
while still writing seed-meta (so the seeder is confirmed to have run).
coverageOk flag in the summary payload lets operators drill into the
exact coverage state.

* fix(intelligence): tighten coverage gate to expectedRegions-1 (review P2 on koala73#2989)

…ss (koala73#3134)

* fix(sector-valuations): proxy Yahoo quoteSummary via Decodo curl egress

Yahoo's /v10/finance/quoteSummary returns HTTP 401 from Railway container
IPs. Railway logs 2026-04-16 show all 12 sector ETFs failing every 5-min
cron:

  [Sector] Yahoo quoteSummary XLK HTTP 401  (x12 per tick)
  [Market] Seeded 12/12 sectors, 0 valuations

Add a curl-based proxy fallback that matches scripts/_yahoo-fetch.mjs:
hit us.decodo.com (curl egress pool) NOT gate.decodo.com (CONNECT egress
pool). Per the 2026-04-16 probe documented in _yahoo-fetch.mjs header,
Yahoo blocks Decodo's CONNECT egress IPs but accepts the curl egress.
Reusing ytFetchViaProxy here would keep failing silently.

Shares the existing _yahooProxyFailCount / _yahooProxyCooldownUntil
cooldown state with fetchYahooChartDirect so both Yahoo paths pause
together if Decodo's curl pool also gets blocked.

No change to direct-path behavior when Yahoo is healthy.

* fix(sector-valuations): don't proxy on empty quoteSummary result (review)

Direct 200 with data.quoteSummary.result[0] absent is an app-level "no
data for this symbol" signal (e.g. delisted ETF). Proxy won't return
different data for a symbol Yahoo itself doesn't carry — falling back
would burn the 5-failure cooldown budget on structurally empty symbols
and mask a genuine proxy outage.

Resolve null on !result; keep JSON.parse catch going to proxy (garbage
body IS a transport-level signal — captive portal, Cloudflare challenge).

Review feedback from PR koala73#3134.

* fix(sector-valuations): split cooldown per egress route, cover transport failures (review)

Review feedback on PR koala73#3134, both P1.

P1 #1 — transport failures bypassed cooldown
  execFileSync timeouts, proxy-connect refusals, and JSON.parse on garbage
  bodies all went through the catch block and returned null without
  ticking _yahooProxyFailCount. In the exact failure mode this PR hardens
  against, the relay would have thrashed through 12 × 20s curl attempts
  per tick with no backoff. Extract a bumpCooldown() helper and call it
  from both the non-2xx branch and the catch block.

P1 #2 — two Decodo egress pools shared one cooldown budget
  fetchYahooChartDirect uses CONNECT via gate.decodo.com.
  _yahooQuoteSummaryProxyFallback uses curl via us.decodo.com.
  These are independent egress IP pools — per the 2026-04-16 probe,
  Yahoo blocks CONNECT but accepts curl. Sharing cooldown means 5
  CONNECT failures suppress the healthy curl path (and vice versa).
  Split into _yahooConnectProxy* (chart) and _yahooCurlProxy* (sector
  valuations).

Also: on proxy 200 with empty result, reset the curl counter. The route
is healthy even if this specific symbol has no data — don't pretend it's
a failure.

* fix(sector-valuations): non-blocking curl + settle guard (review round 3)

Review feedback on PR koala73#3134, both P1.

P1 #1 - double proxy invocation on timeout/error race
  req.destroy() inside the timeout handler can still emit 'error', and
  both handlers eagerly called resolve(_yahooQuoteSummaryProxyFallback(...)).
  A single upstream timeout therefore launched two curl subprocesses,
  double-ticked the cooldown counter, and blocked twice. Add a settled
  flag; settle() exits early on the second handler before evaluating the
  fallback.

P1 #2 - execFileSync blocks the relay event loop
  The relay serves HTTP/WS traffic on the same thread that awaits
  seedSectorSummary's per-symbol Yahoo fetch. execFileSync for up to 20s
  per failure x 5 failures before cooldown = ~100s of frozen event loop.
  Switch to promisify(execFile). resolve(promise) chains the Promise
  through fetchYahooQuoteSummary's outer Promise, so the main-loop await
  yields while curl runs. Other traffic continues during the fetch.

tests/sector-valuations.test.mjs: bump the static-analysis window from
1500 to 2000 chars so the field-extraction markers (ytdReturn etc.)
stay inside the window after the settle guard was added.

… gauges (koala73#3141)

* feat(positioning): standalone 24/7 Positioning panel with directional gauges

Extract Hyperliquid perp positioning from the "Perp Flow" tab inside
CommoditiesPanel into a dedicated "24/7 Positioning" panel (koala73#3140).

New panel features:
- SVG arc gauge (0-100) per asset with color encoding:
  green = bullish/longs crowded, red = bearish/shorts crowded
- Grouped by asset class: Commodities, Crypto, FX
- Visual elevation (glow + border) for scores above 40
- Click-through to relevant panels (crypto, commodities)
- Hover tooltips with score, funding rate, OI delta
- Warmup banner, stale badges, unavailable state
- Compact secondary metrics (funding rate + OI delta 1h)
- Mobile responsive (2-column grid below 600px)

Registration: panel-layout.ts, App.ts (primeTask + 5min refresh),
commands.ts (CMD+K), panels.ts (all variants), index.ts export,
en.json i18n keys.

Removals from CommoditiesPanel:
- "Perp Flow" tab and its tab type
- fetchHyperliquidFlow() method
- _renderFlow() and _renderFlowGrid() methods
- Associated state (_flow, _flowLoading)
- App.ts primeTask and refreshScheduler entries

No backend changes. Reuses existing getHyperliquidFlow RPC and
market:hyperliquid-flow:v1 Redis key.

* fix(positioning): correct oiDelta lookback, click targets, warmup state (review)

P1 #1: oiDelta1h used 2-point lookback (5min delta mislabeled as 1h).
  Restored to 13-point lookback matching the original MarketPanel.ts
  implementation (12 samples back = 1h at 5min cadence).

P1 #2: CLICK_TARGETS used display-style names (GOLD, WTI, BRENT) but
  seeded symbols are xyz:-prefixed Hyperliquid names (xyz:GOLD, xyz:CL,
  xyz:BRENTOIL). Fixed to match actual seeder output from
  seed-hyperliquid-flow.mjs.

P2: Empty/unavailable snapshots (cold seed, fresh deploy) now show
  warmup guidance instead of a hard error. The prior Perp Flow tab
  intentionally treated this as normal bootstrap. The new panel
  regressed to showError which is confusing on first deploy.

* fix(positioning): use data-panel selector, guard missing target panels (review)

P1: Click-through used getElementById('panel-X') but panels mount with
data-panel="X" attribute. Changed to querySelector('[data-panel="X"]').

P2: Crypto cards (BTC/ETH/SOL) target the 'crypto' panel which doesn't
exist in the commodity variant. resolveClickTarget() now checks if the
target panel is actually in the DOM before adding clickable styling and
data-pos-navigate. Cards whose target panel is absent render as
non-clickable (no cursor pointer, no navigate attribute).

koala73#3204)

* fix(brief): bundle resvg linux-x64-gnu native binding with carousel fn

Real root cause of every Telegram carousel WEBPAGE_CURL_FAILED
since PR koala73#3174 merged. Not middleware (last PR fixed that
theoretical path but not the observed failure). The Vercel
function itself crashes HTTP 500 FUNCTION_INVOCATION_FAILED on
every request including OPTIONS - the isolate can't initialise.

The handler imports brief-carousel-render which lazy-imports
@resvg/resvg-js. That package's js-binding.js does runtime
require(@resvg/resvg-js-<platform>-<arch>-<libc>). On Vercel
Lambda (Amazon Linux 2 glibc) that resolves to
@resvg/resvg-js-linux-x64-gnu. Vercel nft tracing does NOT
follow this conditional require so the optional peer package
isnt bundled. Cold start throws MODULE_NOT_FOUND, isolate
crashes, Vercel returns FUNCTION_INVOCATION_FAILED, Telegram
reports WEBPAGE_CURL_FAILED.

Fix: vercel.json functions.includeFiles forces linux-x64-gnu
binding into the carousel functions bundle. Only this route
needs it; every other api route is unaffected.

Verified:
- deploy-config tests 21/21 pass
- JSON valid
- Reproduced 500 via curl on all methods and UAs
- resvg-js/js-binding.js confirms linux-x64-gnu is the runtime
  binary on Amazon Linux 2 glibc

Post-merge: curl with TelegramBot UA should return 200 image/png
instead of 500; next cron tick should clear the Railway
[digest] Telegram carousel 400 line.

* Address Greptile P2s: regression guard + arch-assumption reasoning

Two P2 findings on PR koala73#3204:

P2 #1 (inline on vercel.json:6): Platform architecture assumption
undocumented. If Vercel migrates to Graviton/arm64 Lambda the
cold-start crash silently returns. vercel.json is strict JSON so
comments aren't possible inline.

P2 #2 (tests/deploy-config.test.mjs:17): No regression guard for
the carousel includeFiles rule. A future vercel.json tidy-up
could silently revert the fix with no CI signal.

Fixed both in a single block:

- New describe() in deploy-config.test.mjs asserts the carousel
  route's functions entry exists AND its includeFiles points at
  @resvg/resvg-js-linux-x64-gnu. Any drift fails the build.
- The block comment above it documents the Amazon Linux 2 x86_64
  glibc assumption that would have lived next to the includeFiles
  entry if JSON supported comments. Includes the Graviton/arm64
  migration pointer.

tests 22/22 pass (was 21, +1 new).

…a73#3207) (koala73#3242)

* chore(api): enforce sebuf contract via exceptions manifest (koala73#3207)

Adds api/api-route-exceptions.json as the single source of truth for
non-proto /api/ endpoints, with scripts/enforce-sebuf-api-contract.mjs
gating every PR via npm run lint:api-contract. Fixes the root-only blind
spot in the prior allowlist (tests/edge-functions.test.mjs), which only
scanned top-level *.js files and missed nested paths and .ts endpoints —
the gap that let api/supply-chain/v1/country-products.ts and friends
drift under proto domain URL prefixes unchallenged.

Checks both directions: every api/<domain>/v<N>/[rpc].ts must pair with
a generated service_server.ts (so a deleted proto fails CI), and every
generated service must have an HTTP gateway (no orphaned generated code).

Manifest entries require category + reason + owner, with removal_issue
mandatory for temporary categories (deferred, migration-pending) and
forbidden for permanent ones. .github/CODEOWNERS pins the manifest to
@SebastienMelki so new exceptions don't slip through review.

The manifest only shrinks: migration-pending entries (19 today) will be
removed as subsequent commits in this PR land each migration.

* refactor(maritime): migrate /api/ais-snapshot → maritime/v1.GetVesselSnapshot (koala73#3207)

The proto VesselSnapshot was carrying density + disruptions but the frontend
also needed sequence, relay status, and candidate_reports to drive the
position-callback system. Those only lived on the raw relay passthrough, so
the client had to keep hitting /api/ais-snapshot whenever callbacks were
registered and fall back to the proto RPC only when the relay URL was gone.

This commit pushes all three missing fields through the proto contract and
collapses the dual-fetch-path into one proto client call.

Proto changes (proto/worldmonitor/maritime/v1/):
  - VesselSnapshot gains sequence, status, candidate_reports.
  - GetVesselSnapshotRequest gains include_candidates (query: include_candidates).

Handler (server/worldmonitor/maritime/v1/get-vessel-snapshot.ts):
  - Forwards include_candidates to ?candidates=... on the relay.
  - Separate 5-min in-memory caches for the candidates=on and candidates=off
    variants; they have very different payload sizes and should not share a slot.
  - Per-request in-flight dedup preserved per-variant.

Frontend (src/services/maritime/index.ts):
  - fetchSnapshotPayload now calls MaritimeServiceClient.getVesselSnapshot
    directly with includeCandidates threaded through. The raw-relay path,
    SNAPSHOT_PROXY_URL, DIRECT_RAILWAY_SNAPSHOT_URL and LOCAL_SNAPSHOT_FALLBACK
    are gone — production already routed via Vercel, the "direct" branch only
    ever fired on localhost, and the proto gateway covers both.
  - New toLegacyCandidateReport helper mirrors toDensityZone/toDisruptionEvent.

api/ais-snapshot.js deleted; manifest entry removed. Only reduced the codegen
scope to worldmonitor.maritime.v1 (buf generate --path) — regenerating the
full tree drops // @ts-nocheck from every client/server file and surfaces
pre-existing type errors across 30+ unrelated services, which is not in
scope for this PR.

Shape-diff vs legacy payload:
  - disruptions / density: proto carries the same fields, just with the
    GeoCoordinates wrapper and enum strings (remapped client-side via
    existing toDisruptionEvent / toDensityZone helpers).
  - sequence, status.{connected,vessels,messages}: now populated from the
    proto response — was hardcoded to 0/false in the prior proto fallback.
  - candidateReports: same shape; optional numeric fields come through as
    0 instead of undefined, which the legacy consumer already handled.

* refactor(sanctions): migrate /api/sanctions-entity-search → LookupSanctionEntity (koala73#3207)

The proto docstring already claimed "OFAC + OpenSanctions" coverage but the
handler only fuzzy-matched a local OFAC Redis index — narrower than the
legacy /api/sanctions-entity-search, which proxied OpenSanctions live (the
source advertised in docs/api-proxies.mdx). Deleting the legacy without
expanding the handler would have been a silent coverage regression for
external consumers.

Handler changes (server/worldmonitor/sanctions/v1/lookup-entity.ts):
  - Primary path: live search against api.opensanctions.org/search/default
    with an 8s timeout and the same User-Agent the legacy edge fn used.
  - Fallback path: the existing OFAC local fuzzy match, kept intact for when
    OpenSanctions is unreachable / rate-limiting.
  - Response source field flips between 'opensanctions' (happy path) and
    'ofac' (fallback) so clients can tell which index answered.
  - Query validation tightened: rejects q > 200 chars (matches legacy cap).

Rate limiting:
  - Added /api/sanctions/v1/lookup-entity to ENDPOINT_RATE_POLICIES at 30/min
    per IP — matches the legacy createIpRateLimiter budget. The gateway
    already enforces per-endpoint policies via checkEndpointRateLimit.

Docs:
  - docs/api-proxies.mdx — dropped the /api/sanctions-entity-search row
    (plus the orphaned /api/ais-snapshot row left over from the previous
    commit in this PR).
  - docs/panels/sanctions-pressure.mdx — points at the new RPC URL and
    describes the OpenSanctions-primary / OFAC-fallback semantics.

api/sanctions-entity-search.js deleted; manifest entry removed.

* refactor(military): migrate /api/military-flights → ListMilitaryFlights (koala73#3207)

Legacy /api/military-flights read a pre-baked Redis blob written by the
seed-military-flights cron and returned flights in a flat app-friendly
shape (lat/lon, lowercase enums, lastSeenMs). The proto RPC takes a bbox,
fetches OpenSky live, classifies server-side, and returns nested
GeoCoordinates + MILITARY_*_TYPE_* enum strings + lastSeenAt — same data,
different contract.

fetchFromRedis in src/services/military-flights.ts was doing nothing
sebuf-aware. Renamed it to fetchViaProto and rewrote to:

  - Instantiate MilitaryServiceClient against getRpcBaseUrl().
  - Iterate MILITARY_QUERY_REGIONS (PACIFIC + WESTERN) in parallel — same
    regions the desktop OpenSky path and the seed cron already use, so
    dashboard coverage tracks the analytic pipeline.
  - Dedup by hexCode across regions.
  - Map proto → app shape via new mapProtoFlight helper plus three reverse
    enum maps (AIRCRAFT_TYPE_REVERSE, OPERATOR_REVERSE, CONFIDENCE_REVERSE).

The seed cron (scripts/seed-military-flights.mjs) stays put: it feeds
regional-snapshot mobility, cross-source signals, correlation, and the
health freshness check (api/health.js: 'military:flights:v1'). None of
those read the legacy HTTP endpoint; they read the Redis key directly.
The proto handler uses its own per-bbox cache keys under the same prefix,
so dashboard traffic no longer races the seed cron's blob — the two paths
diverge by a small refresh lag, which is acceptable.

Docs: dropped the /api/military-flights row from docs/api-proxies.mdx.

api/military-flights.js deleted; manifest entry removed.

Shape-diff vs legacy:
  - f.location.{latitude,longitude} → f.lat, f.lon
  - f.aircraftType: MILITARY_AIRCRAFT_TYPE_TANKER → 'tanker' via reverse map
  - f.operator: MILITARY_OPERATOR_USAF → 'usaf' via reverse map
  - f.confidence: MILITARY_CONFIDENCE_LOW → 'low' via reverse map
  - f.lastSeenAt (number) → f.lastSeen (Date)
  - f.enrichment → f.enriched (with field renames)
  - Extra fields registration / aircraftModel / origin / destination /
    firstSeenAt now flow through where proto populates them.

* fix(supply-chain): thread includeCandidates through chokepoint status (koala73#3207)

Caught by tsconfig.api.json typecheck in the pre-push hook (not covered
by the plain tsc --noEmit run that ran before I pushed the ais-snapshot
commit). The chokepoint status handler calls getVesselSnapshot internally
with a static no-auth request — now required to include the new
includeCandidates bool from the proto extension.

Passing false: server-internal callers don't need per-vessel reports.

* test(maritime): update getVesselSnapshot cache assertions (koala73#3207)

The ais-snapshot migration replaced the single cachedSnapshot/cacheTimestamp
pair with a per-variant cache so candidates-on and candidates-off payloads
don't evict each other. Pre-push hook surfaced that tests/server-handlers
still asserted the old variable names. Rewriting the assertions to match
the new shape while preserving the invariants they actually guard:

  - Freshness check against slot TTL.
  - Cache read before relay call.
  - Per-slot in-flight dedup.
  - Stale-serve on relay failure (result ?? slot.snapshot).

* chore(proto): restore // @ts-nocheck on regenerated maritime files (koala73#3207)

I ran 'buf generate --path worldmonitor/maritime/v1' to scope the proto
regen to the one service I was changing (to avoid the toolchain drift
that drops @ts-nocheck from 60+ unrelated files — separate issue). But
the repo convention is the 'make generate' target, which runs buf and
then sed-prepends '// @ts-nocheck' to every generated .ts file. My
scoped command skipped the sed step. The proto-check CI enforces the
sed output, so the two maritime files need the directive restored.

* refactor(enrichment): decomm /api/enrichment/{company,signals} legacy edge fns (koala73#3207)

Both endpoints were already ported to IntelligenceService:
  - getCompanyEnrichment  (/api/intelligence/v1/get-company-enrichment)
  - listCompanySignals    (/api/intelligence/v1/list-company-signals)

No frontend callers of the legacy /api/enrichment/* paths exist. Removes:
  - api/enrichment/company.js, signals.js, _domain.js
  - api-route-exceptions.json migration-pending entries (58 remain)
  - docs/api-proxies.mdx rows for /api/enrichment/{company,signals}
  - docs/architecture.mdx reference updated to the IntelligenceService RPCs

Verified: typecheck, typecheck:api, lint:api-contract (89 files / 58 entries),
lint:boundaries, tests/edge-functions.test.mjs (136 pass),
tests/enrichment-caching.test.mjs (14 pass — still guards the intelligence/v1
handlers), make generate is zero-diff.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor(leads): migrate /api/{contact,register-interest} → LeadsService (koala73#3207)

New leads/v1 sebuf service with two POST RPCs:
  - SubmitContact    → /api/leads/v1/submit-contact
  - RegisterInterest → /api/leads/v1/register-interest

Handler logic ported 1:1 from api/contact.js + api/register-interest.js:
  - Turnstile verification (desktop sources bypass, preserved)
  - Honeypot (website field) silently accepts without upstream calls
  - Free-email-domain gate on SubmitContact (422 ApiError)
  - validateEmail (disposable/offensive/typo-TLD/MX) on RegisterInterest
  - Convex writes via ConvexHttpClient (contactMessages:submit, registerInterest:register)
  - Resend notification + confirmation emails (HTML templates unchanged)

Shared helpers moved to server/_shared/:
  - turnstile.ts (getClientIp + verifyTurnstile)
  - email-validation.ts (disposable/offensive/MX checks)

Rate limits preserved via ENDPOINT_RATE_POLICIES:
  - submit-contact:    3/hour per IP (was in-memory 3/hr)
  - register-interest: 5/hour per IP (was in-memory 5/hr; desktop
    sources previously capped at 2/hr via shared in-memory map —
    now 5/hr like everyone else, accepting the small regression in
    exchange for Upstash-backed global limiting)

Callers updated:
  - pro-test/src/App.tsx contact form → new submit-contact path
  - src-tauri/sidecar/local-api-server.mjs cloud-fallback rewrites
    /api/register-interest → /api/leads/v1/register-interest when
    proxying; keeps local path for older desktop builds
  - src/services/runtime.ts isKeyFreeApiTarget allows both old and
    new paths through the WORLDMONITOR_API_KEY-optional gate

Tests:
  - tests/contact-handler.test.mjs rewritten to call submitContact
    handler directly; asserts on ValidationError / ApiError
  - tests/email-validation.test.mjs + tests/turnstile.test.mjs
    point at the new server/_shared/ modules

Deleted: api/contact.js, api/register-interest.js, api/_ip-rate-limit.js,
api/_turnstile.js, api/_email-validation.js, api/_turnstile.test.mjs.
Manifest entries removed (58 → 56). Docs updated (api-platform,
api-commerce, usage-rate-limits).

Verified: npm run typecheck + typecheck:api + lint:api-contract
(88 files / 56 entries) + lint:boundaries pass; full test:data
(5852 tests) passes; make generate is zero-diff.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* chore(pro-test): rebuild bundle for leads/v1 contact form (koala73#3207)

Updates the enterprise contact form to POST to /api/leads/v1/submit-contact
(old path /api/contact removed in the previous commit).

Bundle is rebuilt from pro-test/src/App.tsx source change in 9ccd309.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review): address HIGH review findings 1-3 (koala73#3207)

Three review findings from @koala73 on the sebuf-migration PR, all
silent bugs that would have shipped to prod:

### 1. Sanctions rate-limit policy was dead code

ENDPOINT_RATE_POLICIES keyed the 30/min budget under
/api/sanctions/v1/lookup-entity, but the generated route (from the
proto RPC LookupSanctionEntity) is /api/sanctions/v1/lookup-sanction-entity.
hasEndpointRatePolicy / getEndpointRatelimit are exact-string pathname
lookups, so the mismatch meant the endpoint fell through to the
generic 600/min global limiter instead of the advertised 30/min.

Net effect: the live OpenSanctions proxy endpoint (unauthenticated,
external upstream) had 20x the intended rate budget. Fixed by renaming
the policy key to match the generated route.

### 2. Lost stale-seed fallback on military-flights

Legacy api/military-flights.js cascaded military:flights:v1 →
military:flights:stale:v1 before returning empty. The new proto
handler went straight to live OpenSky/relay and returned null on miss.

Relay or OpenSky hiccup used to serve stale seeded data (24h TTL);
under the new handler it showed an empty map. Both keys are still
written by scripts/seed-military-flights.mjs on every run — fix just
reads the stale key when the live fetch returns null, converts the
seed's app-shape flights (flat lat/lon, lowercase enums, lastSeenMs)
to the proto shape (nested GeoCoordinates, enum strings, lastSeenAt),
and filters to the request bbox.

Read via getRawJson (unprefixed) to match the seed cron's writes,
which bypass the env-prefix system.

### 3. Hex-code casing mismatch broke getFlightByHex

The seed cron writes hexCode: icao24.toUpperCase() (uppercase);
src/services/military-flights.ts:getFlightByHex uppercases the lookup
input: f.hexCode === hexCode.toUpperCase(). The new proto handler
preserved OpenSky's lowercase icao24, and mapProtoFlight is a
pass-through. getFlightByHex was silently returning undefined for
every call after the migration.

Fix: uppercase in the proto handler (live + stale paths), and document
the invariant in a comment on MilitaryFlight.hex_code in
military_flight.proto so future handlers don't re-break it.

### Verified

- typecheck + typecheck:api clean
- lint:api-contract (56 entries) / lint:boundaries clean
- tests/edge-functions.test.mjs 130 pass
- make generate zero-diff (openapi spec regenerated for proto comment)

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review): restore desktop 2/hr rate cap on register-interest (koala73#3207)

Addresses HIGH review finding #4 from @koala73. The legacy
api/register-interest.js applied a nested 2/hr per-IP cap when
`source === 'desktop-settings'`, on top of the generic 5/hr endpoint
budget. The sebuf migration lost this — desktop-source requests now
enjoy the full 5/hr cap.

Since `source` is an unsigned client-supplied field, anyone sending
`source: 'desktop-settings'` skips Turnstile AND gets 5/hr. Without
the tighter cap the Turnstile bypass is cheaper to abuse.

Added `checkScopedRateLimit` to `server/_shared/rate-limit.ts` — a
reusable second-stage Upstash limiter keyed on an opaque scope string
+ caller identifier. Fail-open on Redis errors to match existing
checkRateLimit / checkEndpointRateLimit semantics. Handlers that need
per-subscope caps on top of the gateway-level endpoint budget use this
helper.

In register-interest: when `isDesktopSource`, call checkScopedRateLimit
with scope `/api/leads/v1/register-interest#desktop`, limit=2, window=1h,
IP as identifier. On exceeded → throw ApiError(429).

### What this does not fix

This caps the blast radius of the Turnstile bypass but does not close
it — an attacker sending `source: 'desktop-settings'` still skips
Turnstile (just at 2/hr instead of 5/hr). The proper fix is a signed
desktop-secret header that authenticates the bypass; filed as
follow-up koala73#3252. That requires coordinated Tauri build + Vercel env
changes out of scope for koala73#3207.

### Verified

- typecheck + typecheck:api clean
- lint:api-contract (56 entries)
- tests/edge-functions.test.mjs + contact-handler.test.mjs (147 pass)

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review): MEDIUM + LOW + rate-limit-policy CI check (koala73#3207)

Closes out the remaining @koala73 review findings from koala73#3242 that
didn't already land in the HIGH-fix commits, plus the requested CI
check that would have caught HIGH #1 (dead-code policy key) at
review time.

### MEDIUM #5 — Turnstile missing-secret policy default

Flip `verifyTurnstile`'s default `missingSecretPolicy` from `'allow'`
to `'allow-in-development'`. Dev with no secret = pass (expected
local); prod with no secret = reject + log. submit-contact was
already explicitly overriding to `'allow-in-development'`;
register-interest was silently getting `'allow'`. Safe default now
means a future missing-secret misconfiguration in prod gets caught
instead of silently letting bots through. Removed the now-redundant
override in submit-contact.

### MEDIUM #6 — Silent enum fallbacks in maritime client

`toDisruptionEvent` mapped `AIS_DISRUPTION_TYPE_UNSPECIFIED` / unknown
enum values → `gap_spike` / `low` silently. Refactored to return null
when either enum is unknown; caller filters nulls out of the array.
Handler doesn't produce UNSPECIFIED today, but the `gap_spike`
default would have mislabeled the first new enum value the proto
ever adds — dropping unknowns is safer than shipping wrong labels.

### LOW — Copy drift in register-interest email

Email template hardcoded `435+ Sources`; PR koala73#3241 bumped marketing to
`500+`. Bumped in the rewritten file to stay consistent.

The `as any` on Convex mutation names carried over from legacy and
filed as follow-up koala73#3253.

### Rate-limit-policy coverage lint

`scripts/enforce-rate-limit-policies.mjs` validates every key in
`ENDPOINT_RATE_POLICIES` resolves to a proto-generated gateway route
by cross-referencing `docs/api/*.openapi.yaml`. Fails with the
sanctions-entity-search incident referenced in the error message so
future drift has a paper trail.

Wired into package.json (`lint:rate-limit-policies`) and the pre-push
hook alongside `lint:boundaries`. Smoke-tested both directions —
clean repo passes (5 policies / 175 routes), seeded drift (the exact
HIGH #1 typo) fails with the advertised remedy text.

### Verified
- `lint:rate-limit-policies` ✓
- `typecheck` + `typecheck:api` ✓
- `lint:api-contract` ✓ (56 entries)
- `lint:boundaries` ✓
- edge-functions + contact-handler tests (147 pass)

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor(commit 5): decomm /api/eia/* + migrate /api/satellites → IntelligenceService (koala73#3207)

Both targets turned out to be decomm-not-migration cases. The original
plan called for two new services (economic/v1.GetEiaSeries +
natural/v1.ListSatellitePositions) but research found neither was
needed:

### /api/eia/[[...path]].js — pure decomm, zero consumers

The "catch-all" is a misnomer — only two paths actually worked,
/api/eia/health and /api/eia/petroleum, both Redis-only readers.
Zero frontend callers in src/. Zero server-side readers. Nothing
consumes the `energy:eia-petroleum:v1` key that seed-eia-petroleum.mjs
writes daily.

The EIA data the frontend actually uses goes through existing typed
RPCs in economic/v1: GetEnergyPrices, GetCrudeInventories,
GetNatGasStorage, GetEnergyCapacity. None of those touch /api/eia/*.

Building GetEiaSeries would have been dead code. Deleted the legacy
file + its test (tests/api-eia-petroleum.test.mjs — it only covered
the legacy endpoint, no behavior to preserve). Empty api/eia/ dir
removed.

**Note for review:** the Redis seed cron keeps running daily and
nothing consumes it. If that stays unused, seed-eia-petroleum.mjs
should be retired too (separate PR). Out of scope for sebuf-migration.

### /api/satellites.js — Learning #2 strikes again

IntelligenceService.ListSatellites already exists at
/api/intelligence/v1/list-satellites, reads the same Redis key
(intelligence:satellites:tle:v1), and supports an optional country
filter the legacy didn't have.

One frontend caller in src/services/satellites.ts needed to switch
from `fetch(toApiUrl('/api/satellites'))` to the typed
IntelligenceServiceClient.listSatellites. Shape diff was tiny —
legacy `noradId` became proto `id` (handler line 36 already picks
either), everything else identical. alt/velocity/inclination in the
proto are ignored by the caller since it propagates positions
client-side via satellite.js.

Kept the client-side cache + failure cooldown + 20s timeout (still
valid concerns at the caller level).

### Manifest + docs
- api-route-exceptions.json: 56 → 54 entries (both removed)
- docs/api-proxies.mdx: dropped the two rows from the Raw-data
  passthroughs table

### Verified
- typecheck + typecheck:api ✓
- lint:api-contract (54 entries) / lint:boundaries / lint:rate-limit-policies ✓
- tests/edge-functions.test.mjs 127 pass (down from 130 — 3 tests were
  for the deleted eia endpoint)
- make generate zero-diff (no proto changes)

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor(commit 6): migrate /api/supply-chain/v1/{country-products,multi-sector-cost-shock} → SupplyChainService (koala73#3207)

Both endpoints were hand-rolled TS handlers sitting under a proto URL prefix —
the exact drift the manifest guardrail flagged. Promoted both to typed RPCs:

- GetCountryProducts → /api/supply-chain/v1/get-country-products
- GetMultiSectorCostShock → /api/supply-chain/v1/get-multi-sector-cost-shock

Handlers preserve the existing semantics: PRO-gate via isCallerPremium(ctx.request),
iso2 / chokepointId validation, raw bilateral-hs4 Redis read (skip env-prefix to
match seeder writes), CHOKEPOINT_STATUS_KEY for war-risk tier, and the math from
_multi-sector-shock.ts unchanged. Empty-data and non-PRO paths return the typed
empty payload (no 403 — the sebuf gateway pattern is empty-payload-on-deny).

Client wrapper switches from premiumFetch to client.getCountryProducts/
client.getMultiSectorCostShock. Legacy MultiSectorShock / MultiSectorShockResponse /
CountryProductsResponse names remain as type aliases of the generated proto types
so CountryBriefPanel + CountryDeepDivePanel callsites compile with zero churn.

Manifest 54 → 52. Rate-limit gateway routes 175 → 177.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(gateway): add cache-tier entries for new supply-chain RPCs (koala73#3207)

Pre-push tests/route-cache-tier.test.mjs caught the missing entries.
Both PRO-gated, request-varying — match the existing supply-chain PRO cohort
(get-country-cost-shock, get-bypass-options, etc.) at slow-browser tier.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor(commit 7): migrate /api/scenario/v1/{run,status,templates} → ScenarioService (koala73#3207)

Promote the three literal-filename scenario endpoints to a typed sebuf
service with three RPCs:

  POST /api/scenario/v1/run-scenario        (RunScenario)
  GET  /api/scenario/v1/get-scenario-status (GetScenarioStatus)
  GET  /api/scenario/v1/list-scenario-templates (ListScenarioTemplates)

Preserves all security invariants from the legacy handlers:
- 405 for wrong method (sebuf service-config method gate)
- scenarioId validation against SCENARIO_TEMPLATES registry
- iso2 regex ^[A-Z]{2}$
- JOB_ID_RE path-traversal guard on status
- Per-IP 10/min rate limit (moved to gateway ENDPOINT_RATE_POLICIES)
- Queue-depth backpressure (>100 → 429)
- PRO gating via isCallerPremium
- AbortSignal.timeout on every Redis pipeline (runRedisPipeline helper)

Wire-level diffs vs legacy:
- Per-user RL now enforced at the gateway (same 10/min/IP budget).
- Rate-limit response omits Retry-After header; retryAfter is in the
  body per error-mapper.ts convention.
- ListScenarioTemplates emits affectedHs2: [] when the registry entry
  is null (all-sectors sentinel); proto repeated cannot carry null.
- RunScenario returns { jobId, status } (no statusUrl field — unused
  by SupplyChainPanel, drop from wire).

Gateway wiring:
- server/gateway.ts RPC_CACHE_TIER: list-scenario-templates → 'daily'
  (matches legacy max-age=3600); get-scenario-status → 'slow-browser'
  (premium short-circuit target, explicit entry required by
  tests/route-cache-tier.test.mjs).
- src/shared/premium-paths.ts: swap old run/status for the new
  run-scenario/get-scenario-status paths.
- api/scenario/v1/{run,status,templates}.ts deleted; 3 manifest
  exceptions removed (63 → 52 → 49 migration-pending).

Client:
- src/services/scenario/index.ts — typed client wrapper using
  premiumFetch (injects Clerk bearer / API key).
- src/components/SupplyChainPanel.ts — polling loop swapped from
  premiumFetch strings to runScenario/getScenarioStatus. Hard 20s
  timeout on run preserved via AbortSignal.any.

Tests:
- tests/scenario-handler.test.mjs — 18 new handler-level tests
  covering every security invariant + the worker envelope coercion.
- tests/edge-functions.test.mjs — scenario sections removed,
  replaced with a breadcrumb pointer to the new test file.

Docs: api-scenarios.mdx, scenario-engine.mdx, usage-rate-limits.mdx,
usage-errors.mdx, supply-chain.mdx refreshed with new paths.

Verified: typecheck, typecheck:api, lint:api-contract (49 entries),
lint:rate-limit-policies (6/180), lint:boundaries, route-cache-tier
(parity), full edge-functions (117) + scenario-handler (18).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor(commit 8): migrate /api/v2/shipping/{route-intelligence,webhooks} → ShippingV2Service (koala73#3207)

Partner-facing endpoints promoted to a typed sebuf service. Wire shape
preserved byte-for-byte (camelCase field names, ISO-8601 fetchedAt, the
same subscriberId/secret formats, the same SET + SADD + EXPIRE 30-day
Redis pipeline). Partner URLs /api/v2/shipping/* are unchanged.

RPCs landed:
- GET  /route-intelligence  → RouteIntelligence  (PRO, slow-browser)
- POST /webhooks            → RegisterWebhook    (PRO)
- GET  /webhooks            → ListWebhooks       (PRO, slow-browser)

The existing path-parameter URLs remain on the legacy edge-function
layout because sebuf's HTTP annotations don't currently model path
params (grep proto/**/*.proto for `path: "{…}"` returns zero). Those
endpoints are split into two Vercel dynamic-route files under
api/v2/shipping/webhooks/, behaviorally identical to the previous
hybrid file but cleanly separated:
- GET  /webhooks/{subscriberId}                → [subscriberId].ts
- POST /webhooks/{subscriberId}/rotate-secret  → [subscriberId]/[action].ts
- POST /webhooks/{subscriberId}/reactivate     → [subscriberId]/[action].ts

Both get manifest entries under `migration-pending` pointing at koala73#3207.

Other changes
- scripts/enforce-sebuf-api-contract.mjs: extended GATEWAY_RE to accept
  api/v{N}/{domain}/[rpc].ts (version-first) alongside the canonical
  api/{domain}/v{N}/[rpc].ts; first-use of the reversed ordering is
  shipping/v2 because that's the partner contract.
- vite.config.ts: dev-server sebuf interceptor regex extended to match
  both layouts; shipping/v2 import + allRoutes entry added.
- server/gateway.ts: RPC_CACHE_TIER entries for /api/v2/shipping/
  route-intelligence + /webhooks (slow-browser; premium-gated endpoints
  short-circuit to slow-browser but the entries are required by
  tests/route-cache-tier.test.mjs).
- src/shared/premium-paths.ts: route-intelligence + webhooks added.
- tests/shipping-v2-handler.test.mjs: 18 handler-level tests covering
  PRO gate, iso2/cargoType/hs2 coercion, SSRF guards (http://, RFC1918,
  cloud metadata, IMDS), chokepoint whitelist, alertThreshold range,
  secret/subscriberId format, pipeline shape + 30-day TTL, cross-tenant
  owner isolation, `secret` omission from list response.

Manifest delta
- Removed: api/v2/shipping/route-intelligence.ts, api/v2/shipping/webhooks.ts
- Added:   api/v2/shipping/webhooks/[subscriberId].ts (migration-pending)
- Added:   api/v2/shipping/webhooks/[subscriberId]/[action].ts (migration-pending)
- Added:   api/internal/brief-why-matters.ts (internal-helper) — regression
  surface from the koala73#3248 main merge, which introduced the file without a
  manifest entry. Filed here to keep the lint green; not strictly in scope
  for commit 8 but unblocking.

Net result: 49 → 47 `migration-pending` entries (one net-removal even
though webhook path-params stay pending, because two files collapsed
into two dynamic routes).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review HIGH 1): SupplyChainServiceClient must use premiumFetch (koala73#3207)

Signed-in browser pro users were silently hitting 401 on 8 supply-chain
premium endpoints (country-products, multi-sector-cost-shock,
country-chokepoint-index, bypass-options, country-cost-shock,
sector-dependency, route-explorer-lane, route-impact). The shared
client was constructed with globalThis.fetch, so no Clerk bearer or
X-WorldMonitor-Key was injected. The gateway's validateApiKey runs
with forceKey=true for PREMIUM_RPC_PATHS and 401s before isCallerPremium
is consulted. The generated client's try/catch collapses the 401 into
an empty-fallback return, leaving panels blank with no visible error.

Fix is one line at the client constructor: swap globalThis.fetch for
premiumFetch. The same pattern is already in use for insider-transactions,
stock-analysis, stock-backtest, scenario, trade (premiumClient) — this
was an omission on this client, not a new pattern.

premiumFetch no-ops safely when no credentials are available, so the
5 non-premium methods on this client (shippingRates, chokepointStatus,
chokepointHistory, criticalMinerals, shippingStress) continue to work
unchanged.

This also fixes two panels that were pre-existing latently broken on
main (chokepoint-index, bypass-options, etc. — predating koala73#3207, not
regressions from it). Commit 6 expanded the surface by routing two more
methods through the same buggy client; this commit fixes the class.

From koala73 review (koala73#3242 second-pass, HIGH new #1):
> Exact class PR koala73#3233 fixed for RegionalIntelligenceBoard /
> DeductionPanel / trade / country-intel. Supply-chain was not in
> koala73#3233's scope.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review HIGH 2): restore 400 on input-shape errors for 2 supply-chain handlers (koala73#3207)

Commit 6 collapsed all non-happy paths into empty-200 on
`get-country-products` and `get-multi-sector-cost-shock`, including
caller-bug cases that legacy returned 400 for:

- get-country-products: malformed iso2 → empty 200 (was 400)
- get-multi-sector-cost-shock: malformed iso2 / missing chokepointId /
  unknown chokepointId → empty 200 (was 400)

The commit message for 6 called out the 403-for-non-pro → empty-200
shift ("sebuf gateway pattern is empty-payload-on-deny") but not the
400 shift. They're different classes:

- Empty-payload-200 for PRO-deny: intentional contract change, already
  documented and applied across the service. Generated clients treat
  "you lack PRO" as "no data" — fine.
- Empty-payload-200 for malformed input: caller bug silently masked.
  External API consumers can't distinguish "bad wiring" from "genuinely
  no data", test harnesses lose the signal, bad calling code doesn't
  surface in Sentry.

Fix: `throw new ValidationError(violations)` on the 3 input-shape
branches. The generated sebuf server maps ValidationError → HTTP 400
(see src/generated/server/.../service_server.ts and leads/v1 which
already uses this pattern).

PRO-gate deny stays as empty-200 — that contract shift was intentional
and is preserved.

Regression tests added at tests/supply-chain-validation.test.mjs (8
cases) pinning the three-way contract:
- bad input                         → 400 (ValidationError)
- PRO-gate deny on valid input      → 200 empty
- valid PRO input, no data in Redis → 200 empty (unchanged)

From koala73 review (koala73#3242 second-pass, HIGH new #2).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review HIGH 3): restore statusUrl on RunScenarioResponse + document 202→200 wire break (koala73#3207)

Commit 7 silently shifted /api/scenario/v1/run-scenario's response
contract in two ways that the commit message covered only partially:

1. HTTP 202 Accepted → HTTP 200 OK
2. Dropped `statusUrl` string from the response body

The `statusUrl` drop was mentioned as "unused by SupplyChainPanel" but
not framed as a contract change. The 202 → 200 shift was not mentioned
at all. This is a same-version (v1 → v1) migration, so external callers
that key off either signal — `response.status === 202` or
`response.body.statusUrl` — silently branch incorrectly.

Evaluated options:
  (a) sebuf per-RPC status-code config — not available. sebuf's
      HttpConfig only models `path` and `method`; no status annotation.
  (b) Bump to scenario/v2 — judged heavier than the break itself for
      a single status-code shift. No in-repo caller uses 202 or
      statusUrl; the docs-level impact is containable.
  (c) Accept the break, document explicitly, partially restore.

Took option (c):

- Restored `statusUrl` in the proto (new field `string status_url = 3`
  on RunScenarioResponse). Server computes
  `/api/scenario/v1/get-scenario-status?jobId=<encoded job_id>` and
  populates it on every successful enqueue. External callers that
  followed this URL keep working unchanged.
- 202 → 200 is not recoverable inside the sebuf generator, so it is
  called out explicitly in two places:
    - docs/api-scenarios.mdx now includes a prominent `<Warning>` block
      documenting the v1→v1 contract shift + the suggested migration
      (branch on response body shape, not HTTP status).
    - RunScenarioResponse proto comment explains why 200 is the new
      success status on enqueue.
  OpenAPI bundle regenerated to reflect the restored statusUrl field.

- Regression test added in tests/scenario-handler.test.mjs pinning
  `statusUrl` to the exact URL-encoded shape — locks the invariant so
  a future proto rename or handler refactor can't silently drop it
  again.

From koala73 review (koala73#3242 second-pass, HIGH new #3).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review HIGH 1/2): close webhook tenant-isolation gap on shipping/v2 (koala73#3207)

Koala flagged this as a merge blocker in PR koala73#3242 review.

server/worldmonitor/shipping/v2/{register-webhook,list-webhooks}.ts
migrated without reinstating validateApiKey(req, { forceKey: true }),
diverging from both the sibling api/v2/shipping/webhooks/[subscriberId]
routes and the documented "X-WorldMonitor-Key required" contract in
docs/api-shipping-v2.mdx.

Attack surface: the gateway accepts Clerk bearer auth as a pro signal.
A Clerk-authenticated pro user with no X-WorldMonitor-Key reaches the
handler, callerFingerprint() falls back to 'anon', and every such
caller collapses into a shared webhook:owner:anon:v1 bucket. The
defense-in-depth ownerTag !== ownerHash check in list-webhooks.ts
doesn't catch it because both sides equal 'anon' — every Clerk-session
holder could enumerate / overwrite every other Clerk-session pro
tenant's registered webhook URLs.

Fix: reinstate validateApiKey(ctx.request, { forceKey: true }) at the
top of each handler, throwing ApiError(401) when absent. Matches the
sibling routes exactly and the published partner contract.

Tests:
- tests/shipping-v2-handler.test.mjs: two existing "non-PRO → 403"
  tests for register/list were using makeCtx() with no key, which now
  fails at the 401 layer first. Renamed to "no API key → 401
  (tenant-isolation gate)" with a comment explaining the failure mode
  being tested. 18/18 pass.

Verified: typecheck:api, lint:api-contract (no change), lint:boundaries,
lint:rate-limit-policies, test:data (6005/6005).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(review HIGH 2/2): restore v1 path aliases on scenario + supply-chain (koala73#3207)

Koala flagged this as a merge blocker in PR koala73#3242 review.

Commits 6 + 7 of koala73#3207 renamed five documented v1 URLs to the sebuf
method-derived paths and deleted the legacy edge-function files:

  POST /api/scenario/v1/run                       → run-scenario
  GET  /api/scenario/v1/status                    → get-scenario-status
  GET  /api/scenario/v1/templates                 → list-scenario-templates
  GET  /api/supply-chain/v1/country-products      → get-country-products
  GET  /api/supply-chain/v1/multi-sector-cost-shock → get-multi-sector-cost-shock

server/router.ts is an exact static-match table (Map keyed on `METHOD
PATH`), so any external caller — docs, partner scripts, grep-the-
internet — hitting the old documented URL would 404 on first request
after merge. Commit 8 (shipping/v2) preserved partner URLs byte-for-
byte; the scenario + supply-chain renames missed that discipline.

Fix: add five thin alias edge functions that rewrite the pathname to
the canonical sebuf path and delegate to the domain [rpc].ts gateway
via a new server/alias-rewrite.ts helper. Premium gating, rate limits,
entitlement checks, and cache-tier lookups all fire on the canonical
path — aliases are pure URL rewrites, not a duplicate handler pipeline.

  api/scenario/v1/{run,status,templates}.ts
  api/supply-chain/v1/{country-products,multi-sector-cost-shock}.ts

Vite dev parity: file-based routing at api/ is a Vercel concern, so the
dev middleware (vite.config.ts) gets a matching V1_ALIASES rewrite map
before the router dispatch.

Manifest: 5 new entries under `deferred` with removal_issue=koala73#3282
(tracking their retirement at the next v1→v2 break). lint:api-contract
stays green (89 files checked, 55 manifest entries validated).

Docs:
- docs/api-scenarios.mdx: migration callout at the top with the full
  old→new URL table and a link to the retirement issue.
- CHANGELOG.md + docs/changelog.mdx: Changed entry documenting the
  rename + alias compat + the 202→200 shift (from commit 23c821a).

Verified: typecheck:api, lint:api-contract, lint:rate-limit-policies,
lint:boundaries, test:data (6005/6005).

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>

…koala73#3358)

* fix(insights): trust cluster rank, stop LLM from re-picking top story

WORLD BRIEF panel published "Iran's new supreme leader was seriously
wounded, leading him to delegate power to the Revolutionary Guards. This
development comes amid an ongoing war with Israel." to every visitor for
3h. Payload: openrouter / gemini-2.5-flash.

Root cause: callLLM sent all 10 clustered headlines with "pick the ONE
most significant and summarize ONLY that story". Clustering ranked
Lebanon journalist killing #1 (2 corroborating sources); News24 Iran
rumor ranked #3 (1 source). Gemini overrode the rank, picked #3, and
embellished with war framing from story #4. Objective rank (sourceCount,
velocity, isAlert) lost to model vibe.

Shrink the LLM's job to phrasing. Clustering already ranks — pass only
topStories[0].primaryTitle and instruct the model to rewrite it using
ONLY facts from the headline. No name/place/context invention.

Also:
- temperature 0.3 -> 0.1 (factual summary, not creative)
- CACHE_TTL 3h -> 30m so a bad brief ages out in one cron cycle
- Drop dead MAX_HEADLINES const

Payload shape unchanged; frontend untouched.

* fix(insights): corroboration gate + revert TTL + drop unconditional WHERE

Follow-up to review feedback on the ranking contract, TTL, and prompt:

1. Corroboration gate (P1a). scoreImportance() in scripts/_clustering.mjs
   is keyword-heavy (violence +125 on a single word, flashpoint +75, ^1.5
   multiplier when both hit), so a single-source sensational rumor can
   outrank a 2-source lead purely on lexical signals. Blindly trusting
   topStories[0] would let the ranker's keyword bias still pick bad
   stories. Walk topStories for sourceCount >= 2 instead — corroboration
   becomes a hard requirement, not a tiebreaker. If no cluster qualifies,
   publish status=degraded with no brief (frontend already handles this).

2. CACHE_TTL back to 10800 (P1b). 30m TTL == one cron cadence means the
   key expires on any missed or delayed run and /api/bootstrap loses
   insights entirely (api/bootstrap.js reads news:insights:v1 directly,
   no LKG across TTL-gap). The short TTL was defense-in-depth for bad
   content; the real safety is now upstream (corroboration gate + grounded
   prompt), so the LKG window doesn't need to be sacrificed for it.

3. Prompt: location conditional (P2). "Use ONLY facts present" + "Lead
   with WHAT happened and WHERE" conflicted for headlines without an
   explicit location and pushed the model toward inferred-place
   hallucination. Replaced with "Include a location, person, or
   organization ONLY if it appears in the headline."

* test(insights): lock corroboration gate + grounded-prompt invariants

Review P2: the corroboration gate and the prompt's no-invention rules
had no tests, so future edits to selectTopStories() ordering or prompt
text could silently reintroduce the original hallucination.

Extract the brief-selection helper and prompt builders into a pure
module (scripts/_insights-brief.mjs) so tests can import them without
triggering seed-insights.mjs's top-level runSeed() call:

- pickBriefCluster(topStories) returns first sourceCount>=2 cluster
- briefSystemPrompt(dateISO) returns the system prompt
- briefUserPrompt(headline) returns the user prompt

Regression tests (tests/seed-insights-brief.test.mjs, 12 cases) lock:
- pickBriefCluster skips single-source rumors even when ranked above a
  multi-sourced lead (explicit regression: News24 Iran supreme leader
  2026-04-23 scenario with realistic scores)
- pickBriefCluster tolerates missing/null entries
- briefSystemPrompt forbids invented facts and proper nouns
- briefSystemPrompt's "location" rule is conditional (no unconditional
  "Lead with WHAT and WHERE" directive that would push the model toward
  place-inference when the headline has no location)
- briefSystemPrompt does not contain "pick the most important" style
  language (ranking is done by pickBriefCluster upstream)
- briefUserPrompt passes the headline verbatim and instructs
  "only facts from this headline"

Also fix a misleading comment on CACHE_TTL: corroboration is gated at
brief-selection time, not on the topStories payload itself (which still
includes single-source clusters rendered as the headline list).

test:data: 6657/6657 pass (was 6645; +12).

…ala73#3370)

* feat(news/parser): extract RSS/Atom description for LLM grounding (U1)

Add description field to ParsedItem, extract from the first non-empty of
description/content:encoded (RSS) or summary/content (Atom), picking the
longest after HTML-strip + entity-decode + whitespace-normalize. Clip to
400 chars. Reject empty, <40 chars after strip, or normalize-equal to the
headline — downstream consumers fall back to the cleaned headline on '',
preserving current behavior for feeds without a description.

CDATA end is anchored to the closing tag so internal ]]> sequences do not
truncate the match. Preserves cached rss:feed:v1 row compatibility during
the 1h TTL bleed since the field is additive.

Part of fix: pipe RSS description end-to-end so LLM surfaces stop
hallucinating named actors (docs/plans/2026-04-24-001-...).

Covers R1, R7.

* feat(news/story-track): persist description on story:track:v1 HSET (U2)

Append description to the story:track:v1 HSET only when non-empty. Additive
— no key version bump. Old rows and rows from feeds without a description
return undefined on HGETALL, letting downstream readers fall back to the
cleaned headline (R6).

Extract buildStoryTrackHsetFields as a pure helper so the inclusion gate is
unit-testable without Redis.

Update the contract comment in cache-keys.ts so the next reader of the
schema sees description as an optional field.

Covers R2, R6.

* feat(proto): NewsItem.snippet + SummarizeArticleRequest.bodies (U3)

Add two additive proto fields so the article description can ride to every
LLM-adjacent consumer without a breaking change:

- NewsItem.snippet (field 12): RSS/Atom description, HTML-stripped,
  ≤400 chars, empty when unavailable. Wired on toProtoItem.
- SummarizeArticleRequest.bodies (field 8): optional article bodies
  paired 1:1 with headlines for prompt grounding. Empty array is today's
  headline-only behavior.

Regenerated TS client/server stubs and OpenAPI YAML/JSON via sebuf v0.11.1
(PATH=~/go/bin required — Homebrew's protoc-gen-openapiv3 is an older
pre-bundle-mode build that collides on duplicate emission).

Pre-emptive bodies:[] placeholders at the two existing SummarizeArticle
call sites in src/services/summarization.ts; U6 replaces them with real
article bodies once SummarizeArticle handler reads the field.

Covers R3, R5.

* feat(brief/digest): forward RSS description end-to-end through brief envelope (U4)

Digest accumulator reader (seed-digest-notifications.mjs::buildDigest) now
plumbs the optional `description` field off each story:track:v1 HGETALL into
the digest story object. The brief adapter (brief-compose.mjs::
digestStoryToUpstreamTopStory) prefers the real RSS description over the
cleaned headline; when the upstream row has no description (old rows in the
48h bleed, feeds that don't carry one), we fall back to the cleaned headline
so today behavior is preserved (R6).

This is the upstream half of the description cache path. U5 lands the LLM-
side grounding + cache-prefix bump so Gemini actually sees the article body
instead of hallucinating a named actor from the headline.

Covers R4 (upstream half), R6.

* feat(brief/llm): RSS grounding + sanitisation + 4 cache prefix bumps (U5)

The actual fix for the headline-only named-actor hallucination class:
Gemini 2.5 Flash now receives the real article body as grounding context,
so it paraphrases what the article says instead of filling role-label
headlines from parametric priors ("Iran's new supreme leader" → "Ali
Khamenei" was the 2026-04-24 reproduction; with grounding, it becomes
the actual article-named actor).

Changes:

- buildStoryDescriptionPrompt interpolates a `Context: <body>` line
  between the metadata block and the "One editorial sentence" instruction
  when description is non-empty AND not normalise-equal to the headline.
  Clips to 400 chars as a second belt-and-braces after the U1 parser cap.
  No Context line → identical prompt to pre-fix (R6 preserved).

- sanitizeStoryForPrompt extended to cover `description`. Closes the
  asymmetry where whyMatters was sanitised and description wasn't —
  untrusted RSS bodies now flow through the same injection-marker
  neutraliser before prompt interpolation. generateStoryDescription wraps
  the story in sanitizeStoryForPrompt before calling the builder,
  matching generateWhyMatters.

- Four cache prefixes bumped atomically to evict pre-grounding rows:
    scripts/lib/brief-llm.mjs:
      brief:llm:description:v1 → v2  (Railway, description path)
      brief:llm:whymatters:v2 → v3   (Railway, whyMatters fallback)
    api/internal/brief-why-matters.ts:
      brief:llm:whymatters:v6 → v7                (edge, primary)
      brief:llm:whymatters:shadow:v4 → shadow:v5  (edge, shadow)
  hashBriefStory already includes description in the 6-field material
  (v5 contract) so identity naturally drifts; the prefix bump is the
  belt-and-braces that guarantees a clean cold-start on first tick.

- Tests: 8 new + 2 prefix-match updates on tests/brief-llm.test.mjs.
  Covers Context-line injection, empty/dup-of-headline rejection,
  400-char clip, sanitisation of adversarial descriptions, v2 write,
  and legacy-v1 row dark (forced cold-start).

Covers R4 + new sanitisation requirement.

* feat(news/summarize): accept bodies + bump summary cache v5→v6 (U6)

SummarizeArticle now grounds on per-headline article bodies when callers
supply them, so the dashboard "News summary" path stops hallucinating
across unrelated headlines when the upstream RSS carried context.

Three coordinated changes:

1. SummarizeArticleRequest handler reads req.bodies, sanitises each entry
   through sanitizeForPrompt (same trust treatment as geoContext — bodies
   are untrusted RSS text), clips to 400 chars, and pads to the headlines
   length so pair-wise identity is stable.

2. buildArticlePrompts accepts optional bodies and interleaves a
   `    Context: <body>` line under each numbered headline that has a
   non-empty body. Skipped in translate mode (headline[0]-only) and when
   all bodies are empty — yielding a byte-identical prompt to pre-U6
   for every current caller (R6 preserved).

3. summary-cache-key bumps CACHE_VERSION v5→v6 so the pre-grounding rows
   (produced from headline-only prompts) cold-start cleanly. Extends
   canonicalizeSummaryInputs + buildSummaryCacheKey with a pair-wise
   bodies segment `:bd<hash>`; the prefix is `:bd` rather than `:b` to
   avoid colliding with `:brief:` when pattern-matching keys. Translate
   mode is headline[0]-only and intentionally does not shift on bodies.

Dedup reorder preserved: the handler re-pairs bodies to the deduplicated
top-5 via findIndex, so layout matches without breaking cache identity.

New tests: 7 on buildArticlePrompts (bodies interleave, partial fill,
translate-mode skip, clip, short-array tolerance), 8 on
buildSummaryCacheKey (pair-wise sort, cache-bust on body drift, translate
skip). Existing summary-cache-key assertions updated v5→v6.

Covers R3, R4.

* feat(consumers): surface RSS snippet across dashboard, email, relay, MCP + audit (U7)

Thread the RSS description from the ingestion path (U1-U5) into every
user-facing LLM-adjacent surface. Audit the notification producers so
RSS-origin and domain-origin events stay on distinct contracts.

Dashboard (proto snippet → client → panel):
- src/types/index.ts NewsItem.snippet?:string (client-side field).
- src/app/data-loader.ts proto→client mapper propagates p.snippet.
- src/components/NewsPanel.ts renders snippet as a truncated (~200 chars,
  word-boundary ellipsis) `.item-snippet` line under each headline.
- NewsPanel.currentBodies tracks per-headline bodies paired 1:1 with
  currentHeadlines; passed as options.bodies to generateSummary so the
  server-side SummarizeArticle LLM grounds on the article body.

Summary plumbing:
- src/services/summarization.ts threads bodies through SummarizeOptions
  → generateSummary → runApiChain → tryApiProvider; cache key now includes
  bodies (via U6's buildSummaryCacheKey signature).

MCP world-brief:
- api/mcp.ts pairs headlines with their RSS snippets and POSTs `bodies`
  to /api/news/v1/summarize-article so the MCP tool surface is no longer
  starved.

Email digest:
- scripts/seed-digest-notifications.mjs plain-text formatDigest appends
  a ~200-char truncated snippet line under each story; HTML formatDigestHtml
  renders a dim-grey description div between title and meta. Both gated
  on non-empty description (R6 — empty → today's behavior).

Real-time alerts:
- src/services/breaking-news-alerts.ts BreakingAlert gains optional
  description; checkBatchForBreakingAlerts reads item.snippet; dispatchAlert
  includes `description` in the /api/notify payload when present.

Notification relay:
- scripts/notification-relay.cjs formatMessage gated on
  NOTIFY_RELAY_INCLUDE_SNIPPET=1 (default off). When on, RSS-origin
  payloads render a `> <snippet>` context line under the title. When off
  or payload.description absent, output is byte-identical to pre-U7.

Audit (RSS vs domain):
- tests/notification-relay-payload-audit.test.mjs enforces file-level
  @notification-source tags on every producer, rejects `description:` in
  domain-origin payload blocks, and verifies the relay codepath gates
  snippet rendering under the flag.
- Tag added to ais-relay.cjs (domain), seed-aviation.mjs (domain),
  alert-emitter.mjs (domain), breaking-news-alerts.ts (rss).

Deferred (plan explicitly flags): InsightsPanel + cluster-producer
plumbing (bodies default to [] — will unlock gradually once news:insights:v1
producer also carries primarySnippet).

Covers R5, R6.

* docs+test: grounding-path note + bump pinned CACHE_VERSION v5→v6 (U8)

Final verification for the RSS-description-end-to-end fix:

- docs/architecture.mdx — one-paragraph "News Grounding Pipeline"
  subsection tracing parser → story:track:v1.description → NewsItem.snippet
  → brief / SummarizeArticle / dashboard / email / relay / MCP, with the
  empty-description R6 fallback rule called out explicitly.
- tests/summarize-reasoning.test.mjs — Fix-4 static-analysis pin updated
  to match the v6 bump from U6. Without this the summary cache bump silently
  regressed CI's pinned-version assertion.

Final sweep (2026-04-24):
- grep -rn 'brief:llm:description:v1' → only in the U5 legacy-row test
  simulation (by design: proves the v2 bump forces cold-start).
- grep -rn 'brief:llm:whymatters:v2/v6/shadow:v4' → no live references.
- grep -rn 'summary:v5' → no references.
- CACHE_VERSION = 'v6' in src/utils/summary-cache-key.ts.
- Full tsx --test sweep across all tests/*.test.{mjs,mts}: 6747/6747 pass.
- npm run typecheck + typecheck:api: both clean.

Covers R4, R6, R7.

* fix(rss-description): address /ce:review findings before merge

14 fixes from structured code review across 13 reviewer personas.

Correctness-critical (P1 — fixes that prevent R6/U7 contract violations):
- NewsPanel signature covers currentBodies so view-mode toggles that leave
  headlines identical but bodies different now invalidate in-flight summaries.
  Without this, switching renderItems → renderClusters mid-summary let a
  grounded response arrive under a stale (now-orphaned) cache key.
- summarize-article.ts re-pairs bodies with headlines BEFORE dedup via a
  single zip-sanitize-filter-dedup pass. Previously bodies[] was indexed by
  position in light-sanitized headlines while findIndex looked up the
  full-sanitized array — any headline that sanitizeHeadlines emptied
  mispaired every subsequent body, grounding the LLM on the wrong story.
- Client skips the pre-chain cache lookup when bodies are present, since
  client builds keys from RAW bodies while server sanitizes first. The
  keys diverge on injection content, which would silently miss the
  server's authoritative cache every call.

Test + audit hardening:
- Legacy v1 eviction test now uses the real hashBriefStory(story()) suffix
  instead of a literal "somehash", so a bug where the reader still queried
  the v1 prefix at the real key would actually be caught.
- tests/summary-cache-key.test.mts adds 400-char clip identity coverage so
  the canonicalizer's clip and any downstream clip can't silently drift.
- tests/news-rss-description-extract.test.mts renames the well-formed
  CDATA test and adds a new test documenting the malformed-]]> fallback
  behavior (plain regex captures, article content survives).

Safe_auto cleanups:
- Deleted dead SNIPPET_PUSH_MAX constant in notification-relay.cjs.
- BETA-mode groq warm call now passes bodies, warming the right cache slot.
- seed-digest shares a local normalize-equality helper for description !=
  headline comparison, matching the parser's contract.
- Pair-wise sort in summary-cache-key tie-breaks on body so duplicate
  headlines produce stable order across runs.
- buildSummaryCacheKey gained JSDoc documenting the client/server contract
  and the bodies parameter semantics.
- MCP get_world_brief tool description now mentions RSS article-body
  grounding so calling agents see the current contract.
- _shared.ts `opts.bodies![i]!` double-bang replaced with `?? ''`.
- extractRawTagBody regexes cached in module-level Map, mirroring the
  existing TAG_REGEX_CACHE pattern.

Deferred to follow-up (tracked for PR description / separate issue):
- Promote shared MAX_BODY constant across the 5 clip sites
- Promote shared truncateForDisplay helper across 4 render sites
- Collapse NewsPanel.{currentHeadlines, currentBodies} → Array<{title, snippet}>
- Promote sanitizeStoryForPrompt to shared/brief-llm-core.js
- Split list-feed-digest.ts parser helpers into sibling -utils.ts
- Strengthen audit test: forward-sweep + behavioral gate test

Tests: 6749/6749 pass. Typecheck clean on both configs.

* fix(summarization): thread bodies through browser T5 path (Codex #2)

Addresses the second of two Codex-raised findings on PR koala73#3370:

The PR threaded bodies through the server-side API provider chain
(Ollama → Groq → OpenRouter → /api/news/v1/summarize-article) but the
local browser T5 path at tryBrowserT5 was still summarising from
headlines alone. In BETA_MODE that ungrounded path runs BEFORE the
grounded server providers; in normal mode it remains the last
fallback. Whenever T5-small won, the dashboard summary surface
regressed to the headline-only path — the exact hallucination class
this PR exists to eliminate.

Fix: tryBrowserT5 accepts an optional `bodies` parameter and
interleaves each body with its paired headline via a `headline —
body` separator in the combined text (clipped to 200 chars per body
to stay within T5-small's ~512-token context window). All three call
sites (BETA warm, BETA cold, normal-mode fallback) now pass the
bodies threaded down from generateSummary options.bodies.

When bodies is empty/omitted, the combined text is byte-identical to
pre-fix (R6 preserved).

On Codex finding #1 (story:track:v1 additive-only HSET keeps a body
from an earlier mention of the same normalized title), declining to
change. The current rule — "if this mention has a body, overwrite;
otherwise leave the prior body alone" — is defensible: a body from
mention A is not falsified by mention B being body-less (a wire
reprint doesn't invalidate the original source's body). A feed that
publishes a corrected headline creates a new normalized-title hash,
so no stale body carries forward. The failure window is narrow (live
story evolving while keeping the same title through hours of
body-less wire reprints) and the 7-day STORY_TTL is the backstop.
Opening a follow-up issue to revisit semantics if real-world evidence
surfaces a stale-grounding case.

* fix(story-track): description always-written to overwrite stale bodies (Codex #1)

Revisiting Codex finding #1 on PR koala73#3370 after re-review. The previous
response declined the fix with reasoning; on reflection the argument
was over-defending the current behavior.

Problem: buildStoryTrackHsetFields previously wrote `description` only
when non-empty. Because story:track:v1 rows are collapsed by
normalized-title hash, an earlier mention's body would persist for up
to STORY_TTL (7 days) on subsequent body-less mentions of the same
story. Consumers reading `track.description` via HGETALL could not
distinguish "this mention's body" from "some mention's body from the
last week," silently grounding brief / whyMatters / SummarizeArticle
LLMs on text the current mention never supplied. That violates the
grounding contract advertised to every downstream surface in this PR.

Fix: HSET `description` unconditionally on every mention — empty
string when the current item has no body, real body when it does. An
empty value overwrites any prior mention's body so the row is always
authoritative for the current cycle. Consumers continue to treat
empty description as "fall back to cleaned headline" (R6 preserved).
The 7-day STORY_TTL and normalized-title hash semantics are unchanged.

Trade-off accepted: a valid body from Feed A (NYT) is wiped when Feed
B (AP body-less wire reprint) arrives for the same normalized title,
even though Feed A's body is factually correct. Rationale: the
alternative — keeping Feed A's body indefinitely — means the user
sees Feed A's body attributed (by proximity) to an AP mention at a
later timestamp, which is at minimum misleading and at worst carries
retracted/corrected details. Honest absence beats unlabeled presence.

Tests: new stale-body overwrite sequence test (T0 body → T1 empty →
T2 new body), existing "writes description when non-empty" preserved,
existing "omits when empty" inverted to "writes empty, overwriting."
cache-keys.ts contract comment updated to mark description as
always-written rather than optional.

)

* chore(api-manifest): rewrite brief-why-matters reason as proper internal-helper justification

Carried in from koala73#3248 merge as a band-aid (called out in koala73#3242 review followup
checklist item 7). The endpoint genuinely belongs in internal-helper —
RELAY_SHARED_SECRET-bearer auth, cron-only caller, never reached by dashboards
or partners. Same shape constraint as api/notify.ts.

Replaces the apologetic "filed here to keep the lint green" framing with a
proper structural justification: modeling it as a generated service would
publish internal cron plumbing as user-facing API surface.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* feat(lint): premium-fetch parity check for ServiceClients (closes koala73#3279)

Adds scripts/enforce-premium-fetch.mjs — AST-walks src/, finds every
`new <ServiceClient>(...)` (variable decl OR `this.foo =` assignment),
tracks which methods each instance actually calls, and fails if any
called method targets a path in src/shared/premium-paths.ts
PREMIUM_RPC_PATHS without `{ fetch: premiumFetch }` on the constructor.

Per-call-site analysis (not class-level) keeps the trade/index.ts pattern
clean — publicClient with globalThis.fetch + premiumClient with
premiumFetch on the same TradeServiceClient class — since publicClient
never calls a premium method.

Wired into:
- npm run lint:premium-fetch
- .husky/pre-push (right after lint:rate-limit-policies)
- .github/workflows/lint-code.yml (right after lint:api-contract)

Found and fixed three latent instances of the HIGH(new) #1 class from
koala73#3242 review (silent 401 → empty fallback for signed-in browser pros):

- src/services/correlation-engine/engine.ts — IntelligenceServiceClient
  built with no fetch option called deductSituation. LLM-assessment overlay
  on convergence cards never landed for browser pros without a WM key.
- src/services/economic/index.ts — EconomicServiceClient with
  globalThis.fetch called getNationalDebt. National-debt panel rendered
  empty for browser pros.
- src/services/sanctions-pressure.ts — SanctionsServiceClient with
  globalThis.fetch called listSanctionsPressure. Sanctions-pressure panel
  rendered empty for browser pros.

All three swap to premiumFetch (single shared client, mirrors the
supply-chain/index.ts justification — premiumFetch no-ops safely on
public methods, so the public methods on those clients keep working).

Verification:
- lint:premium-fetch clean (34 ServiceClient classes, 28 premium paths,
  466 src/ files analyzed)
- Negative test: revert any of the three to globalThis.fetch → exit 1
  with file:line and called-premium-method names
- typecheck + typecheck:api clean
- lint:api-contract / lint:rate-limit-policies / lint:boundaries clean
- tests/sanctions-pressure.test.mjs + premium-fetch.test.mts: 16/16 pass

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(military): fetchStaleFallback NEG_TTL=30s parity (closes koala73#3277)

The legacy /api/military-flights handler had NEG_TTL = 30_000ms — a short
suppression window after a failed live + stale read so we don't Redis-hammer
the stale key during sustained relay+seed outages.

Carried into the sebuf list-military-flights handler:
- Module-scoped `staleNegUntil` timestamp (per-isolate on Vercel Edge,
  which is fine — each warm isolate gets its own 30s suppression window).
- Set whenever fetchStaleFallback returns null (key missing, parse fail,
  empty array after staleToProto filter, or thrown error).
- Checked at the entry of fetchStaleFallback before doing the Redis read.
- Test seam `_resetStaleNegativeCacheForTests()` exposed for unit tests.

Test pinned in tests/redis-caching.test.mjs: drives a stale-empty cycle
three times — first read hits Redis, second within window doesn't, after
test-only reset it does again.

Verified: 18/18 redis-caching tests pass, typecheck:api clean,
lint:premium-fetch clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor(lint): rate-limit-policies regex → import() (closes koala73#3278)

The previous lint regex-parsed ENDPOINT_RATE_POLICIES from the source
file. That worked because the literal happens to fit a single line per
key today, but a future reformat (multi-line key wrap, formatter swap,
etc.) would silently break the lint without breaking the build —
exactly the failure mode that's worse than no lint at all.

Fix:
- Export ENDPOINT_RATE_POLICIES from server/_shared/rate-limit.ts.
- Convert scripts/enforce-rate-limit-policies.mjs to async + dynamic
  import() of the policy object directly. Same TS module that the
  gateway uses at runtime → no source-of-truth drift possible.
- Run via tsx (already a dev dep, used by test:data) so the .mjs
  shebang can resolve a .ts import.
- npm script swapped to `tsx scripts/...`. .husky/pre-push uses
  `npm run lint:rate-limit-policies` so no hook change needed.

Verified:
- Clean: 6 policies / 182 gateway routes.
- Negative test (rename a key to the original sanctions typo
  /api/sanctions/v1/lookup-entity): exit 1 with the same incident-
  attributed remedy message as before.
- Reformat test (split a single-line entry across multiple lines):
  still passes — the property is what's read, not the source layout.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* fix(shipping/v2): alertThreshold: 0 preserved; drop dead validation branch (koala73#3242 followup)

Before: alert_threshold was a plain int32. proto3 scalar default is 0, so
the handler couldn't distinguish "partner explicitly sent 0 (deliver every
disruption)" from "partner omitted the field (apply legacy default 50)" —
both arrived as 0 and got coerced to 50 by `> 0 ? : 50`. Silent intent-drop
for any partner who wanted every alert. The subsequent `alertThreshold < 0`
branch was also unreachable after that coercion.

After:
- Proto field is `optional int32 alert_threshold` — TS type becomes
  `alertThreshold?: number`, so omitted = undefined and explicit 0 stays 0.
- Handler uses `req.alertThreshold ?? 50` — undefined → 50, any number
  passes through unchanged.
- Dead `< 0 || > 100` runtime check removed; buf.validate `int32.gte = 0,
  int32.lte = 100` already enforces the range at the wire layer.

Partner wire contract: identical for the omit-field and 1..100 cases.
Only behavioural change is explicit 0 — previously impossible to request,
now honored per proto3 optional semantics.

Scoped `buf generate --path worldmonitor/shipping/v2` to avoid the full-
regen `@ts-nocheck` drift Seb documented in the koala73#3242 PR comments.
Re-applied `@ts-nocheck` on the two regenerated files manually.

Tests:
- `alertThreshold 0 coerces to 50` flipped to `alertThreshold 0 preserved`.
- New test: `alertThreshold omitted (undefined) applies legacy default 50`.
- `rejects > 100` test removed — proto/wire validation handles it; direct
  handler calls intentionally bypass wire and the handler no longer carries
  a redundant runtime range check.

Verified: 18/18 shipping-v2-handler tests pass, typecheck + typecheck:api
clean, all 4 custom lints clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* docs(shipping/v2): document missing webhook delivery worker + DNS-rebinding contract (koala73#3242 followup)

koala73#3242 followup checklist item 6 from @koala73 — sanity-check that the
delivery worker honors the re-resolve-and-re-check contract that
isBlockedCallbackUrl explicitly delegates to it.

Audit finding: no delivery worker for shipping/v2 webhooks exists in this
repo. Grep across the entire tree (excluding generated/dist) shows the
only readers of webhook:sub:* records are the registration / inspection /
rotate-secret handlers themselves. No code reads them and POSTs to the
stored callbackUrl. The delivery worker is presumed to live in Railway
(separate repo) or hasn't been built yet — neither is auditable from
this repo.

Refreshes the comment block at the top of webhook-shared.ts to:
- explicitly state DNS rebinding is NOT mitigated at registration
- spell out the four-step contract the delivery worker MUST follow
  (re-validate URL, dns.lookup, re-check resolved IP against patterns,
   fetch with resolved IP + Host header preserved)
- flag the in-repo gap so anyone landing delivery code can't miss it

Tracking the gap as koala73#3288 — acceptance there is "delivery worker imports
the patterns + helpers from webhook-shared.ts and applies the four steps
before each send." Action moves to wherever the delivery worker actually
lives (Railway likely).

No code change. Tests + lints unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* ci(lint): add rate-limit-policies step (greptile P1 koala73#3287)

Pre-push hook ran lint:rate-limit-policies but the CI workflow did not,
so fork PRs and --no-verify pushes bypassed the exact drift check the
lint was added to enforce (closes koala73#3278). Adding it right after
lint:api-contract so it runs in the same context the lint was designed
for.

* refactor(lint): premium-fetch regex → import() + loop classRe (greptile P2 koala73#3287)

Two fragilities greptile flagged on enforce-premium-fetch.mjs:

1. loadPremiumPaths regex-parsed src/shared/premium-paths.ts with
   /'(\/api\/[^']+)'/g — same class of silent drift we just removed
   from enforce-rate-limit-policies in koala73#3278. Reformatting the source
   Set (double quotes, spread, helper-computed entries) would drop
   paths from the lint while leaving the runtime untouched. Fix: flip
   the shebang to `#!/usr/bin/env -S npx tsx` and dynamic-import
   PREMIUM_RPC_PATHS directly, mirroring the rate-limit pattern.
   package.json lint:premium-fetch now invokes via tsx too so the
   npm-script path matches direct execution.

2. loadClientClassMap ran classRe.exec once, silently dropping every
   ServiceClient after the first if a file ever contained more than
   one. Current codegen emits one class per file so this was latent,
   but a template change would ship un-linted classes. Fix: collect
   every class-open match with matchAll, slice each class body with
   the next class's start as the boundary, and scan methods per-body
   so method-to-class binding stays correct even with multiple
   classes per file.

Verification:
- lint:premium-fetch clean (34 classes / 28 premium paths / 466 files
  — identical counts to pre-refactor, so no coverage regression).
- Negative test: revert src/services/economic/index.ts to
  globalThis.fetch → exit 1 with file:line, bound var name, and
  premium method list (getNationalDebt). Restore → clean.
- lint:rate-limit-policies still clean.

* fix(shipping/v2): re-add alertThreshold handler range guard (greptile nit 1 koala73#3287)

Wire-layer buf.validate enforces 0..100, but direct handler invocation
(internal jobs, test harnesses, future transports) bypasses it. Cheap
invariant-at-the-boundary — rejects < 0 or > 100 with ValidationError
before the record is stored.

Tests: restored the rejects-out-of-range cases that were dropped when the
branch was (correctly) deleted as dead code on the previous commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor(lint): premium-fetch method-regex → TS AST (greptile nits 2+5 koala73#3287)

loadClientClassMap:
  The method regex `async (\w+)\s*\([^)]*\)\s*:\s*Promise<[^>]+>\s*\{\s*let
  path = "..."` assumed (a) no nested `)` in arg types, (b) no nested `>`
  in the return type, (c) `let path = "..."` as the literal first statement.
  Any codegen template shift would silently drop methods with the lint still
  passing clean — the same silent-drift class koala73#3287 just closed on the
  premium-paths side.

  Now walks the service_client.ts AST, matches `export class *ServiceClient`,
  iterates `MethodDeclaration` members, and reads the first
  `let path: string = '...'` variable statement as a StringLiteral. Tolerant
  to any reformatting of arg/return types or method shape.

findCalls scope-blindness:
  Added limitation comment — the walker matches `<varName>.<method>()`
  anywhere in the file without respecting scope. Two constructions in
  different function scopes sharing a var name merge their called-method
  sets. No current src/ file hits this; the lint errs cautiously (flags
  both instances). Keeping the walker simple until scope-aware binding
  is needed.

webhook-shared.ts:
  Inlined issue reference (koala73#3288) so the breadcrumb resolves without
  bouncing through an MDX that isn't in the diff.

Verification:
- lint:premium-fetch clean — 34 classes / 28 premium paths / 489 files.
  Pre-refactor: 34 / 28 / 466. Class + path counts identical; file bump
  is from the main-branch rebase, not the refactor.
- Negative test: revert src/services/economic/index.ts premiumFetch →
  globalThis.fetch. Lint exits 1 at `src/services/economic/index.ts:64:7`
  with `premium method(s) called: getNationalDebt`. Restore → clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* refactor(lint): rate-limit OpenAPI regex → yaml parser (greptile nit 3 koala73#3287)

Input side (ENDPOINT_RATE_POLICIES) was flipped to live `import()` in
4e79d02. Output side (OpenAPI routes) still regex-scraped top-level
`paths:` keys with `/^\s{4}(\/api\/[^\s:]+):/gm` — hard-coded 4-space
indent. Any YAML formatter change (2-space indent, flow style, line
folding) would silently drop routes and let policy-drift slip through
— same silent-drift class the input-side fix closed.

Now uses the `yaml` package (already a dep) to parse each
.openapi.yaml and reads `doc.paths` directly.

Verification:
- Clean: 6 policies / 189 routes (was 182 — yaml parser picks up a
  handful the regex missed, closing a silent coverage gap).
- Negative test: rename policy key back to /api/sanctions/v1/lookup-entity
  → exits 1 with the same incident-attributed remedy. Restore → clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

* chore(codegen): regenerate unified OpenAPI bundle for alert_threshold proto change

The shipping/v2 webhook alert_threshold field was flipped from `int32` to
`optional int32` with an expanded doc comment in f333946. That comment
now surfaces in the unified docs/api/worldmonitor.openapi.yaml bundle
(introduced by koala73#3341). Regenerated with sebuf v0.11.1 to pick it up.

No behaviour change — bundle-only documentation drift.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>

…read (koala73#3385)

* feat(seed): BUNDLE_RUN_STARTED_AT_MS env + runSeed SIGTERM cleanup

Prereq for the re-export-share Comtrade seeder (plan 2026-04-24-003),
usable by any cohort seeder whose consumer needs bundle-level freshness.

Two coupled changes:

1. `_bundle-runner.mjs` injects `BUNDLE_RUN_STARTED_AT_MS` into every
   spawned child. All siblings in a single bundle run share one value
   (captured at `runBundle` start, not spawn time). Consumers use this
   to detect stale peer keys — if a peer's seed-meta predates the
   current bundle run, fall back to a hard default rather than read
   a cohort-peer's last-week output.

2. `_seed-utils.mjs::runSeed` registers a `process.once('SIGTERM')`
   handler that releases the acquired lock and extends existing-data
   TTL before exiting 143. `_bundle-runner.mjs` sends SIGTERM on
   section timeout, then SIGKILL after KILL_GRACE_MS (5s). Without
   this handler the `finally` path never runs on SIGKILL, leaving
   the 30-min acquireLock reservation in place until its own TTL
   expires — the next cron tick silently skips the resource.

Regression guard memory: `bundle-runner-sigkill-leaks-child-lock` (PR
koala73#3128 root cause).

Tests added:
- bundle-runner env injection (value within run bounds)
- sibling sections share the same timestamp (critical for the
  consumer freshness guard)
- runSeed SIGTERM path: exit 143 + cleanup log
- process.once contract: second SIGTERM does not re-enter handler

* fix(seed): address P1/P2 review findings on SIGTERM + bundle contracts

Addresses PR koala73#3384 review findings (todos 256, 257, 259, 260):

koala73#256 (P1) — SIGTERM handler narrowed to fetch phase only. Was installed
at runSeed entry and armed through every `process.exit` path; could
race `emptyDataIsFailure: true` strict-floor exits (IMF-External,
WB-bulk) and extend seed-meta TTL when the contract forbids it —
silently re-masking 30-day outages. Now the handler is attached
immediately before `withRetry(fetchFn)` and removed in a try/finally
that covers all fetch-phase exit branches.

koala73#257 (P1) — `BUNDLE_RUN_STARTED_AT_MS` now has a first-class helper.
Exported `getBundleRunStartedAtMs()` from `_seed-utils.mjs` with JSDoc
describing the bundle-freshness contract. Fleet-wide helper so the
next consumer seeder imports instead of rediscovering the idiom.

koala73#259 (P2) — SIGTERM cleanup runs `Promise.allSettled` on disjoint-key
ops (`releaseLock` + `extendExistingTtl`). Serialising compounded
Upstash latency during the exact failure mode (Redis degraded) this
handler exists to handle, risking breach of the 5s SIGKILL grace.

koala73#260 (P2) — `_bundle-runner.mjs` asserts topological order on
optional `dependsOn` section field. Throws on unknown-label refs and
on deps appearing at a later index. Fleet-wide contract replacing
the previous prose-comment ordering guarantee.

Tests added/updated:
- New: SIGTERM handler removed after fetchFn completes (narrowed-scope
  contract — post-fetch SIGTERM must NOT trigger TTL extension)
- New: dependsOn unknown-label + out-of-order + happy-path (3 tests)

Full test suite: 6,866 tests pass (+4 net).

* fix(seed): getBundleRunStartedAtMs returns null outside a bundle run

Review follow-up: the earlier `Math.floor(Date.now()/1000)*1000` fallback
regressed standalone (non-bundle) runs. A consumer seeder invoked
manually just after its peer wrote `fetchedAt = (now - 5s)` would see
`bundleStartMs = Date.now()`, reject the perfectly-fresh peer envelope
as "stale", and fall back to defaults — defeating the point of the
peer-read path outside the bundle.

Returning null when `BUNDLE_RUN_STARTED_AT_MS` is unset/invalid keeps
the freshness gate scoped to its real purpose (across-bundle-tick
staleness) and lets standalone runs skip the gate entirely. Consumers
check `bundleStartMs != null` before applying the comparison; see the
companion `seed-sovereign-wealth.mjs` change on the stacked PR.

* test(seed): SIGTERM cleanup test now verifies Redis DEL + EXPIRE calls

Greptile review P2 on PR koala73#3384: the existing test only asserted exit
code + log line, not that the Redis ops were actually issued. The
log claim was ahead of the test.

Fixture now logs every Upstash fetch call's shape (EVAL / pipeline-
EXPIRE / other) to stderr. Test asserts:

- >=1 EVAL op was issued during SIGTERM cleanup (releaseLock Lua
  script on the lock key)
- >=1 pipeline-EXPIRE op was issued (extendExistingTtl on canonical
  + seed-meta keys)
- The EVAL body carries the runSeed-generated runId (proves it's
  THIS run's release, not a phantom op)
- The EXPIRE pipeline touches both the canonicalKey AND the
  seed-meta key (proves the keys[] array was built correctly
  including the extraKeys merge path)

Full test suite: 6,866 tests pass, typecheck clean.

* feat(resilience): Comtrade-backed re-export-share seeder + SWF Redis read

Plan ref: docs/plans/2026-04-24-003-feat-reexport-share-comtrade-seeder-plan.md

Motivating case. Before this PR, the SWF `rawMonths` denominator for
the `sovereignFiscalBuffer` dimension used GROSS annual imports for
every country. For re-export hubs (goods transiting without domestic
settlement), this structurally under-reports resilience: UAE's 2023
$941B of imports include $334B of transit flow that never represents
domestic consumption. Net imports = gross × (1 − reexport_share).

The previous (PR 3A) design flattened a hand-curated YAML into Redis;
the YAML shipped empty and never populated, so the correction never
applied and the cohort audit showed no movement.

Gap #2 (this PR). Two coupled changes to make the correction actually
apply:

1. Comtrade-backed seeder (`scripts/seed-recovery-reexport-share.mjs`).
   Rewritten to fetch UN Comtrade `flowCode=RX` (re-exports) and
   `flowCode=M` (imports) per cohort member, compute share = RX/M at
   the latest co-populated year, clamp to [0.05, 0.95], publish the
   envelope. Header auth (`Ocp-Apim-Subscription-Key`) — subscription
   key never reaches URL/logs/Redis. `maxRecords=250000` cap with
   truncation detection. Sequential + retry-on-429 with backoff.

   Hub cohort resolved by Phase 0 empirical probe (plan §Phase 0):
   ['AE', 'PA']. Six candidates (SG/HK/NL/BE/MY/LT) return HTTP 200
   with zero RX rows — Comtrade doesn't expose RX for those reporters.

2. SWF seeder reads from Redis (`scripts/seed-sovereign-wealth.mjs`).
   Swaps `loadReexportShareByCountry()` (YAML) for
   `loadReexportShareFromRedis()` (Redis key written by #1). Guarded
   by bundle-run freshness: if the sibling Reexport-Share seeder's
   `seed-meta` predates `BUNDLE_RUN_STARTED_AT_MS` (set by the
   prereq PR's `_bundle-runner.mjs` env-injection), HARD fallback
   to gross imports rather than apply last-month's stale share.

Health registries. Both new keys registered in BOTH `api/health.js`
SEED_META (60-day alert threshold) and `api/seed-health.js`
SEED_DOMAINS (43200min interval). feedback_two_health_endpoints_must_match.

Bundle wiring. `seed-bundle-resilience-recovery` Reexport-Share
timeout bumped 60s → 300s (Comtrade + retry can take 2-3 min
worst-case). Ordering preserved: Reexport-Share before Sovereign-
Wealth so the SWF seeder reads a freshly-written key in the same
cron tick.

Deletions. YAML + loader + 7 obsolete loader tests removed; single
source of truth is now Comtrade → Redis.

Prereq. Stacks on PR koala73#3384 (feat/bundle-runner-env-sigterm)
which adds BUNDLE_RUN_STARTED_AT_MS env injection + runSeed
SIGTERM cleanup. This PR's bundle-freshness guard depends on
that env variable.

Tests (19 new, 7 deleted, +12 net):
- Pure math: parseComtradeFlowResponse, computeShareFromFlows,
  clampShare, declareRecords + credential-leak source scan (15)
- Integration (Gap #2 regression guards): SWF seeder loadReexport
  ShareFromRedis — fresh/absent/malformed/stale-meta/missing-meta (5)
- Health registry dual-registry drift guard — scoped to this PR's
  keys, respecting pre-existing asymmetry (4)
- Bundle-ordering + timeout assertions (2)

Phase 0 cohort validation committed to plan. Full test suite
passes: 6,881 tests.

* fix(resilience): address P1/P2 review findings — adopt shared helpers, pin freshness boundary

Addresses PR koala73#3385 review findings:

koala73#257 (P1) consumer — `seed-sovereign-wealth.mjs` imports the shared
`getBundleRunStartedAtMs` helper from `_seed-utils.mjs` (added in the
prereq commit) instead of its own `getBundleStartMs`. Single source of
truth for the bundle-freshness contract.

koala73#258 (P2) — `seed-recovery-reexport-share.mjs` isMain guard uses the
canonical `pathToFileURL(process.argv[1]).href === import.meta.url`
form instead of basename-suffix matching. Handles symlinks, case-
different paths on macOS HFS+, and Windows path separators without
string munging.

koala73#260 (P2) consumer — Sovereign-Wealth declares `dependsOn:
['Reexport-Share']` in the bundle spec. `_bundle-runner.mjs` (prereq
commit) now enforces topological order on load and throws on
violation — replaces the previous prose-comment ordering contract.

koala73#261 (P2) — added a test to `tests/seed-sovereign-wealth-reads-redis-
reexport-share.test.mts` pinning the inclusive-boundary semantic:
`fetchedAtMs === bundleStartMs` must be treated as FRESH. Guards
against a future refactor to `<=` that would silently reject peers
writing at the very first millisecond of the bundle run.

Rebased onto updated prereq. Full test suite: 6,886 tests pass (+5 net).

* fix(resilience): freshness gate skipped in standalone mode; meta still required

Review catch: the previous `bundleStartMs = Date.now()` fallback made
standalone/manual `seed-sovereign-wealth.mjs` runs ALWAYS reject any
previously-seeded re-export-share meta as "stale" — even when the
operator ran the Reexport seeder milliseconds beforehand. Defeated
the point of the peer-read path outside the bundle.

With `getBundleRunStartedAtMs()` now returning null outside a bundle
(companion commit on the prereq branch), the consumer only applies
the freshness gate when `bundleStartMs != null`. Standalone runs
accept any `fetchedAt` — the operator is responsible for ordering.

Two guards survive the change:
- Meta MUST exist (absence = peer-outage fail-safe, both modes)
- In-bundle: meta MUST be at or after `BUNDLE_RUN_STARTED_AT_MS`

Two new tests pin both modes:
- standalone: accepts meta written 10 min before this process started
- standalone: still rejects missing meta (peer-outage fail-safe
  survives gate bypass)

Rebased onto updated prereq. Full test suite: 6,888 tests (+2 net).

* fix(resilience): filter world-aggregate Comtrade rows + skip final-retry sleep

Greptile review of PR koala73#3385 flagged two P2s in the Comtrade seeder.

Finding #3 (parseComtradeFlowResponse double-count risk):
`cmdCode=TOTAL` without a partner filter currently returns only
world-aggregate rows in practice — but `parseComtradeFlowResponse`
summed every row unconditionally. A future refactor adding per-
partner querying would silently double-count (world-aggregate row +
partner-level rows for the same year), cutting the derived share in
half with no test signal.

Fix: explicit `partnerCode ∈ {'0', 0, null/undefined}` filter. Matches
current empirical behavior (aggregate-only responses) and makes the
construct robust to a future partner-level query.

Finding #4 (wasted backoff on final retry):
429 and 5xx branches slept `backoffMs` before `continue`, but on
`attempt === RETRY_MAX_ATTEMPTS` the loop condition fails immediately
after — the sleep was pure waste. Added early-return (parallel to the
existing pattern in the network-error catch branch) so the final
attempt exits the retry loop at the first non-success response
without extra latency.

Tests:
- 3 new `parseComtradeFlowResponse` variants: world-only filter,
  numeric-0 partnerCode shape, rows without partnerCode field
- Existing tests updated: the double-count assertion replaced with
  a "per-partner rows must NOT sum into the world-aggregate total"
  assertion that pins the new contract

Rebased onto updated prereq. Full test suite: 6,890 tests (+2 net).

…lead divergence (koala73#3396)

* feat(brief-llm): canonical synthesis prompt + v3 cache key

Extends generateDigestProse to be the single source of truth for
brief executive-summary synthesis (canonicalises what was previously
split between brief-llm's generateDigestProse and seed-digest-
notifications.mjs's generateAISummary). Ports Brain B's prompt
features into buildDigestPrompt:

- ctx={profile, greeting, isPublic} parameter (back-compat: 4-arg
  callers behave like today)
- per-story severity uppercased + short-hash prefix [h:XXXX] so the
  model can emit rankedStoryHashes for stable re-ranking
- profile lines + greeting opener appear only when ctx.isPublic !== true

validateDigestProseShape gains optional rankedStoryHashes (≥4-char
strings, capped to MAX_STORIES_PER_USER × 2). v2-shaped rows still
pass — field defaults to [].

hashDigestInput v3:
- material includes profile-SHA, greeting bucket, isPublic flag,
  per-story hash
- isPublic=true substitutes literal 'public' for userId in the cache
  key so all share-URL readers of the same (date, sensitivity, pool)
  hit ONE cache row (no PII in public cache key)

Adds generateDigestProsePublic(stories, sensitivity, deps) wrapper —
no userId param by design — for the share-URL surface.

Cache prefix bumped brief:llm:digest:v2 → v3. v2 rows expire on TTL.
Per the v1→v2 precedent (see hashDigestInput comment), one-tick cost
on rollout is acceptable for cache-key correctness.

Tests: 72/72 passing in tests/brief-llm.test.mjs (8 new for the v3
behaviors), full data suite 6952/6952.

Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md
Step 1, Codex-approved (5 rounds).

* feat(brief): envelope v3 — adds digest.publicLead for share-URL surface

Bumps BRIEF_ENVELOPE_VERSION 2 → 3. Adds optional
BriefDigest.publicLead — non-personalised executive lead generated
by generateDigestProsePublic (already in this branch from the
previous commit) for the public share-URL surface. Personalised
`lead` is the canonical synthesis for authenticated channels;
publicLead is its profile-stripped sibling so api/brief/public/*
never serves user-specific content (watched assets/regions).

SUPPORTED_ENVELOPE_VERSIONS = [1, 2, 3] keeps v1 + v2 envelopes
in the 7-day TTL window readable through the rollout — the
composer only ever writes the current version, but readers must
tolerate older shapes that haven't expired yet. Same rollout
pattern used at the v1 → v2 bump.

Renderer changes (server/_shared/brief-render.js):
- ALLOWED_DIGEST_KEYS gains 'publicLead' (closed-key-set still
  enforced; v2 envelopes pass because publicLead === undefined is
  the v2 shape).
- assertBriefEnvelope: new isNonEmptyString check on publicLead
  when present. Type contract enforced; absence is OK.

Tests (tests/brief-magazine-render.test.mjs):
- New describe block "v3 publicLead field": v3 envelope renders;
  malformed publicLead rejected; v2 envelope still passes; ad-hoc
  digest keys (e.g. synthesisLevel) still rejected — confirming
  the closed-key-set defense holds for the cron-local-only fields
  the orchestrator must NOT persist.
- BRIEF_ENVELOPE_VERSION pin updated 2 → 3 with rollout-rationale
  comment.

Test results: 182 brief-related tests pass; full data suite
6956/6956.

Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md
Step 2, Codex Round-3 Medium #2.

* feat(brief): synthesis splice + rankedStoryHashes pre-cap re-order

Plumbs the canonical synthesis output (lead, threads, signals,
publicLead, rankedStoryHashes from generateDigestProse) through the
pure composer so the orchestration layer can hand pre-resolved data
into envelope.digest. Composer stays sync / no I/O — Codex Round-2
High #2 honored.

Changes:

scripts/lib/brief-compose.mjs:
- digestStoryToUpstreamTopStory now emits `hash` (the digest story's
  stable identifier, falls back to titleHash when absent). Without
  this, rankedStoryHashes from the LLM has nothing to match against.
- composeBriefFromDigestStories accepts opts.synthesis = {lead,
  threads, signals, rankedStoryHashes?, publicLead?}. When passed,
  splices into envelope.digest after the stub is built. Partial
  synthesis (e.g. only `lead` populated) keeps stub defaults for the
  other fields — graceful degradation when L2 fallback fires.

shared/brief-filter.js:
- filterTopStories accepts optional rankedStoryHashes. New helper
  applyRankedOrder re-orders stories by short-hash prefix match
  BEFORE the cap is applied, so the model's editorial judgment of
  importance survives MAX_STORIES_PER_USER. Stable for ties; stories
  not in the ranking come after in original order. Empty/missing
  ranking is a no-op (legacy callers unchanged).

shared/brief-filter.d.ts:
- filterTopStories signature gains rankedStoryHashes?: string[].
- UpstreamTopStory gains hash?: unknown (carried through from
  digestStoryToUpstreamTopStory).

Tests added (tests/brief-from-digest-stories.test.mjs):
- synthesis substitutes lead/threads/signals/publicLead.
- legacy 4-arg callers (no synthesis) keep stub lead.
- partial synthesis (only lead) keeps stub threads/signals.
- rankedStoryHashes re-orders pool before cap.
- short-hash prefix match (model emits 8 chars; story carries full).
- unranked stories go after in original order.

Test results: 33/33 in brief-from-digest-stories; 182/182 across all
brief tests; full data suite 6956/6956.

Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md
Step 3, Codex Round-2 Low + Round-2 High #2.

* feat(brief): single canonical synthesis per user; rewire all channels

Restructures the digest cron's per-user compose + send loops to
produce ONE canonical synthesis per user per issueSlot — the lead
text every channel (email HTML, plain-text, Telegram, Slack,
Discord, webhook) and the magazine show is byte-identical. This
eliminates the "two-brain" divergence that was producing different
exec summaries on different surfaces (observed 2026-04-25 0802).

Architecture:

composeBriefsForRun (orchestration):
- Pre-annotates every eligible rule with lastSentAt + isDue once,
  before the per-user pass. Same getLastSentAt helper the send loop
  uses so compose + send agree on lastSentAt for every rule.

composeAndStoreBriefForUser (per-user):
- Two-pass winner walk: try DUE rules first (sortedDue), fall back
  to ALL eligible rules (sortedAll) for compose-only ticks.
  Preserves today's dashboard refresh contract for weekly /
  twice_daily users on non-due ticks (Codex Round-4 High #1).
- Within each pass, walk by compareRules priority and pick the
  FIRST candidate with a non-empty pool — mirrors today's behavior
  at scripts/seed-digest-notifications.mjs:1044 and prevents the
  "highest-priority but empty pool" edge case (Codex Round-4
  Medium #2).
- Three-level synthesis fallback chain:
    L1: generateDigestProse(fullPool, ctx={profile,greeting,!public})
    L2: generateDigestProse(envelope-sized slice, ctx={})
    L3: stub from assembleStubbedBriefEnvelope
  Distinct log lines per fallback level so ops can quantify
  failure-mode distribution.
- Generates publicLead in parallel via generateDigestProsePublic
  (no userId param; cache-shared across all share-URL readers).
- Splices synthesis into envelope via composer's optional
  `synthesis` arg (Step 3); rankedStoryHashes re-orders the pool
  BEFORE the cap so editorial importance survives MAX_STORIES.
- synthesisLevel stored in the cron-local briefByUser entry — NOT
  persisted in the envelope (renderer's assertNoExtraKeys would
  reject; Codex Round-2 Medium #5).

Send loop:
- Reads lastSentAt via shared getLastSentAt helper (single source
  of truth with compose flow).
- briefLead = brief?.envelope?.data?.digest?.lead — the canonical
  lead. Passed to buildChannelBodies (text/Telegram/Slack/Discord),
  injectEmailSummary (HTML email), and sendWebhook (webhook
  payload's `summary` field). All-channel parity (Codex Round-1
  Medium #6).
- Subject ternary reads cron-local synthesisLevel: 1 or 2 →
  "Intelligence Brief", 3 → "Digest" (preserves today's UX for
  fallback paths; Codex Round-1 Missing #5).

Removed:
- generateAISummary() — the second LLM call that produced the
  divergent email lead. ~85 lines.
- AI_SUMMARY_CACHE_TTL constant — no longer referenced. The
  digest:ai-summary:v1:* cache rows expire on their existing 1h
  TTL (no cleanup pass).

Helpers added:
- getLastSentAt(rule) — extracted Upstash GET for digest:last-sent
  so compose + send both call one source of truth.
- buildSynthesisCtx(rule, nowMs) — formats profile + greeting for
  the canonical synthesis call. Preserves all today's prefs-fetch
  failure-mode behavior.

Composer:
- compareRules now exported from scripts/lib/brief-compose.mjs so
  the cron can sort each pass identically to groupEligibleRulesByUser.

Test results: full data suite 6962/6962 (was 6956 pre-Step 4; +6
new compose-synthesis tests from Step 3).

Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md
Steps 4 + 4b. Codex-approved (5 rounds).

* fix(brief-render): public-share lead fail-safe — never leak personalised lead

Public-share render path (api/brief/public/[hash].ts → renderer
publicMode=true) MUST NEVER serve the personalised digest.lead
because that string can carry profile context — watched assets,
saved-region names, etc. — written by generateDigestProse with
ctx.profile populated.

Previously: redactForPublic redacted user.name and stories.whyMatters
but passed digest.lead through unchanged. Codex Round-2 High
(security finding).

Now (v3 envelope contract):
- redactForPublic substitutes digest.lead = digest.publicLead when
  the v3 envelope carries one (generated by generateDigestProsePublic
  with profile=null, cache-shared across all public readers).
- When publicLead is absent (v2 envelope still in TTL window OR v3
  envelope where publicLead generation failed), redactForPublic sets
  digest.lead to empty string.
- renderDigestGreeting: when lead is empty, OMIT the <blockquote>
  pull-quote entirely. Page still renders complete (greeting +
  horizontal rule), just without the italic lead block.
- NEVER falls back to the original personalised lead.

assertBriefEnvelope still validates publicLead's contract (when
present, must be a non-empty string) BEFORE redactForPublic runs,
so a malformed publicLead throws before any leak risk.

Tests added (tests/brief-magazine-render.test.mjs):
- v3 envelope renders publicLead in pull-quote, personalised lead
  text never appears.
- v2 envelope (no publicLead) omits pull-quote; rest of page
  intact.
- empty-string publicLead rejected by validator (defensive).
- private render still uses personalised lead.

Test results: 68 brief-magazine-render tests pass; full data suite
remains green from prior commit.

Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md
Step 5, Codex Round-2 High (security).

* feat(digest): brief lead parity log + extra acceptance tests

Adds the parity-contract observability line and supplementary
acceptance tests for the canonical synthesis path.

Parity log (per send, after successful delivery):
  [digest] brief lead parity user=<id> rule=<v>:<s>:<lang>
    synthesis_level=<1|2|3> exec_len=<n> brief_lead_len=<n>
    channels_equal=<bool> public_lead_len=<n>

When channels_equal=false an extra WARN line fires —
"PARITY REGRESSION user=… — email lead != envelope lead." Sentry's
existing console-breadcrumb hook lifts this without an explicit
captureMessage call. Plan acceptance criterion A5.

Tests added (tests/brief-llm.test.mjs, +9):
- generateDigestProsePublic: two distinct callers with identical
  (sensitivity, story-pool) hit the SAME cache row (per Codex
  Round-2 Medium #4 — "no PII in public cache key").
- public + private writes never collide on cache key (defensive).
- greeting bucket change re-keys the personalised cache (Brain B
  parity).
- profile change re-keys the personalised cache.
- v3 cache prefix used (no v2 writes).

Test results: 77/77 in brief-llm; full data suite 6971/6971
(was 6962 pre-Step-7; +9 new public-cache tests).

Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md
Steps 6 (partial) + 7. Acceptance A5, A6.g, A6.f.

* test(digest): backfill A6.h/i/l/m acceptance tests via helper extraction

* fix(brief): close two correctness regressions on multi-rule + public surface

Two findings from human review of the canonical-synthesis PR:

1. Public-share redaction leaked personalised signals + threads.
   The new prompt explicitly personalises both `lead` and `signals`
   ("personalise lead and signals"), but redactForPublic only
   substituted `lead` — leaving `signals` and `threads` intact.
   Public renderer's hasSignals gate would emit the signals page
   whenever `digest.signals.length > 0`, exposing watched-asset /
   region phrasing to anonymous readers. Same privacy bug class
   the original PR was meant to close, just on different fields.

2. Multi-rule users got cross-pool lead/storyList mismatch.
   composeAndStoreBriefForUser picks ONE winning rule for the
   canonical envelope. The send loop then injected that ONE
   `briefLead` into every due rule's channel body — even though
   each rule's storyList came from its own (per-rule) digest pool.
   Multi-rule users (e.g. `full` + `finance`) ended up with email
   bodies leading on geopolitics while listing finance stories.
   Cross-rule editorial mismatch reintroduced after the cross-
   surface fix.

Fix 1 — public signals + threads:
- Envelope shape: BriefDigest gains `publicSignals?: string[]` +
  `publicThreads?: BriefThread[]` (sibling fields to publicLead).
  Renderer's ALLOWED_DIGEST_KEYS extended; assertBriefEnvelope
  validates them when present.
- generateDigestProsePublic already returned a full prose object
  (lead + signals + threads) — orchestration now captures all
  three instead of just `.lead`. Composer splices each into its
  envelope slot.
- redactForPublic substitutes:
    digest.lead    ← publicLead (or empty → omits pull-quote)
    digest.signals ← publicSignals (or empty → omits signals page)
    digest.threads ← publicThreads (or category-derived stub via
                     new derivePublicThreadsStub helper — never
                     falls back to the personalised threads)
- New tests cover all three substitutions + their fail-safes.

Fix 2 — per-rule synthesis in send loop:
- Each due rule independently calls runSynthesisWithFallback over
  ITS OWN pool + ctx. Channel body lead is internally consistent
  with the storyList (both from the same pool).
- Cache absorbs the cost: when this is the winner rule, the
  synthesis hits the cache row written during the compose pass
  (same userId/sensitivity/pool/ctx) — no extra LLM call. Only
  multi-rule users with non-overlapping pools incur additional
  LLM calls.
- magazineUrl still points at the winner's envelope (single brief
  per user per slot — `(userId, issueSlot)` URL contract). Channel
  lead vs magazine lead may differ for non-winner rule sends;
  documented as acceptable trade-off (URL/key shape change to
  support per-rule magazines is out of scope for this PR).
- Parity log refined: adds `winner_match=<bool>` field. The
  PARITY REGRESSION warning now fires only when winner_match=true
  AND the channel lead differs from the envelope lead (the actual
  contract regression). Non-winner sends with legitimately
  different leads no longer spam the alert.

Test results:
- tests/brief-magazine-render.test.mjs: 75/75 (+7 new for public
  signals/threads + validator + private-mode-ignores-public-fields)
- Full data suite: 6995/6995 (was 6988; +7 net)
- typecheck + typecheck:api: clean

Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md
Addresses 2 review findings on PR koala73#3396 not anticipated in the
5-round Codex review.

* fix(brief): unify compose+send window, fall through filter-rejection

Address two residual risks in PR koala73#3396 (single-canonical-brain refactor):

Risk 1 — canonical lead synthesized from a fixed 24h pool while the
send loop ships stories from `lastSentAt ?? 24h`. For weekly users
that meant a 24h-pool lead bolted onto a 7d email body — the same
cross-surface divergence the refactor was meant to eliminate, just in
a different shape. Twice-daily users hit a 12h-vs-24h variant.

Fix: extract the window formula to `digestWindowStartMs(lastSentAt,
nowMs, defaultLookbackMs)` in digest-orchestration-helpers.mjs and
call it from BOTH the compose path's digestFor closure AND the send
loop. The compose path now derives windowStart per-candidate from
`cand.lastSentAt`, identical to what the send loop will use for that
rule. Removed the now-unused BRIEF_STORY_WINDOW_MS constant.

Side-effect: digestFor now receives the full annotated candidate
(`cand`) instead of just the rule, so it can reach `cand.lastSentAt`.
Backwards-compatible at the helper level — pickWinningCandidateWithPool
forwards `cand` instead of `cand.rule`.

Cache memo hit rate drops since lastSentAt varies per-rule, but
correctness > a few extra Upstash GETs.

Risk 2 — pickWinningCandidateWithPool returned the first candidate
with a non-empty raw pool as winner. If composeBriefFromDigestStories
then dropped every story (URL/headline/shape filters), the caller
bailed without trying lower-priority candidates. Pre-PR behaviour was
to keep walking. This regressed multi-rule users whose top-priority
rule's pool happens to be entirely filter-rejected.

Fix: optional `tryCompose(cand, stories)` callback on
pickWinningCandidateWithPool. When provided, the helper calls it after
the non-empty pool check; falsy return → log filter-rejected and walk
to the next candidate; truthy → returns `{winner, stories,
composeResult}` so the caller can reuse the result. Without the
callback, legacy semantics preserved (existing tests + callers
unaffected).

Caller composeAndStoreBriefForUser passes a no-synthesis compose call
as tryCompose — cheap pure-JS, no I/O. Synthesis only runs once after
the winner is locked in, so the perf cost is one extra compose per
filter-rejected candidate, no extra LLM round-trips.

Tests:
- 10 new cases in tests/digest-orchestration-helpers.test.mjs
  covering: digestFor receiving full candidate; tryCompose
  fall-through to lower-priority; all-rejected returns null;
  composeResult forwarded; legacy semantics without tryCompose;
  digestWindowStartMs lastSentAt-vs-default branches; weekly +
  twice-daily window parity assertions; epoch-zero ?? guard.
- Updated tests/digest-cache-key-sensitivity.test.mjs static-shape
  regex to match the new `cand.rule.sensitivity` cache-key shape
  (intent unchanged: cache key MUST include sensitivity).

Stacked on PR koala73#3396 — targets feat/brief-two-brain-divergence.

…2 — flag-gated dark) (koala73#3407)

* feat(resilience): financialSystemExposure dim scaffold (Phase 2 Ship 2 — flag-gated dark)

Adds the new `financialSystemExposure` resilience dimension introduced in
plan 2026-04-25-004 Phase 2, behind the
`RESILIENCE_FIN_SYS_EXPOSURE_ENABLED` env flag. The flag defaults OFF so
the dim ships dark — the scorer returns the empty-data shape (score=0,
coverage=0, imputationClass=null) until the 3 component seeders
(seed-bis-lbs, seed-fatf-listing, seed-wb-external-debt) are populating
in production. Rollout pattern matches energy v2 (plan
2026-04-24-001).

Components (weights total 1.0 inside the dim):
  short_term_external_debt_pct_gni    0.35  (WB IDS, lowerBetter, 15-0)
  bis_lbs_xborder_us_eu_uk_pct_gdp    0.30  (BIS LBS by-parent, U-shape)
  fatf_listing_status                  0.20  (FATF, discrete: black=0, gray=30, compliant=100)
  financial_center_redundancy          0.15  (BIS LBS by-parent count, higherBetter, 1-10)

When the flag flips ON, the scorer preflights all 3 required seed-meta
envelopes (the BIS LBS seed serves both Component 2 and Component 4 —
no separate Component 4 seeder). Missing envelopes throw
`ResilienceConfigurationError(message, missingKeys)` (two-arg form per
Codex R3 P1 #2) which `scoreAllDimensions` catches and routes to
`imputationClass='source-failure'`. Per-country data gaps inside an
otherwise-published envelope are distinct: per-component reads return
null, the slot drops out of the weighted blend.

Cache prefixes bumped in lockstep with the new dim:
  resilience:score:v13:    → v14:
  resilience:ranking:v13   → v14
  resilience:history:v8:   → v9:

The scaffold also reweights `tradePolicy` 1.0 → 0.5 in
RESILIENCE_DIMENSION_WEIGHTS (the new dim shares the economic-domain
weight). This shifts the headline overallScore by ~0.46 points in the
flag-off baseline because the half-weighted tradePolicy contributes
proportionally less to the coverage-weighted economic-domain mean.
When the flag flips on with seeders populated, the new dim's actual
signal will rebalance the headline score.

What this PR ships:
  - new `scoreFinancialSystemExposure` + `normalizeBandLowerBetter` U-shape helper
  - 4 new INDICATOR_REGISTRY entries (BIS-derived tagged non-commercial/enrichment per Codex R1 #8)
  - api/health.js + api/seed-health.js dual-registry entries for the 3 new seed keys
  - frontend label map + ResilienceWidget "20 dimensions" copy
  - methodology doc heading "Financial System Exposure" + indicator table
  - tests/resilience-financial-system-exposure.test.mts: 13 tests pinning the
    formula contract + fail-closed preflight + flag-off rollout posture
  - parity test extension for v14/v9 prefixes
  - 22-dim count update across release-gate + handlers + indicator-registry tests
  - flag-gated-dark exemption in release-gate's coverage check

What this PR does NOT ship (deliberately deferred):
  - The 3 component seeders themselves (seed-bis-lbs, seed-fatf-listing,
    seed-wb-external-debt). These need real-API integration with BIS SDMX,
    FATF HTML scraping, and WB IDS — best landed as a separate PR with
    fixture-recorded tests.
  - Methodology doc `docs/methodology/financial-system-exposure.md` with
    full alternatives + license attribution.
  - Bundle registration in scripts/seed-bundle-macro.mjs.
  - Cohort sanity-check anchor test (RU/IR/KP < 20 on
    financialSystemExposure) — needs real seeder data.

Once those land and seeders populate Redis, flip
`RESILIENCE_FIN_SYS_EXPOSURE_ENABLED=true` in Vercel + Railway env
config to activate the dim.

Stacked on PR koala73#3405 (Phase 1 Ship 1).

🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0

Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>

* feat(resilience): financialSystemExposure component seeders + methodology doc

Completes Phase 2 of plan 2026-04-25-004 by shipping the 3 component
seeders, registering them in the macro bundle, adding 34 new fixture-
based tests, and writing the full methodology doc with license attribution.

Seeders (per plan §Files (Phase 2)):
  scripts/seed-wb-external-debt.mjs    — WB IDS short-term external debt as % GNI
                                         (DT.DOD.DSTC.IR.ZS × DT.DOD.DECT.GN.ZS,
                                          mrv=5 + pickLatestPerCountry per memory
                                          `feedback_wb_bulk_mrv1_null_coverage_trap`)
  scripts/seed-bis-lbs.mjs              — BIS LBS by-parent SDMX integration
                                         (12-dim key with L_POS_TYPE=N per Codex
                                          R3 P1 #1; 16 enumerated parent ISO2 codes
                                          per Codex R4 P1 #2 — no `4F` aggregate;
                                          ISO2 direct, no M49 mapping per CL_BIS_IF_REF_AREA)
  scripts/seed-fatf-listing.mjs         — FATF entry-page parser with dynamic
                                         publication-URL follow + sanity-check
                                         band gates + monthly cadence + 90d cache
                                         TTL fallback

Bundle registration:
  scripts/seed-bundle-macro.mjs — Option A per Codex R1 #5 (less operational
                                  overhead than provisioning a new bundle).

Tests (34 new):
  tests/seed-wb-external-debt.test.mjs  — combineExternalDebt formula pinning,
                                          IMF Article IV 15% GNI anchor,
                                          conservative-year selection, validate floor
  tests/seed-bis-lbs.test.mjs           — combineLbsByCounterparty Brazil 2024Q4
                                          ground-truth anchor, 1% GDP threshold
                                          for parentCount, BIS aggregate-code
                                          allow-list (5J/5A skipped), SDMX-JSON
                                          shape parsing, parser-regression guard
  tests/seed-fatf-listing.test.mjs      — findPublicationLink entry-page anchor
                                          extraction (case-insensitive), country-
                                          name lookup with apostrophe handling
                                          (`People's` → `peoples`), pub-date
                                          inference from URL slug + header,
                                          DPRK-on-call-for-action invariant

Methodology doc:
  docs/methodology/financial-system-exposure.md — full construct definition,
                                                   per-component formulas + score
                                                   shapes + coverage matrices,
                                                   fail-closed preflight,
                                                   methodology invariants,
                                                   sanctions-isolated-jurisdiction
                                                   sanity-check anchors (RU/IR/KP/...
                                                   < 20), bounded-movement gate
                                                   (60%+ |Δ|<3pt), data-source
                                                   licensing (BIS terms of use),
                                                   alternatives considered (5),
                                                   future considerations (Phase 3-5)

Quality:
  - npm run typecheck + typecheck:api: clean
  - npm run test:data: 7149/7149 pass (was 7115; +34 new seeder tests)
  - npm run lint + lint:md + version:check: clean
  - Edge function bundle check: clean

Activation runbook (when ready to flip the dim live):
  1. Merge this PR
  2. Run seed-wb-external-debt + seed-bis-lbs + seed-fatf-listing manually
     via `railway run --service seed-bundle-macro -- node scripts/<seeder>.mjs`
  3. Verify all 3 seed-meta envelopes published:
       redis-cli GET 'seed-meta:economic:wb-external-debt'
       redis-cli GET 'seed-meta:economic:bis-lbs'
       redis-cli GET 'seed-meta:economic:fatf-listing'
  4. Set RESILIENCE_FIN_SYS_EXPOSURE_ENABLED=true in Vercel + Railway env config
  5. Flush v14 caches: bulk DEL resilience:score:v14:* + DEL resilience:ranking:v14
  6. Run seed-resilience-scores.mjs to bulk-warm v14 with the new dim's signal
  7. Cohort audit: snapshot resilience:score:v14:* for all 222 countries;
     RU/IR/KP must score < 20 on financialSystemExposure (gate the construct
     before stable-rollout)
  8. Bounded-movement gate: 60% of countries |Δ|<3pt; outliers > 12pt must
     be in the explicitly-predicted RU/IR/KP/CU/VE/BY/LY/MM set
  9. Remove FLAG_GATED_DARK_DIMENSIONS allow-list entry in
     tests/resilience-release-gate.test.mts in the same commit that flips
     the flag

🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0

Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>

* chore(resilience): address self-review hardening (P1 + 4×P2 + 4×P3)

P1 — BIS LBS sequential-fetch timeout math:
  Sequential 16 parents × 60s timeout = 960s worst-case, exceeding the
  bundle's 600s timeoutMs. SIGTERM mid-flight would have leaked the
  child-lock + produced a covertly-degraded payload (per memory
  `bundle-runner-sigkill-leaks-child-lock`). Fix: parallelize parent
  fetches with concurrency=4 via a bounded-concurrency runner. Caps wall
  time at ~4 × 60s = 240s on the slow path while staying polite to BIS.

P2 — BIS LBS parent-success gate:
  Previous "all 16 failed" short-circuit was too permissive. If 15 of 16
  parent fetches failed, a single-parent payload would pass validate
  (>100 counterparty floor) and skew Component 4 (financialCenterRedundancy)
  low for every counterparty until the next successful run. Added
  MIN_SUCCESSFUL_PARENTS=12 gate; below that, throw → seed-meta unchanged
  → previous valid payload stays alive under cache TTL.

P2 — FATF findPublicationLink prefers highest-year anchor:
  Previous "first anchor matching label" was vulnerable to FATF page
  layouts where a sidebar links to historical publications using the
  same wording. Fix: collect all candidates, sort by year (URL slug or
  anchor text), return highest. Logs all candidates at WARN when more
  than one matches so ops can spot drift.

P2 — FATF unmatched country-name surfacing:
  Previous parser silently dropped country names not in
  shared/country-names.json. If FATF introduced a new spelling
  ("Mauretania", "Türkiye"), the country fell out of the listing and
  defaulted to "compliant" (score 100) — materially shifting its
  financialSystemExposure score under a fresh seed-meta. Fix:
  extractListedCountries now returns { listed, unmatchedCandidates }.
  Seeder logs unmatched at WARN; throws if > 2 candidates unmatched
  (indicates parser drift / new spellings the lookup needs to learn).

P2 — WB external-debt yearMismatch metadata:
  Composition can mix vintages of the two source indicators (different
  WB IDS lag patterns). Previous output silently used min(year) as the
  conservative anchor. Fix: emit `yearMismatch: boolean` +
  `shortTermPctOfTotalDebtYear` + `totalDebtPctOfGniYear` on each
  country record so the dashboard / scorer / audit can flag countries
  with cross-year composition. Pinning test asserts min(year) selection
  + correct flag.

P3 — BIS LBS upper-bound corruption guard:
  Added `latestVal > 1e8` skip in extractClaimsByCounterparty. 1e8
  millions = $100T (>half of global GDP), well above any plausible
  bilateral claim. Drops corrupt SDMX values silently rather than
  letting them inflate totalXborderPctGdp downstream.

P3 — BIS LBS validation floor 100 → 150:
  BIS LBS counterparty coverage is ~200+ jurisdictions. Floor of 100
  would have accepted a payload with massive coverage regression.
  Tightened to 150.

P3 — FATF grey-list floor 8 → 12 + band 8-40 → 12-40:
  Historical FATF grey-list size has been 15+ since 2020. Floor of 8
  was too lenient. Tightened to 12 with comment explaining the
  historical band.

P3 — FATF parse-failure WARN logging:
  All sanity-check throw paths in seed-fatf-listing now emit a
  console.warn explaining the failure and noting "previous valid
  payload remains under cache TTL" before throwing. Plan called for
  "warn loudly"; the implicit fall-back-to-cache pattern is now
  diagnostic-friendly.

Suggestion — BIS LBS droppedForMissingGdp provenance:
  Counterparties seen in BIS LBS but dropped because no WB GDP record
  was available are now collected into a `droppedForMissingGdp` array
  on the seed payload. Surfaces silent coverage gaps for ops triage
  without polluting the main `countries` map.

Suggestion — methodology doc operational footguns + smoke test:
  New §"Common operational footguns" section in
  docs/methodology/financial-system-exposure.md surfacing the BIS LBS
  4F-aggregate-rejection lesson + ISO 3166-1 (not M49) clarification +
  pre-flag-flip smoke test commands.

Quality gates:
  - npm run typecheck + typecheck:api: clean
  - npm run lint + lint:md + version:check: clean
  - npm run test:data: 7153/7153 pass (was 7149; +4 hardening tests)

🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0

Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>

* fix(resilience): scoreFinancialSystemExposure preflights UNVERSIONED seed-meta keys

P1 reviewer catch: the preflight in scoreFinancialSystemExposure was
reading `seed-meta:economic:<key>:v1` while runSeed
(scripts/_seed-utils.mjs) writes the freshness record at
`seed-meta:${dataKey.replace(/:v\d+$/, '')}` — i.e. with the trailing
:v\d+ stripped. Once `RESILIENCE_FIN_SYS_EXPOSURE_ENABLED=true` was
flipped, every /api/resilience/* request would have hit the missing-
seed-meta path indefinitely, throwing ResilienceConfigurationError and
stamping every country's financialSystemExposure as
imputationClass='source-failure' even with healthy seeders running.

The same unversioned shape is already used by api/health.js +
api/seed-health.js + every other in-tree scorer that walks
seed-meta keys via _dimension-freshness.ts.

Fix: route the preflight through `resolveSeedMetaKey` from
_dimension-freshness.ts. That helper already strips the trailing
:v\d+ AND applies the SOURCE_KEY_META_OVERRIDES table — the canonical
in-tree pattern for "given a registry data-key, return the seed-meta
key that runSeed actually writes." Inlining the regex would have
re-introduced the same writer/reader drift this helper exists to
prevent.

Tests:
  - All formula + preflight tests (which previously mocked the
    incorrect versioned form) updated to the unversioned key shape so
    they actually exercise the production read path.
  - New regression-guard test "preflight reads UNVERSIONED seed-meta
    keys (matches runSeed write-key shape)" pins the exact key shape
    the scorer probes. Asserts both presence of the unversioned form
    AND absence of the versioned form. A future refactor that
    accidentally re-versions the preflight will fail loudly here
    instead of silently routing every country to source-failure.

Quality gates:
  - npm run typecheck:api: clean
  - npm run test:data: 7154/7154 pass (was 7153; +1 contract guard)

🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0

Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>

* fix(resilience): address Greptile P1 + P2 catches on PR koala73#3407 (4 issues)

P1 — 30-point scoring cliff at 25% boundary in normalizeBandLowerBetter:
  Original draft had piecewise-linear segments with mismatched endpoints:
    - At value=25:    sweet spot ended at 100, over-exposed started at 70
                      → 30-point cliff × 0.30 weight ≈ 9-pt headline swing
    - At value=5:     low-int ended at 70, sweet started at 75 → 5-pt jump
  Cliffs in piecewise-linear scorers cause ranking instability for
  countries near band edges (24.9% scores ~99.9, 25.1% scores ~70).
  Re-anchored adjacent segments to share endpoints — function is now
  piecewise-CONTINUOUS at 5%, 25%, and 60% transitions:
    0%-5%:    60 → 75   (slope +3/pct, was +2)
    5%-25%:   75 → 100  (unchanged)
    25%-60%:  100 → 30  (slope −2/pct, was 70 → 30 / slope −1.14)
    60%+:     30 → 0    (unchanged)
  New regression test pins continuity at all three boundaries (samples
  values immediately above/below each transition, asserts |Δ| ≤ 1pt).

P2 — readFatfStatus defaults empty listings dict to "compliant":
  An empty `listings: {}` payload that bypassed the seeder's validate()
  would silently score every country at 100 (compliant default), masking
  a parser regression. Added defense-in-depth guard: empty dict → null
  component score → slot drops out of weighted blend → coverage shrinks
  visibly rather than the dim looking healthy. Seeder validate already
  enforces ≥1 black + ≥12 grey so this can't reach production through
  the normal write path; the guard costs nothing and catches malformed
  payloads that bypass validation. New regression test pins.

P2 — bisLbsXborderPctGdp goalposts mismatch U-shape peak:
  Registry entry had `goalposts: { worst: 60, best: 15 }` implying a
  linear lowerBetter scale peaking at 15%. The U-shape function actually
  peaks at 25% (value=15% scores 87.5, not 100). Updated to
  `{ worst: 60, best: 25 }` (over-exposed branch, peak anchor) with an
  explicit comment that goalposts here are documentation-only — the
  actual scorer uses normalizeBandLowerBetter, not a generic linear
  normalizer. Tooling reading the registry should consult the scorer
  helper directly.

P2 (resolved differently than originally suggested) —
api/bootstrap.js missing 3 new data keys:
  Greptile flagged that AGENTS.md requires bootstrap hydration for new
  data sources. Verified the bootstrap-hydration-coverage test enforces
  this via a `getHydratedData` consumer requirement in src/.
  The 3 new keys (economic:wb-external-debt:v1, economic:bis-lbs:v1,
  economic:fatf-listing:v1) are SERVER-ONLY — they feed
  scoreFinancialSystemExposure inside /api/resilience/* handlers; no
  client panel consumes them directly. Adding them to bootstrap without
  a consumer would fail the parity tests. Documented in api/bootstrap.js
  as a deferred entry: "if a future PR adds a client panel that displays
  raw BIS LBS / FATF / WB external-debt data, register the keys here
  AND add the corresponding consumer + cache-keys.ts entries in the
  same PR."

Note on Greptile's P1 #1 (preflight `:v1` suffix mismatch): already
resolved in commit d904103 (uses `resolveSeedMetaKey` from
_dimension-freshness.ts). Greptile reviewed an earlier commit.

Quality gates:
  - npm run typecheck:api: clean
  - npm run lint + lint:md + version:check: clean
  - npm run test:data: 7156/7156 pass (was 7154; +2 regression guards)

🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0

Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>

…flow (financialSystemExposure activation) (koala73#3412)

* fix(resilience): correct WB indicator + switch BIS LBS → BIS CBS dataflow

Two activation-time bugs caught when running the financialSystemExposure
seeders against production Redis (PR koala73#3407 follow-up).

== Fix #1 — WB IDS short-term external debt indicator ==

Original code used `DT.DOD.DSTC.IR.ZS × DT.DOD.DECT.GN.ZS / 100`,
intended to compose "short-term external debt as % of GNI." But
`DT.DOD.DSTC.IR.ZS` is "% of total RESERVES" (NOT "% of total
external debt"). Argentina (164%), Turkey (114%), and Sri Lanka all
had intermediate ratios > 100% because their short-term debt exceeds
reserves — caught by Redis audit on the first live seed run.

Fix: divide absolute USD values directly.
  shortTermDebtPctGni = (DT.DOD.DSTC.CD / NY.GNP.MKTP.CD) × 100

Live verification: BR=4%, AR=7.77%, TR=13.26%, LK=6.03%, NG=9.05% —
all in plausible 0-15% range matching real-world short-term debt ratios.

== Fix #2 — BIS LBS → BIS CBS dataflow ==

Plan called for "BIS LBS by-parent" data, but `WS_LBS_D_PUB` only
exposes counterparty as the aggregate `5J`. Per-counterparty
breakdowns require `WS_CBS_PUB` (Consolidated Banking Statistics).

CBS: 11 dimensions (LBS had 12), parent is L_REP_CTY, CBS_BASIS=F
selects foreign-claims ultimate-risk basis. SDMX key:
  Q.S.<PARENT>.4B.F.C.A.A.TO1.A.

Live verification: 201 records, all 16 parents fetched, BR=19.6%,
CN=3.1%, AE=46.7%, SG=233%, CH=278% — values match real-world bank
exposure including U-shape penalty for hub jurisdictions.

Filename + Redis key (`economic:bis-lbs:v1`) retained for historical
continuity; scorer-side contract unchanged.

What's pending for full activation: FATF seeder still blocked by
HTTP 403 on fatf-gafi.org. Separate PR.

Quality gates:
  - typecheck:api / lint / lint:md: clean
  - npm run test:data: 7189/7189 pass

🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0

Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>

* fix(resilience): address Greptile P2s on PR koala73#3412 + hub-self-exclusion

Three fixes on top of the BIS LBS → CBS migration.

== Greptile #1 — BIS_AGGREGATE_CODES Set was dead code ==

The filter chain was:
  cpCode.length !== 2 || !/^[A-Z]{2}$/.test(cpCode) || BIS_AGGREGATE_CODES.has(cpCode)

Every entry in BIS_AGGREGATE_CODES had a digit (5J, 1C, A2, 4F, ...),
so all of them already failed the `/^[A-Z]{2}$/` regex BEFORE the Set
was consulted. The Set never caught anything in production.

Real risk the Set was supposed to mitigate: BIS introduces a 2-letter
ALL-ALPHA aggregate (e.g. `EU`) that the regex would pass. Replaced
with `ALPHA_AGGREGATE_CODES` — empty by default, audited each CBS
quarterly publish. Verified against the live `WS_CBS_PUB`
L_CP_COUNTRY codelist (252 values as of 2026-04-25): no current
2-letter all-alpha aggregates exist.

== Greptile #2 — combineLbsByCounterparty alias removed ==

`export { combineCbsByCounterparty as combineLbsByCounterparty }` made
the old LBS name a first-class export of the module. Any future code
importing `combineLbsByCounterparty` would silently get CBS data while
the name implied LBS semantics. Only the test file consumed it.

Removed the alias, renamed test imports to `combineCbsByCounterparty`.

== Self-noted finding — hub-country self-counting in Component 4 ==

Caught during the 2026-04-25 production activation audit. For
Singapore and Switzerland (both in PARENT_COUNTRIES AND on the
counterparty list), the BIS CBS payload included their own domestic
banking claims:
  SG-on-SG:  $584B
  CH-on-CH:  $2.2T

These are domestic loan books, NOT foreign-bank cross-border exposure.
Component 4 (`parentCount`) is supposed to measure "how many INDEPENDENT
FOREIGN USD-clearing routes remain if a major counterparty pulls" —
domestic banks aren't a foreign-fallback route.

Filter: drop entries where `cp === parent` when building
claimsByCpByParent. Live verification of the impact:

  Country  OLD pct      NEW pct      OLD parentCount  NEW parentCount
  SG       232.98%      126.11%      12               11
  CH       277.89%       43.69%      10                9
  US        ...          32.2%       (US-on-US excluded)
  GB        ...          65.96%      (GB-on-GB excluded)
  BR        19.62%       19.62%      4                 4 (unchanged; BR not in PARENT_COUNTRIES)
  AE        46.69%       46.69%      10               10 (unchanged; AE not in PARENT_COUNTRIES)

CH dropped from "Iceland-2008 territory" to "over-exposed" — the old
number was a measurement artifact. The new numbers reflect actual
foreign-bank exposure as the construct intends.

Tests: new "excludes self-claims" regression guard pins the SG/CH
behavior. Methodology doc gets a §"Self-exclusion rule" callout under
Component 4.

Quality gates:
  - typecheck:api / lint / lint:md: clean
  - npm run test:data: 7190/7190 pass (was 7189; +1 self-exclusion guard)
  - Live production seed run: 201 records, hub-self-exclusion verified

🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0

Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>

---------

Co-authored-by: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>

…e mode (koala73#3422)

* fix(brief-ingest): READ-time freshness + direct-RSS migration + U6 age mode

Three coupled gap closures discovered after PR koala73#3419 merged. Brief
2026-04-26-0802 still surfaced two months-old Pentagon items DESPITE
PR koala73#3417's `when:1d` ingest gate working perfectly (verified live: the
gated query returns zero Pentagon items in the last 24h). The cause was
post-deploy residue, not the gate. See:
  skill: ingest-gate-tightening-leaves-residue-in-read-path

Fixes (all on the same PR because they share the publishedAt persistence
precondition):

1. READ-time freshness floor in buildDigest (seed-digest-notifications.mjs).
   Drops story:track:v1 rows whose source publishedAt is older than
   DIGEST_READ_MAX_AGE_HOURS (default 48h). Closes the residue window: when
   any future ingest-side gate is tightened, pre-deploy entries stop
   shipping in briefs even before they TTL-expire from the accumulator.
   Rows missing publishedAt (legacy pre-this-PR entries) fall through —
   back-compat. Operator kill switch: DIGEST_READ_MAX_AGE_HOURS=999.

2. Persist publishedAt in story:track:v1 HSET (list-feed-digest.ts).
   Adds 'publishedAt', String(item.publishedAt) to buildStoryTrackHsetFields.
   Required for #1 AND for U6's age-mode (#3) to actually find the field.
   Pre-existing rows pick it up on their next mention.
   Test: tests/news-story-track-description-persistence.test.mts asserts
   the field is emitted as a numeric string that round-trips through
   parseInt.

3. U6 audit gains --mode=age|url|both (audit-static-page-contamination.mjs).
   Original URL-pattern classifier is BLIND to Google-News-routed entries
   (track.link is news.google.com/rss/articles/CBMi opaque redirect).
   --mode=age matches by track.publishedAt > max-age-hours, the right
   primary signal for evicting post-deploy residue. Per-reason rollup
   surfaces url vs age hits. Operator runbook included in the script
   header. Default --mode=url is back-compat.

4. Feed swap: 3 of 5 gov sources to direct first-party RSS (_feeds.ts).
   Verified 2026-04-26 via live curl + end-to-end parseRssXml smoke test:
     - Pentagon -> https://www.war.gov/DesktopModules/ArticleCS/RSS.ashx?ContentType=1&Site=945
     - White House (briefings) -> /briefings-statements/feed/
     - White House Actions    -> /presidential-actions/feed/ (NEW entry)
   Direct RSS means the publisher's pubDate is authoritative — no
   re-indexing of months-old PDFs creating fresh-looking Google entries.
   State Dept, Treasury, DOJ retain gn(...) with an explanatory comment;
   no working public RSS at any verified path, Federal Register fallback
   bot-blocked. The READ-time freshness floor (#1) mitigates the residue
   gap for those.

5. source-tiers.json: White House Actions = 1 (with scripts/shared mirror).
   New feed gets the same tier-1 source-boost as the existing White House
   entry; without it, Actions items would default to tier-4 with no boost.

250/250 tests pass; importance-score-parity preserved.

Operator runbook for the immediate post-deploy residue:
  1. Wait ~1h for cron tick to start writing publishedAt on existing
     entries via HSET re-mention.
  2. Dry run: node scripts/audit-static-page-contamination.mjs --mode=age
  3. Inspect output, confirm matched titles look stale.
  4. Apply: --mode=age --apply

* fix: address review findings — window-aware floor + residue audit mode (P1×2 + P2/P3)

5 fixes responding to review:

P1 (reviewer): READ-time floor + audit --mode=age both fall through on
missing publishedAt — the legacy state of EVERY pre-PR-3422 row. So
the actual residue this PR was meant to evict (Pentagon items in brief
2026-04-26-0802) wasn't catchable by either gate. Add --mode=residue
that matches rows with missing/unparseable publishedAt — the explicit
opt-in cleanup signal for the post-deploy one-shot. Operator runbook
updated: wait at least 1 cron cycle (so still-active rows have publishedAt
re-mentioned in via HSET), then run --mode=residue --apply.

P1 (self-review): READ-time floor was hardcoded 48h, breaking weekly
users (their 7d window stories all dropped). Replaced with
readTimeAgeCutoffMs(windowStartMs) = windowStart - 24h buffer. Daily
user (24h window) -> 48h cutoff (matches old behavior); weekly user
(7d window) -> 8d cutoff (the broken case). DIGEST_READ_MAX_AGE_HOURS
env var removed — there's no scenario where an operator wants to drop
items younger than the rule's intended window.

P2: Defensive cast in buildStoryTrackHsetFields. Number.isFinite guard
on item.publishedAt prevents the literal "undefined"/"NaN" string from
ever being persisted; degraded input writes '' which the read-side
parseInt treats as legacy back-compat (fall through, not mis-classify
as a stale-row).

P2: Extract shouldDropTrackByAge + readTimeAgeCutoffMs into
scripts/lib/digest-orchestration-helpers.mjs (the existing module for
this purpose). 6 new unit tests cover the predicate matrix
(within/at-cutoff/before/missing/unparseable/zero/null-track) plus 2
integration cases that reproduce the exact failure modes:
  - weekly user with 5d-old story is KEPT (window-aware vs naive 48h)
  - 4-month-old residue (Jan 2026 Pentagon PDF) is DROPPED for daily user

P3: Audit parseArgs now rejects unknown args (--mode age space-not-equals)
with exit-2 + clear error message. classifyTrack refactored to a pure
function (mode + maxAgeMs as options) so tests can exercise it without
the script's module-load argv side effects. main() invocation gated
behind import.meta.url === file://process.argv[1] check (standard
ESM idiom). 10 new unit tests cover the url/age/residue/both modes
+ parseArgs flag handling.

316/316 tests pass; importance-score-parity preserved.

* fix(audit): residue mode safety gate + EOF whitespace (review fixes on koala73#3422)

P2 (reviewer): --mode=residue --apply was too broad. Classifier matched
ANY row with missing publishedAt — but every pre-PR-3422 row lacks the
field, so the recommended one-shot cleanup could delete legitimate recent
stories that simply hadn't been re-mentioned in the first post-deploy
cron cycle.

Fix: residue mode now requires BOTH conditions:
  1. publishedAt missing/unparseable (legacy or anomalous), AND
  2. lastSeen >= residueMinStaleMs ago (default 24h).

Active stories get HSET-touched on every re-mention, so a row that
hasn't been touched in 24h is genuinely abandoned — the new ingest gate
(when:1d) is correctly excluding it. Fresh-lastSeen rows are PROTECTED
from deletion.

New flag: --residue-min-stale-hours=N (default 24, positive-int only).
Operator runbook updated:
  1. Wait >=24h post-deploy.
  2. Dry run: node scripts/audit-static-page-contamination.mjs --mode=residue
  3. Inspect output, confirm rows are stale + look like residue.
  4. Apply: --mode=residue --apply

Defensive fall-throughs:
  - Missing lastSeen: treat as ancient (errs toward eviction). Acceptable
    in this opt-in destructive mode; rows without lastSeen are anomalous.
  - publishedAt present (any value): NOT residue (caught by --mode=age).

Tests:
  - 8 residue-mode cases incl. the explicit P2 fix ("fresh lastSeen must
    protect the row"), boundary at exactly the threshold, missing-lastSeen
    treated as ancient.
  - 4 parseArgs cases for the new --residue-min-stale-hours flag.

P3 (reviewer): tests/digest-orchestration-helpers.test.mjs had a trailing
blank line at EOF. Stripped.

323/323 tests pass; importance-score-parity preserved.

* fix(audit): align residue staleness gate with max digest window (P2 round-2)

Reviewer flagged that --mode=residue --apply with the 24h staleness
default would delete legitimate weekly-user stories. A pre-PR-3422 row
that lacks publishedAt with lastSeen 2-7d ago is still ship-able for a
weekly digest (whose readTimeAgeCutoffMs in
digest-orchestration-helpers.mjs is windowStart - 24h = 8d ago) — but
the 24h gate would mark it as residue and delete it.

Fix: default --residue-min-stale-hours from 24 to 192 (= 7d max digest
window + 24h buffer). Aligns with the readTimeAgeCutoffMs formula so
residue mode can never delete a row still legitimately ship-able for
ANY user. New explicit regression test:
  "SAFETY: does NOT match weekly-user 5d-old story" — pre-fix would
  have failed (deleted the row); now PROTECTED.

Operators with confidence the fleet is daily-only can drop to faster
cleanup via --residue-min-stale-hours=48 (documented escape hatch in
the script header runbook + test case).

The other safety guards remain:
  - publishedAt present → not residue (caught by --mode=age, not residue).
  - lastSeen fresh (< gate) → protected.
  - missing lastSeen → ancient (errs toward eviction in opt-in path).

325/325 tests pass; importance-score-parity preserved.

…a73#3431)

* feat(convex/broadcast): audience-export pipeline for PRO launch

Stacked on feat/customers-normalized-email-index — needs that branch's
customers.normalizedEmail field + index + backfill before this action
can produce a clean dedup.

Adds:
- convex/broadcast/audienceExport.ts: paginated internalAction plus
  three internalQueries that build the deduped audience and push
  contacts to a Resend Audience via the Contacts API.

Dedup formula: registrations − emailSuppressions − customers. Any
user who's been through Dodo checkout (active, cancelled, expired)
is excluded — pitching paid users a "buy PRO!" email is the worst-
case failure mode the launch is trying to avoid.

Idempotent on re-run: Resend's 422 already_exists response is counted
as `alreadyExists`, not `failed`. Cursor-paginated for safe resume
mid-export. Dry-run mode (counts only, no Resend calls) for verifying
exclusion math before the first real send.

Codegen catch-up:
- _generated/api.d.ts updated to declare broadcast/audienceExport
  (this file cross-references its own siblings via internal.* and
  needs the typed reference).
- Also catches up payments/backfillCustomerNormalizedEmail from the
  base PR — that file didn't cross-reference so its CI passed
  without the codegen update; adding now so future calls into it
  resolve correctly.
- Convex regenerates identically on deploy — no drift risk.

Operational:
  npx convex run broadcast/audienceExport:exportProLaunchAudience \
    '{"audienceId":"aud_xxx"}'
  # Loop, passing continueCursor from each response, until isDone:true

* fix(audienceExport): address PR review (P1×3)

All three findings valid; verified line numbers match commit 7a440c1
and the Resend API claim against current docs at
https://resend.com/docs/api-reference/contacts/create-contact.

P1 #1 — Paid-customer exclusion fails open when normalizedEmail missing:
`getPaidEmails` now derives the join key from `customers.email` (the
existing `email.trim().toLowerCase()` convention) when `normalizedEmail`
is unset. The backfill becomes a perf optimization — a missed/incomplete
run can no longer silently leak paid users into the launch audience.

P1 #2 — Wrong Resend endpoint:
Switched from `POST /audiences/{id}/contacts` (legacy, may still resolve
but no longer the canonical 2026 path) to `POST /contacts` with
`segments: [{ id: segmentId }]` in the body, per current Resend docs.
Renamed the action arg from `audienceId` to `segmentId` to match
Resend's renaming of Audiences → Segments. Updated JSDoc usage examples.

P1 #3 — 409/422 catch-all masking validation errors:
Added `isDuplicateContactError()` helper that parses the 422 body and
matches on `name` (`*_already_exists` / `*_duplicate`) and `message`
(`/already (exists|in)|duplicate/`). Only duplicate-shaped responses
increment `alreadyExists`; everything else (missing segment, invalid
email, unauthorized field, etc.) increments `failed` and logs the
parsed body. Also dropped the speculative 409 branch — Resend doesn't
use 409 for contacts; 422 is the real status.

* fix(audienceExport): two-step upsert guarantees segment membership (P1)

Round-2 review finding: previous fix counted POST /contacts 422 duplicates
as alreadyExists without verifying the contact ended up in OUR segment.
Resend's global-Contacts model means a 422 duplicate could mean the
contact exists in a different segment or no segment at all — and the
`segments` field on the duplicate path is not applied — so existing
global contacts could be silently OMITTED from the launch segment while
the export reports success.

New `upsertContactToSegment(apiKey, email, segmentId)` helper does a
deterministic two-step:

  1. POST /contacts with `segments: [{ id }]`. Brand-new globally →
     creates and assigns in one call → outcome=created.
  2. If step 1 returns a duplicate-shaped 422, the contact exists
     globally. Disambiguate with POST /contacts/{email}/segments/{id}:
       - 2xx → was global-only or in another segment, now linked here
         → outcome=linkedExisting
       - 422 duplicate-shaped → was already in this segment
         → outcome=alreadyInSegment
       - anything else → outcome=failed

Stats updated:
- `upserted` now counts BOTH `created` AND `linkedExisting` (anything
  that ended up in the segment via this call).
- `linkedExisting` added as a separate diagnostic counter — operator
  can compare to verify how many were pre-existing global contacts.
- `alreadyExists` now strictly means "was already in this segment
  before this call" — never inferred from a global-duplicate response
  without segment-membership verification.

* fix(audienceExport): address PR review round 3 (P2×3)

Stale: P1 "422 conflates duplicate with validation errors" was cited
on commit 7a440c1; addressed in ebf2bf8 — replied as resolved.

Three valid P2s on current code, all fixed:

P2 — Missing User-Agent header on Resend fetches:
AGENTS.md:185 requires User-Agent on every server-side fetch. Added
USER_AGENT constant and applied to both /contacts and
/contacts/{email}/segments/{id} POSTs.

P2 — Raw email PII written to Convex dashboard logs:
The action's failure-path console.error interpolated `${email}` into
log lines that anyone with project access can read. Added maskEmail()
helper (`[email protected]` → `jo******@example.com`) and applied
to the failure-path log.

P2 — `upserted` semantics differ between dry-run and live:
The previous draft incremented stats.upserted in BOTH dry-run (every
email passing dedup) and live (only successful Resend calls). Operators
comparing dry-run to live totals to validate dedup math would see a
spurious discrepancy on any failure / linkedExisting / alreadyInSegment.

Reshaped ExportStats:
- Dry-run-only: `wouldUpsertAfterDedup` — count of emails that pass
  the dedup filters and would be attempted on a live run.
- Live-only: `upserted`, `linkedExisting`, `alreadyExists`, `failed`
  — strictly result of Resend interactions; all zero in dry-run mode.
- Shared (dedup-only): `suppressedSkipped`, `paidSkipped`, `emptyEmail`.

Operator can now compare dry-run `wouldUpsertAfterDedup` to live
`upserted + alreadyExists` and any divergence is a real Resend issue,
not a counter-semantics artefact.

…#3473)

* feat(broadcast): cron-driven ramp runner with kill-gate halt

Replaces the manual three-command ritual (assignAndExportWave →
createProLaunchBroadcast → sendProLaunchBroadcast) with a daily
cron at 13:00 UTC that:

  1. Fetches the prior wave's getBroadcastStats
  2. Halts (sets killGateTripped=true, deactivates ramp) if bounce
     rate > 4% or complaint rate > 0.08% — operator must clear
     before resume
  3. Otherwise runs assignAndExportWave + create + send for the
     next tier in `rampCurve`

Singleton config table `broadcastRampConfig` (keyed by literal
"current") holds the curve, current tier, kill-gate state, and last-
wave tracking. Admin mutations: initRamp / pauseRamp / resumeRamp /
clearKillGate / abortRamp / getRampStatus.

Safety rails:
- `MIN_DELIVERED_FOR_KILLGATE = 100`: kill-gate ignored until prior
  wave has enough delivered events for stable rate calc (avoids
  trip on sample-size noise: 1 bounce / 10 delivered = 10%)
- `MIN_HOURS_BETWEEN_WAVES = 18`: cron defers if prior wave is
  fresher than 18h (bounces / complaints take time to flow back via
  Resend webhook)
- `UNDERFILL_RATIO = 0.5`: deactivates ramp when assignAndExportWave
  returns < 50% of requested count (pool drained signal)
- Kill-gate latch is one-way — never auto-clears. Operator runs
  `clearKillGate '{"reason":"..."}'` after investigating, which
  stamps the cleared reason into lastRunStatus for audit
- Partial-failure recovery: if assignAndExportWave / create /
  send throws mid-flight, status records as "partial-failure" with
  the offending error and the cron blocks until cleared. Throws
  bubble to Convex auto-Sentry for paging
- `_recordWaveSent` mutation does an `expectedCurrentTier` check
  before patching — two concurrent cron firings can't both advance
  the same tier (defence-in-depth; cron isn't supposed to overlap
  but Convex doesn't guarantee at-most-once on retried cron runs)

Wave-label naming: `${prefix}-${tier + offset}`. Default offset 3
means tier 0 → wave-3, tier 1 → wave-4, etc. — picks up cleanly
after manually-sent canary-250 + wave-2.

Daily-cron timing 13:00 UTC: late enough that overnight bounces /
complaints from the prior 24h have flowed back via webhook, early
enough (9am ET / 6am PT / 3pm CET) that a tripped kill-gate hits
US business hours for triage.

Files:
- convex/schema.ts: new `broadcastRampConfig` table + by_key index
- convex/broadcast/rampRunner.ts: runDailyRamp action + admin
  mutations + the two recording mutations
- convex/crons.ts: wires runDailyRamp to crons.daily
- convex/_generated/api.d.ts: regenerated

Operator setup (run once after deploy):

  npx convex run broadcast/rampRunner:initRamp '{
    "rampCurve": [1500, 5000, 15000, 25000],
    "waveLabelPrefix": "wave",
    "waveLabelOffset": 3
  }'

After that, the cron handles everything until either kill-gate trips
or the pool drains. Status check anytime via:

  npx convex run broadcast/rampRunner:getRampStatus '{}'

* fix(broadcast): seed prior wave + halt on partial export failures

PR koala73#3473 review:

P1 #1 — first automated wave skipped kill-gate for the last manually
sent wave because `initRamp` had no way to seed prior-wave metadata.
With currentTier=-1 and lastWaveBroadcastId=undefined, the kill-gate
block at runDailyRamp's Step 1 was unreachable on the first tick after
init. Add `seedLastWave*` optional args; require them as a pair when
`waveLabelOffset > 0` (operational signal that this is a resumption
after manual waves, not a fresh ramp).

P1 #2 — runner narrowed `assignAndExportWave`'s return type to only
`{segmentId, assigned, underfilled}`, dropping `failed` and
`stampFailed`. A wave that requested 500 with 250 push failures + 250
successes would have proceeded to create + send the broadcast, marking
the tier as cleanly advanced. `stampFailed > 0` is worse: contacts are
in the Resend segment (will be emailed) but unstamped (re-eligible for
the next pick → guaranteed duplicate-email). Now: widen the local type
to the full `WaveExportStats`, export it from audienceWaveExport.ts,
and abort the run with `partial-failure` status if either failure
counter is non-zero. Operator clears via the existing
`lastRunStatus === partial-failure` gate.

* fix(broadcast): add clearPartialFailure recovery mutation

PR koala73#3473 review (third P1):

The partial-failure block I added in 35091b5 (treat any non-zero
export failure counter as halt-don't-proceed) had no recovery path.
`runDailyRamp` refuses to advance while
`lastRunStatus === "partial-failure"`, but `clearKillGate` no-ops when
`killGateTripped` is false — so a partial-failure would block the cron
forever short of `abortRamp` or hand-patching the DB.

Add `clearPartialFailure(reason: string)` matching the `clearKillGate`
shape: requires partial-failure status (else no-op), records audit
reason in `lastRunStatus`, clears `lastRunError`. Kept separate from
`clearKillGate` deliberately — kill-gate is an email-reputation
investigation (bounce/complaint thresholds), partial-failure is a
mechanical export/send investigation (Resend logs, Convex stamp
errors). Different recovery requirements; conflating them would
encourage operators to clear without reading the right log.

Updated operator-usage docstring with the new command.

…eads (koala73#3667)

* fix(brief): grounding gate on digest synthesis — block hallucinated leads

The 2026-05-12 0801 send shipped a "President Biden announced a new
executive order targeting cryptocurrency mixers and privacy coins" lead
to users whose actual story pool was Trump-era geopolitics (Iran
ceasefire, Israeli strikes in Lebanon, Sudan drones, Cuba rhetoric,
Russia/Ukraine, EU sanctions). The magazine envelope rendered the
correct grounded lead; the email Executive Summary block rendered a
fully fabricated narrative with four threads all about the imaginary
Biden EO. Shape was valid — content was a hallucination from training-
data priors.

Root cause: validateDigestProseShape only checks shape (lengths,
types, array presence). Any well-formed JSON whose lead names entities
absent from the input pool was happily cached for 4h and re-served on
every cache hit until the row TTL'd out. The canonical-brain promise
from PR koala73#3396 ("compose-phase synthesis is reused on send-phase cache
read") only guarantees same-output-per-cache-hit; it does not guard
against the LLM committing to a fabricated row in the first place.

This adds checkLeadGrounding(synthesis, stories): extract proper-noun
tokens (capitalised, length ≥4) from story headlines; require ≥2
distinct hits in the lead + thread teasers (relaxed to 1 when the
corpus has <4 anchor tokens). Length cap of 4 deliberately filters
short-form acronyms (US/EU/UN/RSF) which are too generic to
discriminate. Plumbed through validateDigestProseShape (new optional
stories arg, back-compat preserved) → parseDigestProse →
generateDigestProse cache-hit path. Cache key bumped v4 → v5 so
existing v4 rows (which may carry shape-valid hallucinations) are
evicted on rollout; 4h paid-for-once cost matching the established
pattern at this function.

When a cached row fails the grounding gate, the cron's existing 3-level
fallback chain falls through to L2 (capped pool, no profile/greeting)
and ultimately L3 (stub with "Digest" subject). A user gets either a
re-rolled grounded lead or a degraded subject line — never a
hallucinated headline.

Test coverage: the verbatim May 12 hallucinated lead + the verbatim
2026-05-12 story headlines as fixtures, asserting the validator
rejects. Plus the actual magazine lead from the same day as positive
control, plus skip-on-empty-stories back-compat, plus single-story
threshold relaxation, plus short-acronym filtering, plus a
generateDigestProse cache-hit re-LLM regression test.

86/86 brief-llm tests pass. 179/179 brief-related tests pass.
typecheck clean. biome lint clean.

* fix(brief): address PR koala73#3667 review — lead must ground independently + token-set matching

Two real bugs caught on review of the grounding validator.

#1 — Combined haystack let a hallucinated lead pass when teasers
happened to mention real entities.

  Before: lead + threads[].teaser were joined into one string and
  any ≥2 anchor hits across the combination passed. So a synthesis
  with a fabricated "President Biden announced..." lead and grounded
  teasers like "Trump rejected Tehran's response" would accept —
  even though the visible top-of-email lead stayed fabricated.

  After: lead must independently hit ≥1 anchor. Combined check
  (≥2 hits, or ≥1 for sparse corpora) still applies on top of that.
  A hallucinated lead with grounded teasers now correctly rejects.

#2 — haystack.includes(tok) accepted unrelated entities via
substring match.

  Before: anchor "iran" hit on "tirana", "oman" on "romania",
  "india" on "indiana". Any anchor that happens to appear as a
  substring of an unrelated word in the synthesis prose counted.

  After: both sides tokenise on the same delimiter regex into Sets
  and match by membership. Story-side keeps the proper-noun
  capitalisation filter; synthesis-side does not (so possessive
  forms / sentence-medial mentions still count as anchor hits).

Both regressions covered by new golden tests:
  - hallucinated-lead-grounded-threads: rejects with the new lead
    independence requirement, accepts when the lead is also grounded
  - substring-trap-corpus (Iran/Oman/India + Tirana/Romania/Indiana):
    rejects under token-set matching, accepts under real word matches

Tests: 88/88 brief-llm, 8604/8604 full suite, typecheck clean,
biome clean.

* fix(brief): address PR koala73#3667 review round 2 — stopword filter + Unicode delimiters

Two more bugs in the grounding validator caught on second review.

#1 (HIGH) — Generic capitalised words leaked into the anchor set.

  Before: any capitalised word ≥4 chars from a headline became an
  anchor. So "President Trump signed Iran sanctions" added
  "president" to storyTokens. A hallucinated lead like "President
  Biden announced..." passed the lead-anchor check via the shared
  "president" token, and a teaser mentioning Iran satisfied the
  combined threshold — recreating the exact failure mode this PR
  is trying to block.

  After: GROUNDING_ANCHOR_STOPWORDS filters honorifics ("President",
  "Senator", "Minister"), generic role labels ("Officials",
  "Members", "Forces"), quasi-adjectives ("Senior", "Federal",
  "Former"), sentence-start filler ("After", "Following",
  "Despite"), and calendar words. Specific entity names (Iran,
  Trump, Israel, EU) are deliberately NOT on the list. "May" is
  also omitted (Theresa May, May Day, May the month all collide).

#2 (MEDIUM) — Unicode apostrophes weren't delimiters.

  Before: GROUNDING_TOKEN_DELIMS only included ASCII apostrophe.
  Reuters/AP/Guardian headlines use U+2019 ("China's", "Iran's",
  "DPRK's"). The regex didn't split on U+2019, so "China's" became
  a single token "china's" that no normal lead saying "China"
  could ever match — a false negative that would reject genuinely
  grounded leads.

  After: U+2018, U+2019, U+201C, U+201D, U+00B4 added alongside
  ASCII counterparts.

Both regressions covered by new golden tests:
  - "President Trump signed..." headlines + "President Biden
    announced..." hallucinated lead must REJECT
  - U+2019 headlines + ASCII-quote grounded lead must ACCEPT

Tests: 90/90 brief-llm, 8606/8606 full suite, typecheck clean,
biome clean.

* fix(brief): address PR koala73#3667 review round 3 — bigram-leading title stopwords

Round 2 added "President" to the anchor stopword list. Round 3 review
caught that other common bigram titles still leak through.

  Before: "Prime Minister Netanyahu says Iran threats continue" added
  "prime" to anchors. A hallucinated "Prime Minister Trudeau announced
  cryptocurrency restrictions..." passed the lead-anchor check via the
  shared "prime" token. A teaser mentioning Iran satisfied the combined
  threshold. Same shape worked for Chief Justice / Cardinal X /
  Chancellor X / Speaker X / Ambassador X.

  After: GROUNDING_ANCHOR_STOPWORDS extended with bigram-leading
  titles: prime, chief, premier, chancellor, speaker, ambassador,
  envoy, commissioner, attorney, cardinal, archbishop, monsignor,
  reverend, pastor, bishop, lord, lady, dame, congressman/woman/person,
  representative, delegate, baron(ess).

Regression test covers Prime Minister / Chief Justice / Cardinal
ride-along leads, plus a counter-control naming the actual entity
(Netanyahu) which still passes.

Tests: 91/91 brief-llm, 8607/8607 full suite, typecheck clean,
biome clean.

* fix(brief): observability for digest synthesis failures (PR koala73#3667 review #3)

Greptile review caught: when generateDigestProse returns null, ops
can't tell whether the LLM threw, returned no content, returned
malformed JSON, or returned a shape-valid hallucination that the
grounding gate rejected. All four classes look identical in logs —
just "L1 returned null" — so a sustained model regression looks
identical to an infra blip during on-call triage.

Add two distinct console.warn lines:
  - "LLM call threw" — provider/network failure (catches the actual
    error message for triage)
  - "ungrounded or malformed output" — provider returned but parsing
    or grounding rejected (logs text length so 0 vs ~900-char
    distinguishes "no content" from "model drift/hallucination")

Cost note documented inline: we deliberately do NOT cache the
failure with a sentinel under the v5 key. At temperature 0.4 the
next cron tick may roll a grounded output for the same prompt;
caching the failure would block legitimate retries. The L1→L2→L3
fallback in runSynthesisWithFallback handles user-visible
degradation; these logs handle ops visibility.

Tests: 91/91 brief-llm pass, typecheck clean, biome clean.

* fix(brief): PR koala73#3667 review round 4 cleanup — extract grounding helpers + fixture comment

Two reviewer-flagged cleanups:

- P2: extract `extractAnchors` / `tokensetOf` from inside
  checkLeadGrounding to file-level `extractAnchorTokens` /
  `groundingTokenSet`. Avoids closure re-instantiation per call,
  separates the two helpers cleanly, and makes them individually
  inspectable / unit-testable. Behaviour unchanged — same callers,
  same inputs, same outputs.

- P3: add a load-bearing comment on the `story()` test factory
  documenting that the default headline ("Iran threatens to close
  Strait of Hormuz...") is what grounds every `validJson` lead used
  by `generateDigestProse` / `generateDigestProsePublic` cache-shape
  tests. If a future contributor changes the default headline so it
  no longer mentions Iran/Hormuz, those tests would silently reject
  via the v5 grounding gate with cascading "expected truthy, got
  null" failures whose root cause is invisible. Comment names the
  invariant + the escape hatch (override headline + update
  validJson).

Tests: 91/91 brief-llm pass, typecheck clean, biome clean.

* fix(brief): PR koala73#3667 review round 5 — JSDoc attachment, blind-spot warn, stopword heuristic

Three reviewer findings:

#1 — JSDoc block was orphaned by the round-4 helper extraction.
  The big "Cheap content-grounding check..." doc sat at line 519,
  BEFORE extractAnchorTokens and groundingTokenSet — JSDoc parsers
  attach a doc block to the NEXT declaration, so the rich docs were
  describing the wrong function. Moved directly above
  checkLeadGrounding where they belong.

#2 — Lowercase-headline blind spot. If a feed ever produces all-
  lowercase or all-≤3-char headlines, extractAnchorTokens returns
  empty for every story, storyTokens.size === 0, and the gate
  silently returns true (skip). Added a console.warn gated on
  stories.length >= 3 so synthetic single-headline test corpora
  don't spam logs but real production feed regressions surface in
  cron output.

#3 — Stopword maintenance heuristic. Added a comment to
  GROUNDING_ANCHOR_STOPWORDS describing the detection rule: dump a
  week of real headlines, tokenise with stopwords disabled, count
  frequencies, inspect any token appearing in >~10% of headlines
  that isn't a known proper noun. The Prime/Chief/Cardinal gaps
  caught on rounds 2-3 would have surfaced from such a frequency
  audit. Captures the maintenance burden as an actionable signal
  rather than guesswork.

Tests: 91/91 brief-llm pass, typecheck clean, biome clean. The
new console.warn calls are intentionally surface in test output
when triggered (reviewer round 5 #4 — informational, no action).

… cleanup (koala73#3673)

* fix(brief): strip "Video:"/"Watch:" prefixes + match wire-name vs feed-name in suffix

Two real headline-noise issues observed in production briefs.

#1 — Editorial-format prefixes shipped to the magazine.

  May 12 brief, page 16/18:
    "Video: Philippine senator flees ICC arrest over role in drug war"

  The "Video: " tells the user nothing the magazine card body
  doesn't already convey (every card has its own source line and
  body block). Same shape applies to Watch / Live / Photos / Photo /
  Gallery / Listen / Podcast / Breaking / Exclusive / Opinion /
  Analysis / Update.

  New stripHeadlinePrefix helper handles all of the above with a
  REQUIRED trailing colon — preserves legitimate headlines that use
  one of these words as a noun ("Video game regulator fines...",
  "Watch list updated by sanctions office...", "Live broadcasts
  paused...").

#2 — Wire-name suffix not stripped when feed-name is the longer form.

  May 13 brief, story 5:
    "Putin says Russia will deploy new Sarmat nuclear missile this
     year - Reuters"
  primarySource: "Reuters World"

  stripHeadlineSuffix used strict equality: "reuters" !== "reuters
  world", so the suffix shipped. Fix is asymmetric word-boundary
  prefix-match: tail must be a SHORTER prefix of publisher (with a
  trailing space delimiter), e.g. tail "Reuters" against publisher
  "Reuters World" → strip. The inverse direction (publisher prefix
  of tail) is rejected ON PURPOSE — "Story - AP News analysis" with
  publisher "AP News" must NOT strip because "analysis" is editorial
  content extending the publisher name, not a desk-name suffix.

Both helpers wired into digestStoryToUpstreamTopStory in two-stage
order: prefix first, then suffix (handles "Video: Story - Al Jazeera"
correctly).

Test coverage:
  - REGRESSION (May 13 brief) for the Sarmat/Reuters/Reuters World case
  - REGRESSION (May 12 brief) for the Video: Philippine senator case
  - REJECTS-the-inverse coverage for AP News / AP News analysis
  - Word-boundary stem rejection ("reuter" / "Reuters", "iran" /
    "iranian press")
  - All 12 prefix variants individually tested

Tests: 69/69 brief-from-digest-stories pass (was 56), 8669/8669 full
unit suite pass, typecheck clean, biome clean.

* docs(brief): fix stale 'either direction' comment to reflect asymmetric one-directional match (PR koala73#3673 review)

The inline comment in stripHeadlineSuffix said 'word-boundary prefix
in either direction' but the actual implementation (and the
isPublisherWordPrefix docstring) is one-directional only: tail must be
a SHORTER prefix of publisher, never the reverse. The asymmetry is
intentional and load-bearing — it's what blocks editorial suffixes
like 'AP News analysis' from stripping when publisher is 'AP News'.

The 'either direction' wording was left over from round-1 of the same
PR's internal iteration. Updated comment names the asymmetry
explicitly and cross-references the full rationale on
isPublisherWordPrefix.

Comment-only change. Behaviour unchanged: 69/69 brief-from-digest
tests still pass, biome clean.

* fix(brief): prioritize critical topic clusters

* test(brief): lock ordering tie breakers

…erence telemetry (koala73#3690)

* fix(brief): exclude opinion/analysis from the pool + lead↔card-#1 coherence telemetry

PR C of the brief-pipeline plan (Phase 3 F3 + Phase 4 F4). Stacked on
PR B (koala73#3688).

── F3: opinion/analysis exclusion ───────────────────────────────────

The May 14 brief ranked a Le Monde opinion column ("'Russia's
invasion of Ukraine could have warned Trump…'", by columnist Gilles
Paris) as story #1, tagged Critical, ahead of a nuclear ICBM test.
The brief is event-driven intelligence — an op-ed column is not an
event.

- New `server/_shared/opinion-classifier.js` — single shared
  classifier `classifyOpinion({ title, link, description })`. Tiered:
  STRONG signals (URL /opinion//views//commentary//editorial//op-ed/
  /columnists/ segments; explicit Opinion:/Analysis:/Commentary:/
  Op-Ed: headline prefix) classify alone; CORROBORATING signals
  (whole-headline quote-wrap, description framing words, /analysis/
  URL) need a STRONG signal OR two corroborating. Conservative — a
  false negative ships one op-ed; a false positive silently drops a
  real event.

  Note: the plan expected ingest-time byline/section metadata, but
  the parsed RSS item and the story:track:v1 row carry neither — only
  title, link, description. Both layers classify from the same three
  signals.

- Two-layer filter:
  · Ingest (list-feed-digest.ts) — ParsedItem gains `isOpinion`,
    set by classifyOpinion in parseRssXml; buildStoryTrackHsetFields
    stamps `isOpinion` ('1'|'0') on the story:track:v1 row.
  · Read-time (buildDigest) — drops rows where `isOpinion === '1'`;
    for residue rows ingested BEFORE the stamp shipped (no isOpinion
    field), re-classifies from the persisted title/link/description.
    Emits `[digest] buildDigest opinion filter dropped N` telemetry.

  Stamps rather than drops at ingest — story:track rows feed more
  than the brief; only buildDigest (the brief's read path) filters.

── F4: lead ↔ final-card-#1 coherence ───────────────────────────────

The synthesis emits `lead` and `rankedStoryHashes` independently,
with no constraint that the lead is ABOUT the rendered first story.
And `rankedStoryHashes[0]` ≠ `data.stories[0]` — orderBriefCandidates
re-sorts by severity/topic/score after the LLM rank. On May 14 the
lead was about the Ukraine-energy story while data.stories[0] was
the Le Monde column.

- New `leadGroundsAgainstStory(lead, headline)` in brief-llm.mjs —
  reuses the checkLeadGrounding anchor machinery (capitalised, ≥4
  chars, stopword-filtered headline anchors; token-set membership)
  but with a FIXED threshold of 1. checkLeadGrounding is the wrong
  fit: scoped to one story, a single headline can carry ≥4 anchors,
  tripping its size-based threshold up to 2 — too strict for a
  coherence question ("same story?", not "how grounded?").

- `composeAndStoreBriefForUser` runs the check AFTER
  composeBriefFromDigestStories, against `envelope.data.stories[0]`
  (the final ordered first card) — NEVER `rankedStoryHashes[0]`. It
  runs in the orchestration layer, not the pure composer, so
  composeBriefFromDigestStories stays I/O-free. Measure-first
  (plan F4 option b): emits `[digest] lead card1 coherence` every
  brief + a `LEAD/CARD-#1 INCOHERENCE` warn on mismatch; ships the
  brief as-is. Once the mismatch rate is known, decide between
  regenerating the lead bound to stories[0] or a separate
  leadStoryHash.

── Tests ────────────────────────────────────────────────────────────

- 11 classifyOpinion unit tests: STRONG signals alone, two-
  corroborating, one-corroborating-NOT-enough, the verbatim May 14
  Le Monde column → opinion, /analysis/ hard-news NOT dropped,
  quoted-PHRASE (not whole-wrap) hard-news NOT dropped, bare-noun
  "Opinion polls…" NOT caught, input safety.
- parseRssXml carries isOpinion (hard-news false / op-ed true).
- buildStoryTrackHsetFields stamps isOpinion '1'/'0' + legacy
  missing-field → '0'.
- 3 leadGroundsAgainstStory tests incl. the May 14 F4 regression
  (lead about Netanyahu vs Le Monde card-#1 → incoherent) and the
  degenerate no-anchor → skip.
- Dockerfile.digest-notifications COPYs the new opinion-classifier.js
  (the transitive-import-closure test caught the missing COPY).

8728/8728 full unit suite, typecheck clean (both configs), biome
clean (3 pre-existing warnings: buildDigest/main complexity already
5x over limit, one useOptionalChain on untouched list-feed-digest
code — none introduced by this PR).

Note: no cache-prefix bump needed — F3 changes the digest POOL
membership (fewer stories), not the synthesis cache material; F4 is
pure telemetry.

* fix(brief): remove unbounded "/opinion-" URL match — false-positives on hard-news slugs

PR koala73#3690 review catch. STRONG_URL_SEGMENTS included `/opinion-` (no
closing slash) as a substring match — it caught hard-news ARTICLE
SLUGS like `/world/opinion-polls-tighten-election`, classifying real
election/polling coverage as opinion. Since buildDigest drops rows
where classifyOpinion() is true, that silently removed those stories
from briefs.

Every other STRONG_URL_SEGMENTS entry is slash-delimited on both
sides — a real path segment. `/opinion-` was the lone unbounded
prefix. Removed it. Genuine opinion sections (`/opinion/`,
`/opinions/`, `/commentary/`, `/editorial/`, `/op-ed/`,
`/columnists/`, `/columns/`) are still caught.

My original test had the gap that let this through: it checked the
"Opinion polls tighten" HEADLINE with link `/world/polls` — never
the `/world/opinion-polls-…` URL form. Added a regression test for
the exact slug, plus a counter-control confirming a genuine
`/opinion/` section still classifies.

12/12 classifier tests, biome clean, typecheck clean.

…ackoff + temp gate (koala73#3694)

* fix(seed-portwatch): rate-limit recovery — concurrency 12→6 + batch backoff + temporary gate

Option E from the 2026-05-14 incident triage: combined response to the
ongoing ArcGIS degradation where both direct AND proxy paths are
throttled.

Background (post-koala73#3676 + koala73#3681 deployed):
  Run #1 (19:55 UTC May 13): 24/30 proxy successes, gate fails 24<50
  Run #2 (00:03 UTC May 14):  5/30 proxy successes, gate fails 5<50

Both koala73#3676 (cold-fetch cap) and koala73#3681 (proxy fallback on timeout) are
working as designed. The remaining problem: Decodo proxy is now also
being rate-limited — successive runs degrade as our usage spikes the
proxy's bucket. ArcGIS itself appears degraded for this dataset.

Three coordinated changes:

(B) CONCURRENCY 12 → 6

Halve in-flight fetches so neither ArcGIS-direct nor Decodo-proxy
sees a 12-concurrent burst. Math at cold-fetch cap 30:
  5 batches × ~60s realistic + 4×5s backoff ≈ 320s
  5 batches × ~90s worst case + 4×5s backoff ≈ 470s
Both fit the 570s bundle budget.

(C) BATCH_BACKOFF_MS = 5_000 (new)

Sleep 5s between batches (skip on last, skip on signal-abort to
preserve SIGTERM responsiveness). Spaces out per-batch bursts so
neither service hits its rate-limit window from our run alone.
20s total added — negligible against the 570s budget.

(Temporary) MIN_VALID_COUNTRIES 50 → 25

Coverage gate lowered so partial-success runs (5-25 successful
fetches + stale-served from cache) can advance seed-meta. Pre-fix,
seed-meta was frozen at 2026-05-12 00:03 UTC for 60+ hours because
no run reached 50. The 25 floor is sized so a 5-success run still
fails (real outage) but a 25+ partial-success run advances meta and
clears the operator-facing WARNING.

Marked TEMPORARY in the inline comment with a 2026-05-20 review
target — must revert to 50 once ArcGIS recovers and runs reliably
exceed 50 again. Test enforces the comment shape so a future
reviewer can't silently normalise 25 as the new permanent floor.

Tests: 87/87 pass (was 85, +2 for BATCH_BACKOFF + MIN_VALID_COUNTRIES
invariants; +1 modification for CONCURRENCY value).

This week's portwatch series:
  koala73#3676  Structural cold-fetch cap (merged)
  koala73#3681  Proxy retry on timeout (merged)
  This   Rate-limit recovery (concurrency + backoff + gate)

* review: break loop on signal-abort + fix wrong PR number in test (P2×2)

Greptile PR koala73#3694 review:

P2 — Signal-abort skipped the sleep but kept the loop running.

Pre-fix: `if (batchIdx < batches && !signal?.aborted) await sleep(...)`
correctly skipped the 5s sleep on abort, but the for-loop continued
to the next iteration and started a fresh batch of up to 6 concurrent
fetches via withPerCountryTimeout (which creates its own AbortController
not linked to the parent signal). Net effect: the actual SIGTERM
backstop was onSigterm → process.exit(1), not the abort guard.

Fix: `if (signal?.aborted) break;` before the sleep — exits the loop
immediately so SIGTERM doesn't start additional in-flight work.

P2 — Test comment cited wrong PR number (koala73#3683 vs koala73#3694).

The "Halved from 12 → 6 on 2026-05-14 (PR koala73#3683)" comment in the
CONCURRENCY test referenced a non-existent PR in this series. Fixed
to koala73#3694 so a future reviewer following the comment lands on the
right diff.

Tests: 87/87 still pass. Updated the BATCH_BACKOFF_MS test to assert
both the new `break` shape and the simplified sleep condition; the
previous combined `batchIdx < batches && !signal?.aborted` pattern is
now split across the two checks.

* review: bypass 80% degradation guard in cap-mode (P1)

Greptile PR koala73#3694 round 3 P1: with the temp coverage gate lowered to 25,
a cap-mode partial-success run would CLEAR the coverage gate
(countryData.size ≥ 25) but STILL fail the unchanged degradation guard
(countryData.size < prevCount × 0.8 ≈ 139 in the incident state) —
seed-meta never advances, WARNING persists, PR's main recovery claim
broken.

Math at the current incident state:
  prevCount               174
  degradation threshold   0.8 × 174 = 139
  cap-mode realistic      30 cold-fetch + ~24 stale-served = ~54
  → 54 << 139 → DEGRADATION GUARD FAILS → extendTtl + return
  → seed-meta stays frozen, WARNING persists.

Fix shape: signal cap-mode from fetchAll() back to main() via a new
`capTriggered: bool` field plus `servedStaleCount` / `droppedTooOldCount`
/ `droppedNoCacheCount` counters. main() bypasses the 80% guard for
cap-mode runs (intentional partial coverage by design — see koala73#3676's
rotation contract) and logs a `PARTIAL PUBLISH (cap-mode)` line with
the fresh/stale split.

The bypass is scoped: non-cap runs (where needsFetch ≤ 30 and we
fetched all misses normally) STILL apply the 80% guard so silent
data-loss scenarios (ArcGIS regression silently drops 100 → 50
countries) remain caught.

Logic shape:
  if (!validateFn) → extendTtl + return                    [coverage gate]
  if (capTriggered) → log PARTIAL PUBLISH + fall through   [bypass]
  else if (countryData.size < prevCount × 0.8) → extendTtl + return [silent-loss guard]
  → publish + advance seed-meta

Tests (3 new): fetchAll-shape (returns the 4 new fields), bypass
shape (else-if structure ensures cap-mode skips the guard), bypass
exclusivity (non-cap runs still enforce the guard with exactly 2
references to `prevCount × 0.8` — one in condition, one in error msg).
90/90 pass (was 87).

…e retries (koala73#3701)

* fix(seed-portwatch): surface degraded ArcGIS 400 body + cap degenerate retries

ArcGIS Daily_Ports_Data, during degradation episodes, returns HTTP 200
with a 400 error body (`Cannot perform query. Invalid query parameters.`)
after 30-56s of server processing. Our 45s FETCH_TIMEOUT fires before
the body lands, callers see AbortError, the existing circuit-breaker
that pattern-matches the error message never fires, and the seeder
treats every degraded country as a generic timeout (then a doomed proxy
retry, then another doomed first-direct-retry from the one-shot
fetchWithRetryOnInvalidParams).

Two surgical fixes:

1. **Diagnostic body-capture on timeout.** When the direct fetch
   hits its FETCH_TIMEOUT (not a caller-signal abort), make ONE extra
   fetch with +20s budget purely to read the response body. If it
   contains `body.error`, throw with the real ArcGIS message so the
   upstream circuit-breaker can act on the upstream-degradation signal.
   Gated to fire once per process — the proxy budget Greptile P2'd on
   PR koala73#3681 stays intact for subsequent timeouts.

2. **Bail-don't-retry threshold on Invalid-query-parameters.** Preserve
   the legit one-shot retry for transient single-call flakes (2026-04-20
   BRA/IDN/NGA pattern), but cap total retries-on-this-error per
   process at 5. Beyond that, throw a clear `ArcGIS degraded — N errors`
   message so the operator sees the regression class instead of N
   identical retries × 45s burning the 540s container budget.

Investigation that drove this: direct curl probes against
services9.arcgis.com revealed (a) HTTP 200 with a 400 error body, (b)
non-deterministic results across attempts within minutes (same query,
ERR → OK → ERR → 504), (c) WHERE-clause reshaping helps unevenly but
never reliably. The right frame is "upstream is degraded, surface it
and protect the container budget" not "we're being rate-limited."

The existing cap-mode + stale-serve + temp-gate-25 logic (PRs
koala73#3676/koala73#3681/koala73#3694) already rides through partial coverage — these
changes just make sure we don't waste the container budget while
ArcGIS recovers, and that the next diagnostic round has the real error
message instead of generic AbortError.

Tests:
- 3 new structural pattern-match tests covering ERROR_BODY_CAPTURE_EXTRA_MS,
  _captureErrorBodyAfterTimeout helper, once-per-run gating,
  INVALID_PARAMS_RETRY_THRESHOLD, counter increment + threshold check,
  reset helpers.
- 96/96 tests pass (was 93 before).

* review: gate cap-mode bypass on fresh upstream contact + abort-aware backoff sleep (P1+P2)

P1 — Cap-mode bypass could publish a stale-only canonical list as healthy.

Pre-fix the capTriggered bypass of the 80% degradation guard was
unconditional. With the lowered MIN_VALID_COUNTRIES=25 + cap-mode +
servedStale entries seeding countryData, a run with all-stale entries
(freshFetched=0, cacheHits=0, servedStale=27, dropped=147) could:
  - Pass validateFn (countryData.size >= 25 — all from stale-cache)
  - Bypass the 80% guard (capTriggered=true)
  - Shrink the canonical list from ~174 → 27 stale-only entries
  - Advance seed-meta, clearing the operator-facing WARNING

Fix: gate the bypass on freshFetchedCount + cacheHitCount ≥
MIN_FRESH_FETCH_FOR_CAP_BYPASS (5). Below that floor, cap-mode is NOT
rotational steady-state — it's an ArcGIS-completely-down scenario, and
the run falls through to the 80% guard (which extendExistingTtl-only,
preserves canonical list, keeps WARNING visible). New log line
"CAP-MODE BYPASS REFUSED: only N fresh upstream contacts" makes the
refusal observable to operators.

This composes with the INVALID_PARAMS_RETRY_THRESHOLD=5 fix earlier in
this PR — that change increases the rate at which cold fetches return
errors during upstream degradation, making the stale-only scenario
this P1 guards against MORE likely, not less.

P2 — Inter-batch backoff sleep is now abort-aware.

The 5s BATCH_BACKOFF_MS wait used a plain setTimeout that ignored the
caller signal mid-sleep, so a SIGTERM during the wait still forced the
full 5s before observing the abort. Now races the setTimeout against
signal's abort event so the loop exits immediately on a real
cancellation (the signal?.aborted check at the top of the next
iteration then `break`s the loop). Cosmetic change but matches the
PR-body intent that the loop is SIGTERM-responsive.

Tests:
- 2 new structural pattern-match tests covering MIN_FRESH_FETCH_FOR_CAP_BYPASS,
  freshFetchedCount counter, upstream-contact gate, CAP-MODE BYPASS REFUSED
  branch, and the abort-aware sleep shape.
- 4 existing tests updated to match the new destructure shape + new prevCount
  × 0.8 reference count (now 4: 2 conditions + 2 Math.ceil sites, one set per
  guard branch).
- 95/95 portwatch tests pass; 8804/8805 npm run test:data (1 flake is the
  pre-existing scripts/_bundle-fixture-env-*.mjs race between
  bundle-runner.test.mjs and news-classify-cache-prefix-audit.test.mjs,
  unrelated to this change).

* review: gate body-capture on success not attempt + short-circuit proxy-confirmed errors (P2×2)

Two Greptile P2 comments on PR koala73#3701 review round 2.

P2 #1 — Body-capture gate fired on first ATTEMPT, not first SUCCESS.

During consistent degradation, the 20s ERROR_BODY_CAPTURE_EXTRA_MS
window isn't long enough — ArcGIS needs 30-56s of server processing
per query from connection start. A failed first capture (capture also
timing out) flipped the once-per-run flag, locking out every
subsequent timing-out country from ever capturing the body. The
diagnostic value could be lost for an entire run, defeating the whole
point of the capture path.

Fix: track SUCCESS count (cap at 1 — we only need the body shape once)
AND ATTEMPT count (cap at 3 to bound total cost). Null captures consume
an attempt but not a success, so subsequent timing-out countries keep
trying. Worst-case cost: 3 × 20s wall-clock per run, still within
PER_COUNTRY_TIMEOUT_MS=90s caps.

P2 #2 — Proxy-confirmed "Invalid query parameters" still triggered retry.

When arcgisProxyRetry returns an error body, the thrown message has
the `(via proxy after ${reason})` prefix and still matches
`/Invalid query parameters/i`. fetchWithRetryOnInvalidParams counted
it, then retried via fetchWithTimeout — which would timeout → proxy →
hit the same authoritative error, burning a full direct+proxy cycle
for nothing. The proxy response IS the definitive upstream signal;
retrying is meaningless.

Fix: counter still increments (proxy-confirmed errors are valid
degradation signals for the threshold), but short-circuit the retry
with `if (/via proxy after/i.test(msg)) throw err`.

Tests: 2 new structural pattern-match tests + 1 existing test relaxed
to accept the new `if (captured?.error) { successCount++; throw ... }`
shape. 96/96 portwatch tests pass; 8806/8806 npm run test:data
(unrelated bundle-fixture race cleared this run).

* review: threshold-before-short-circuit + realistic capture budget (P1+P2 round 3)

P1 — Proxy-confirmed Invalid Params never reached the degradation threshold.

Pre-fix the proxy-confirmed short-circuit (`/via proxy after/i` → throw)
ran BEFORE the threshold check. So during the actual incident — where
nearly every error arrives via the proxy fallback path — every error
threw the per-country proxy message and the clean
`ArcGIS degraded — N 'Invalid query parameters' errors` message was
unreachable.

Fix: reorder so threshold check fires first, then proxy short-circuit.
Counter still increments for proxy-confirmed errors (they contribute to
the threshold), and once the threshold is exceeded the degraded
message surfaces regardless of whether the error came from direct or
proxy.

P2 — 20s capture budget didn't realistically catch 30-56s responses.

The 3-attempts-× 20s fix was bounded but each attempt was a FRESH
request starting from t=0 (the original 45s direct fetch was already
aborted). For the observed degraded response class (30-56s server-side
from request start), each 20s capture aborts before the body lands.

Fix: bump ERROR_BODY_CAPTURE_EXTRA_MS 20s → 40s. Math under
PER_COUNTRY_TIMEOUT_MS=90s:
  direct (45s timeout) + capture (40s budget) = 85s, leaving 5s before
  the per-country wrap fires. Proxy retry effectively dies for
  timing-out countries in this mode — accepted tradeoff because the
  proxy is itself degraded (Decodo throttled per koala73#3694 history), so it
  wasn't helping anyway. Attempt cap stays at 3 (across the run, not
  per country) so the diagnostic value is captured by 3rd timing-out
  country; subsequent countries go straight to proxy.

Tests: 1 new ordering test using src.search() index comparison to
assert threshold-throw-before-proxy-short-circuit; 2 existing tests
updated for the bumped constant and the longer span between
increment and short-circuit. 97/97 portwatch tests pass; 8806/8807
npm run test:data (1 flake = pre-existing _bundle-fixture-* race
between bundle-runner and news-classify-cache-prefix-audit).

…News placeholders (koala73#3717)

* fix(feeds): address koala73#3715 review — server parity, 0-item placeholders, parity guard

Reviewer P1 #1: I only updated the CLIENT feed config in koala73#3715; server-side
_feeds.ts:263 still pointed Blockworks at the dead blockworks.co/feed, so
the digest path (loadNewsCategory → /api/news/v1/list-feed-digest) kept
hitting the Cloudflare-blocked upstream and caching zero items. Same
mirror-drift class as PR koala73#3712's allowlist miss; recorded in
feedback_implement_every_site_my_own_diagnosis_enumerated.

Reviewer P1 #2: I validated with `grep -c '<item>'` (counts LINES, not
occurrences) and called the URLs "fine". Re-counted with
`grep -o '<item>' | wc -l`:

- Kitco News (50), Kitco Gold (50), Mining Weekly (100), FX Empire Gold
  (76), Mining Journal (19) — all real volume, fine.
- Blockworks: 0 items across every probe (site: + time-window variants).
  Google doesn't index blockworks.co for News (Cloudflare likely blocks
  Googlebot on the same wholesale tier).
- Commodity Trade Mantra: 0 items via site:, 2 items via "" / 30d.
  Effectively unindexed.

Fix: REMOVE both feeds rather than ship silent placeholders.

- Blockworks (client + server): The Block already exists on both sides
  and covers the same institutional-crypto territory.
- Commodity Trade Mantra (client): commodity-news still has Mining.com /
  Bloomberg / Reuters / S&P Global / CNBC — coverage isn't lost.

New parity test (tests/feeds-client-server-parity.test.mjs) fails when a
feed NAME appears on both sides with inconsistent routing (one side
Google News, the other direct). It exposed 12 PRE-EXISTING drifts (each
its own per-feed judgment) — grandfathered as KNOWN_DRIFTS so this PR
doesn't block on unrelated tech debt. New drift fails the test. Locked-in
regression: neither file may re-add `https://blockworks.co/feed`.

* fix(feeds): remove server-side Commodity Trade Mantra + extend parity test

Reviewer caught a P1 on koala73#3717: I removed Commodity Trade Mantra from the
CLIENT in the first commit of this PR but missed the SERVER copy at
server/worldmonitor/news/v1/_feeds.ts:341. The failure mode the reviewer
described is real:

  1. Server fetches 7 feeds for commodity-news, including CTM
  2. Server ranks by importanceScore, then .slice(0, MAX_ITEMS_PER_CATEGORY)
     (list-feed-digest.ts:1076-1082) — CTM items COUNT toward the cap
  3. Client filters by `enabledNames` (data-loader.ts:908-914) — CTM not in
     client config → CTM items DROPPED
  4. Net: invisible items crowd out visible ones, shrinking the panel

Two changes:

1. Server entry removed with a comment naming the exact failure mode.

2. tests/feeds-client-server-parity.test.mjs extended with:
   - Fixed extractor: the previous single-line regex missed ~46 multiline
     locale-keyed entries on the client side (France 24, EuroNews, DW News,
     etc.), which would have made the new orphan check fire false positives.
     New extractor scans up to 600 chars forward from each `name:` for the
     matching `url:`, capturing locale-object URLs intact.
   - New test: `no NEW server-only feed entries`. Server-only orphans are
     the exact crowd-out failure mode the reviewer caught. Five existing
     orphans grandfathered (Trump-Truth-Social, White House Actions, First
     Round Review, YC News, YC Blog) — each should be reviewed and either
     mirrored on the client OR removed from server. Set should SHRINK.
   - New regression: CTM specifically must not reappear on the server.
   - DW News added to KNOWN_DRIFTS: client uses mixed direct/Google-News
     per locale, server uses pure direct; classifier treats any-locale-GN
     as GN. Worth reconciling separately, not in this PR.

All 5 parity tests pass; typecheck + biome clean.

…he panel can't hang forever (koala73#3718)

* fix(daily-market-brief): cap LLM summarizer + total build budget so the panel can't hang forever

## Symptom
PRO users were reporting the Daily Market Brief panel stuck on "Building
daily market brief..." indefinitely after a cache miss. The try/catch
around the summarizer was decorative — it only handles REJECTIONS, and a
hung upstream (LLM provider slow, Vercel cold-start + slow network) leaves
the awaited promise pending forever, never reaching catch.

## Verified root cause (no extrapolation this time)

Traced the chain top-to-bottom for an enforced timeout:

  loadDailyMarketBrief                   data-loader.ts:1635
   → buildDailyMarketBrief               daily-market-brief.ts:381
     → generateSummary                   summarization.ts:197
       → summaryResultBreaker.execute    no timeoutMs option (line 20)
         → tryApiProvider                summarization.ts:76
           → summaryBreaker.execute      no timeoutMs option
             → newsClient.summarizeArticle(...)   accepts options.signal
                                                   but no caller passes one

`vercel.json` has no `maxDuration` override; the default function timeout
is whatever Vercel applies, but the CLIENT awaits TCP keepalive, so from
the browser's perspective the fetch can be effectively forever.

## Fix

`src/utils/with-timeout.ts` — new utility. Races a promise with a
deadline; the source promise is not cancelled (JS has no general cancel)
but the await unblocks via TimeoutError so caller fallback paths fire.

`src/services/daily-market-brief.ts:432` — wrap the
`summaryProvider(...)` call in `withTimeout(..., 45_000, 'daily-brief-
summary')`. On timeout the EXISTING catch (line 444) falls back to the
rules-based summary (`buildRuleSummary`) — already pre-computed at the
top of the function, so this costs nothing extra. Added
`summarizerTimeoutMs?` to `BuildDailyMarketBriefOptions` so tests can
exercise the fallback in ~30ms without waiting 45s.

`src/app/data-loader.ts:1644` — wrap `getCachedDailyMarketBrief(...)` in
a 3s `withTimeout` + `.catch(() => null)`. A hung persistent-cache layer
can't keep the panel on its default Loading state — fall through to
"build from scratch" instead.

`src/app/data-loader.ts:1667` — wrap `buildDailyMarketBrief(...)` in a
60s `withTimeout` (outer budget). Inner summarizer has its own 45s cap;
this catches the dynamic-import path (`getDefaultSummarizer()` import
hanging on a slow chunk fetch), `_collectRegimeContext` /
`_collectYieldCurveContext` / `_collectSectorContext` exotic hangs
(though they're already in `Promise.allSettled`), etc. The existing
catch (line 1693) serves the cached version or shows an error.

## Test plan
- `tests/with-timeout.test.mjs` — 7 unit tests: resolve-first, reject-
  first, hang→TimeoutError, timer cleanup (proves a 5s-budget test
  exits at 5ms not 5s), onTimeout-once, onTimeout-not-called-on-success,
  onTimeout-throw-doesn't-hijack.
- `tests/daily-market-brief.test.mts` — new regression test
  `a hanging summarizer must not stall the brief` reproduces the prod
  shape with an injected `() => new Promise(() => {})` (pending forever)
  and asserts the brief returns within the timeout with
  `provider: 'rules'` and `fallback: true`. Test elapsed ~33ms.

Local gate: typecheck PASS, biome on touched files PASS (4 pre-existing
warnings on data-loader.ts unrelated complexity scores, present on main
too), lint:api-contract PASS.

* fix(daily-market-brief): close Greptile koala73#3718 review gaps

## P2 (Greptile, valid)

The original PR wrapped the upfront cache read in withTimeout but I
missed the recovery cache read inside the catch block at
`data-loader.ts:1715` — same hang risk in the exact path the fix
creates. If the outer 60s build budget fires AND IndexedDB is also
degraded, the recovery read keeps the panel stuck.

Wrap the recovery `getCachedDailyMarketBrief(...)` in the same 3s
`withTimeout` (label `'daily-brief-cache-read-recovery'` so logs
distinguish the two cache reads) + the existing `.catch(() => null)`
absorbs both the TimeoutError and any persistent-cache failure into
the null-result branch that the showError fallback already handles.

## Nits

- Renamed `tests/with-timeout.test.mjs` → `.mts` to match the project
  convention for tests that directly import `.ts` sources (see
  `daily-market-brief.test.mts`). Functionally identical; lines up
  with the rest of the test directory.

- Added the missing test case Greptile flagged: assert `onTimeout` is
  NOT invoked when the source REJECTS first (the prior set only covered
  the resolve-first path). Without `.finally(clearTimeout)` the timer
  would still fire after `Promise.race` already settled — onTimeout
  would run as a phantom side-effect after the caller had already moved
  on. The test waits 80ms past the 50ms budget to prove the timer was
  cleared.

All 8 `withTimeout` cases pass; the daily-market-brief regression test
still falls back within budget; typecheck + biome clean.

* fix(daily-market-brief): close two more hang paths (koala73#3718 review round 2)

Both findings valid — and embarrassing, because they're the same hang
class this PR was opened to fix. The earlier passes wrapped the LLM
summarizer call (the obvious suspect) but missed three more unbounded
awaits in the same function.

## P1 #1 — Context collectors hung before the 60s envelope started

`_collectRegimeContext` calls `client.getFearGreedIndex({})` and
`_collectYieldCurveContext` calls `client.getFredSeriesBatch(...)` —
both unbounded RPCs. `Promise.allSettled` only converts REJECTIONS into
status:rejected; it waits forever for pending-forever promises. My
withTimeout envelope was wrapped around `buildDailyMarketBrief(...)`
AFTER the allSettled line, so a hung context collector kept the panel
on "Building daily market brief..." forever — exactly the symptom this
PR was supposed to fix.

Wrap each RPC-using collector in `withTimeout(..., 8_000)` with its own
label. 8s is generous for an RPC and leaves >36s of the outer 60s
budget for the actual LLM call. `_collectSectorContext` is sync (only
reads hydrated data) so it needs no wrap; allSettled accepts non-
promises directly.

## P1 #2 — Cache write blocked render

`await cacheDailyMarketBrief(brief)` ran BEFORE the render call, so a
hung IndexedDB / Tauri-Store write meant the user never saw the
finished brief even though it was sitting in memory ready to display.
The build budget proved nothing by itself.

Render first, persist after. Cache write is now fire-and-forget with
its own 5s budget (`void withTimeout(...).catch(...)`) — a hung backend
becomes "no warmup for tomorrow's load" instead of "panel stuck on
Building forever."

## On the catch-path cache read

The reviewer also flagged the catch-block `getCachedDailyMarketBrief`
call as unbounded; my previous commit (Greptile P2 fix) already wrapped
that one with `withTimeout(..., 3_000, 'daily-brief-cache-read-recovery')`.
Verified still present in this round.

Local gate: typecheck PASS, biome on data-loader.ts PASS (4 pre-
existing complexity warnings unrelated to this diff), with-timeout
suite 8/8, daily-market-brief regression still passes.

…koala73#3748) (koala73#3750)

PR koala73#3748 added the same fix to feelgood-classifier.js and explicitly
documented the latent gap in opinion-classifier.js:93 as a follow-up
backport. Greptile's review of koala73#3748 also flagged this. Closing the
gap now.

Mechanism (same as adv-002 in koala73#3748):
- Pre-fix: `lowerLink.includes('/opinion/')` on the raw link string
  returns true for any URL whose query string OR fragment contains
  "/opinion/" — including legitimate aggregator tracking params like
  `?utm_campaign=/opinion/promo` or news.google.com style redirects
  whose original `?url=...` carries an /opinion/ section. Hard news
  with that tracking shape would silently classify as opinion and
  drop from the brief.
- Fix: `safePathname(link)` runs `new URL(link).pathname.toLowerCase()`
  inside try/catch and returns '' on malformed URLs. Matching is then
  done against the parsed pathname only — query strings and fragments
  cannot spoof a section match. Mirrors the helper in
  feelgood-classifier.js byte-for-byte.

Two sites in classifyOpinion shared the bug:
- STRONG #1 URL match (line 93)
- CORROBORATING URL match (line 103) for /analysis/ + /analyses/

Both now run against the parsed pathname.

6 new regression tests prove the injection vector is closed (tracking
param, URL fragment, malformed URL defensive) AND that the fix does
not over-correct (genuine /opinion/ + genuine /analysis/ with
tracking params still classify correctly). All 12 original tests
still pass.

…ala73#3786 follow-up) (koala73#3819)

Greptile flagged on the now-merged PR koala73#3786 review that the 4th test
in the secret-guard suite was passing vacuously: it imported
runtime-config.ts and asserted `getRuntimeConfigSnapshot().secrets`
contained no platform-only secret, but in node:test `import.meta.env`
is undefined, so `readEnvSecret()` returns '' for every key regardless
of process.env or requiredSecrets contents. The snapshot is always
empty and the test always passes — even if a contributor adds
WORLDMONITOR_API_KEY to a requiredSecrets array (the exact regression
test #1 catches via source-grep).

This commit was originally made on the koala73#3786 branch but missed the
merge window. Re-applying against new main as a follow-up PR.

The 3 remaining static-analysis tests catch every realistic regression
path. The honest runtime check is a bundle-grep at deploy time:
  npm run build && grep -r "WORLDMONITOR_API_KEY" dist/

Pattern documented in
~/.claude/skills/test-ci-gotchas/reference/vacuous-test-when-runtime-injected-context-absent.md
(extracted this session).

…oala73#3820)

* fix(api): use timing-safe compare for x-probe-secret (koala73#3803)

Issue koala73#3803 flagged that api/seed-contract-probe.ts used `secret !== expected`
for the x-probe-secret header — a timing oracle on RELAY_SHARED_SECRET,
which is shared across Vercel functions, the Railway relay, and probe
endpoints. Every other internal-auth path in the codebase uses the
`timingSafeEqual` helper from server/_shared/internal-auth.ts.

Same pattern as the koala73#3705 and koala73#3756 fixes: bring the outlier site in
line with the safer convention the rest of the codebase already uses.
The helper exists, the cost is one import + a one-line call change.

Changes:
1. Export `timingSafeEqual` from server/_shared/internal-auth.ts (was
   module-private). Added a JSDoc explaining the use case for callers
   whose secret comes in via a non-Authorization header
   (x-probe-secret, x-internal-key, etc.) — they need the primitive,
   not the Bearer-token-shaped `authenticateInternalRequest`.
2. api/seed-contract-probe.ts: import + replace `!==` with
   `timingSafeEqual(secret, expected)` (with a `?? ''` guard for
   missing header). Updated comment to mention the timing-oracle
   rationale and cross-reference issue koala73#3803.

New regression test (tests/no-non-timing-safe-secret-compare.test.mts):
- Walks all api/*.{ts,js,mjs,cjs} files (skipping tests and vendored
  dirs).
- Greps for `(secret|token|bearer|sharedSecret|...) (===|!==)
  (process.env.* | expected*)` — the shape of a timing-leaky secret
  comparison.
- Comment-stripped to avoid false-positives on doc comments.
- An allowlist exists for future documented exceptions (currently
  empty).
- A second targeted test asserts seed-contract-probe.ts specifically
  uses the helper, so this exact regression can't reappear silently.

Test plan:
- [x] 2/2 new tests pass
- [x] `npm run typecheck` clean
- [x] Manually verified the source-grep regex catches the pre-fix
      shape (replacing the new call with the old `!==` makes test #1
      fail with the right file in the violations list).

Closes koala73#3803

* test(security): regex also catches yoda-style reverse comparison (koala73#3820 review)

Greptile P2 review on PR koala73#3820 caught that the source-grep regex only
matched the forward operand order:

  secret === process.env.FOO       ← caught
  process.env.FOO === secret       ← MISSED (yoda-style or copy-paste)

A maintainer who wrote the reverse form (common when porting code
from languages where yoda comparisons are idiomatic, or when
auto-completing from `process.env.FOO`) would have silently slipped
the timing oracle past the guard.

Fix: split the pattern into two named arms (forward + reverse) and
combine via alternation. The reverse arm anchors on the env-var-ish
LHS and matches the secret-variable RHS, mirroring the forward arm.

Also added a meta-test that exercises the regex against 10
representative strings (3 forward, 3 reverse, 4 negative) and
asserts the match/no-match contract. This pins the coverage so a
future "simplification" of the regex can't silently regress it.

3/3 tests pass; typecheck clean.

…diness refresh (koala73#3833)

* fix(map): swallow deck.gl/maplibre interleaved-mode render race

Module-scoped installDeckInterleavedRaceFilter() adds a one-shot
capture-phase window error listener that suppresses
  TypeError: Cannot read properties of null (reading 'id')
when the throwing file matches /deck-stack-…\.js/. Called from
initDeck() — idempotent across recreateWithFallback / HMR.

Trigger is the deck.gl 9.x + maplibre-gl 5.x interleaved-mode race:
setProps({layers}) finalizes a layer while maplibre's painter is
mid-render, so the painter callback iterates a list that just had
a slot nulled. MapboxOverlay.onError can't see this — maplibre, not
deck, owns the throwing callstack.

Sentry's beforeSend in main.ts:313-315 already drops this pattern
for telemetry, so impact is purely console-noise removal. First-
party .id crashes still surface because the filter requires BOTH
the message shape AND the deck-stack chunk filename.

* fix(panels): gate tech-readiness refresh by viewport on non-happy variants

Replace `if (SITE_VARIANT !== 'happy')` with `… && shouldLoad('tech-readiness')`,
matching the thermal-escalation gate one line below.

Why this fixes the energy/finance/commodity-variant 5s timeout:
App.ts:576-583 merges ALL_PANELS into panelSettings on every variant
for cross-variant pref carryover, so shouldCreatePanel('tech-readiness')
returns true everywhere — but the bootstrap seed key only exists on
full + tech variants, so the unconditional refresh() at
services/economic/index.ts:694 times out on every other variant's
data-loader cycle. Viewport-gating prevents the refresh fan-out from
firing on variants whose panel will never be visible.

* fix(panels): close two boot-path bypasses on tech-readiness variant gate

Reviewer found that PR koala73#3833's first cut was incomplete — the
`shouldLoad('tech-readiness')` gate I added in data-loader was
silently bypassed on initial boot, and a parallel auto-refresh path
in panel-layout was never gated at all.

P1 #1 — data-loader.ts:602
  `shouldLoad(id)` returns `forceAll || isPanelNearViewport(id)` at
  data-loader.ts:444. App.ts:1226 calls `loadAllData(true)` on startup,
  forcing `shouldLoad` to true on every variant — so commodity/finance/
  energy were still enqueueing `TechReadinessPanel.refresh()` at boot.

P1 #2 — panel-layout.ts:1278
  The `lazyPanel('tech-readiness', ...)` factory calls `void p.refresh()`
  unconditionally inside the dynamic import. Because App.ts:577-583
  merges ALL_PANELS into panelSettings on every variant for cross-variant
  pref carryover, `shouldCreatePanel('tech-readiness')` (just a key-
  existence check at panel-layout.ts:854) is true everywhere — and the
  hide-disabled path at panel-layout.ts:2028-2030 runs AFTER the import's
  refresh has already fired the 5s `/api/bootstrap?keys=techReadiness`
  fetch. So the variant gate has to live inside the factory, not after.

Fix — introduce `isPanelInVariantDefaults(key)` helper in src/config/panels.ts
that returns `VARIANT_DEFAULTS[SITE_VARIANT].includes(key)`. Use it in
both auto-refresh paths:

  data-loader.ts:602
    if (isPanelInVariantDefaults('tech-readiness') && shouldLoad('tech-readiness'))

  panel-layout.ts:1278 (factory body)
    if (isPanelInVariantDefaults('tech-readiness')) {
      void p.refresh();
    }

The panel is still CREATED on all variants so users who opt-in via
settings can still see and use it — but the eager fetch only fires
where the bootstrap seeder actually populates the key (full + tech).

Regression test — tests/tech-readiness-variant-gate.test.mts uses a
line-walker (not a strict regex) to assert:
  1. data-loader's techReadiness task is gated by isPanelInVariantDefaults
  2. panel-layout's lazyPanel factory wraps p.refresh() in the same gate
  3. the helper is exported from the @/config barrel

Mutation-tested locally: reverting either gate makes the test fail
with a clear diagnostic naming the bypassed location.

Note — national-debt at panel-layout.ts:1286 has the same factory
shape (eager `void p.refresh()` on a variant-restricted panel that's
only in FULL_PANELS). Left for a follow-up so this PR stays scoped
to the reviewer's findings.

…+ lock down (koala73#3832)

* fix(route-explorer): close Asia→DE lane gap for TW/KR/JP/VN/TH/PH/IN + lock down

PR koala73#3828 fixed HK→Germany by adding `china-europe-suez` and `asia-europe-cape`
to HK's `nearestRouteIds` in `scripts/shared/country-port-clusters.json`, and
explicitly deferred TW/KR/JP/VN/TH/PH because each needed review against actual
shipping data. While triaging a reporter's HK→DE / Premium Stock / WM Analyst
empty-state screenshots (pr-3718 session), I confirmed those six countries plus
IN still fail with `noModeledLane: true` against DE — server reports the gap
honestly, UI shows "No modeled lane for this pair."

Root cause for each is identical to HK's: the country has only
Pacific/intra-Asia/Gulf-oil route IDs, none of which appear in DE's cluster
(`china-europe-suez`, `asia-europe-cape`, `transatlantic`). Server intersection
at server/worldmonitor/supply-chain/v1/get-route-explorer-lane.ts:233-234
returns zero shared routes → noModeledLane=true.

Fix: add `china-europe-suez` and `asia-europe-cape` (the trunk Asia↔Europe
container routes terminating at Rotterdam) to TW, KR, JP, VN, TH, PH, IN.
Both routes are tagged `category: 'container', status: 'active'` in
src/config/trade-routes.ts; they are factually correct for every major Asian
export hub shipping to Northern Europe.

Lock-down: tests/country-port-clusters-asia-europe-lane.test.mts (3 cases)

  1. Data invariant — every Asian port country in {CN,HK,TW,JP,KR,SG,MY,ID,
     TH,VN,PH,IN} must have a non-empty `nearestRouteIds` intersection with
     DE in the cluster JSON. Reverting any of the seven new entries flips
     this test red with a clear "add china-europe-suez and/or asia-europe-
     cape to <ISO2>'s nearestRouteIds" error.
  2. Computed lane — `computeLane('HK','DE','85','container')` must return
     `noModeledLane: false` and a non-empty `primaryRouteId`. This is the
     EXACT shape of the reporter's screenshot (HK→Germany, Electrical &
     Electronics HS2=85, Container, AUTO badge); the original PR koala73#3828 had
     no test pinning this case, so a future contributor reverting the HK
     entry would not be caught.
  3. Full Asian-port matrix — same `noModeledLane: false` assertion for
     all 12 Asian export hubs against DE, defending the algorithm boundary
     (computeLane) on top of the data invariant.

Bite-test: reverting just the data change (git stash on the JSON) flips
tests #1 and #3 red, naming the exact 7 offenders (TW/JP/KR/TH/VN/PH/IN);
test #2 stays green because HK was already fixed by koala73#3828. Confirms each
assertion bites its intended regression.

The 30-query smoke matrix in tests/route-explorer-lane.test.mts (38 cases)
stays 38/38 green. tsc clean.

This complements PR koala73#3828's HK fix and clears the deferred-follow-up note
without touching unrelated areas. The reporter's third screenshot (HK→DE
"No modeled lane") will resolve on next deploy regardless of this PR
(PR koala73#3828 already fixed HK); this PR additionally prevents TW/KR/JP/VN/TH/
PH/IN from showing the same dead-end to other Asia-export-hub users, and
makes both fixes regression-proof.

* fix(route-explorer): address PR koala73#3832 review P1 — IN→DE picks india-europe, not china-europe-suez

Reviewer (correct): adding `china-europe-suez` + `asia-europe-cape` to IN
satisfied `noModeledLane === false` for IN→DE but pushed the resolver to
pick `china-europe-suez` (Shanghai → Rotterdam) as the primary route. The
Route Explorer UI highlights the route's from→to ports, so an Indian
shipment to Germany rendered a Shanghai origin port. The test was vacuous
on this dimension — it only asserted `noModeledLane === false`, never
which route resolved.

Root cause of my original choice: DE's `nearestRouteIds` didn't list
`india-europe`, so no intersection existed for IN→DE via that route. I
papered over the missing intersection by polluting IN's profile with
China-origin trunk routes instead of fixing DE's profile.

Correct fix (follows existing convention): every European country that
already has `china-europe-suez` (the Asia-Europe trunk to Rotterdam) also
gets `india-europe` (the India-Europe trunk to Rotterdam). Both routes
terminate at the same Northern-European hub; the data model treats trunk
routes as broadly serving any European port country. Applied uniformly to
DE, GB, FR, IT, ES, NL, BE, PL, SE, DK, FI, GR, PT (the 13 European
countries with `china-europe-suez`).

IN's `nearestRouteIds` reverts to `["india-europe", "india-se-asia",
"gulf-asia-oil"]` — no more China-origin pollution. IN→DE now intersects
on `india-europe` (only shared route), so the resolver picks it
unambiguously.

Test tightening (the actual lock the reviewer asked for):
`tests/country-port-clusters-asia-europe-lane.test.mts` adds an
`EXPECTED_PRIMARY_ROUTE_TO_DE` map and asserts `computeLane(<iso2>, 'DE',
'85', 'container').primaryRouteId === expected` for all 12 Asian-port
countries. IN→DE must equal `'india-europe'`; all others must equal
`'china-europe-suez'`. Removed the weaker "noModeledLane: false for the
matrix" test — the stricter primaryRouteId assertion subsumes it (a
`noModeledLane=true` response fails the equality check with a clear
message naming the offender).

Bite-test of the new assertion:
  - Revert `india-europe` from DE only → both tests #1 and #3 flip red
    naming IN: data invariant says "IN → DE: shared routes = []", lane
    assertion says "IN → DE: noModeledLane=true (expected
    primaryRouteId='india-europe')". Confirms the assertion bites the
    exact reviewer-flagged regression.
  - HK→DE remains green (china-europe-suez is correct for an HK origin).
  - 38/38 of the existing `tests/route-explorer-lane.test.mts` smoke
    matrix stays green. tsc clean.

The other Asian origins (TW/JP/KR/VN/TH/PH/SG/MY/ID) already correctly
intersect DE on `china-europe-suez` (added in this PR's first commit) —
that's the trunk route their containers actually take to Europe, so
their primaryRouteId assertion was already correct.

* test(route-explorer): drop redundant HK-specific case (PR koala73#3832 review P2)

Greptile P2: the standalone HK → DE test was fully subsumed by the matrix
test, which iterates over ASIAN_PORT_COUNTRIES (includes HK at index 1)
and asserts `primaryRouteId === EXPECTED_PRIMARY_ROUTE_TO_DE[iso2]`.
HK's expected value is 'china-europe-suez'; a regression on HK fails the
equality check with a clear message, just like every other origin.

Renamed the surviving matrix test to call out HK→DE explicitly so the
provenance trail to the pr-3718 reporter symptom isn't lost.

koala73#3838)

Follow-up to PR koala73#3669 motivated by post-deploy verification: with the
original 25% cap, Lebanon scored #1 globally on this indicator at 14.6%
CPI — the formula's nominal-growth term (18.6% nominal GDP growth in
Lebanon's 2026 forecast) mechanically erodes its 139% debt-to-GDP ratio,
producing a +25.8pt gap that masks the country's actual fiscal/banking
fragility.

The 25% cap was a guess (round number above Turkey's 28%). 10% has a
principled basis: above ~10% CPI, inflation expectations un-anchor and
the inflation-tax mechanism becomes part of a debt CRISIS rather than a
debt RESOLUTION. Post-WWII US/UK/France did successfully inflate away
war debts at 10-20% — but only with capital controls, financial
repression, deep domestic bond markets, and stable institutions. None
of those apply to Lebanon. The IMF DSA framework's own implicit scope
is sub-10% inflation regimes; this PR aligns our cap with that.

Impact:
- 15 countries currently with gap drop to null: Lebanon, Egypt, Nigeria,
  Kazakhstan, Bolivia, Ethiopia, Angola, Myanmar, Suriname, Haiti,
  Malawi, Burundi, South Sudan, Zambia, Kyrgyzstan.
- Coverage: 176 → 161 (still well above the 100-country validate floor).
- New best-5 (anticipated, will verify post-deploy): Norway, Greece,
  Oman, UAE, Cyprus — all defensible.
- New worst-5: Ukraine, Bahrain, Algeria, Botswana, Qatar — also
  defensible (wartime, oil-dependent, etc.).
- Fiscal-3 indicators still score every dropped country; only the
  fourth sub-score (gap) goes null. weightedBlend redistributes weight.

Diagnostic context (NOT addressed in this PR — separate concerns):
The same post-deploy analysis revealed two other failure modes that
this PR does NOT fix: small-economy noise (Timor-Leste gap=-50,
Kiribati gap=-16 — IMF data sparse for tiny economies amplifies into
extreme gap values) and low-debt instability (Brunei gap=-10.6 at
debt=1%). Those are arguably separate issues; this PR scope is just
the inflation-tax distortion (Lebanon-class), which is what users
would most obviously misinterpret.

Files:
- scripts/seed-recovery-fiscal-space.mjs: INFLATION_GAP_CAP_PCT 25 → 10,
  expanded comment with the rationale + WWII analog
- server/worldmonitor/resilience/v1/_indicator-registry.ts: updated
  description + tier comment; coverage 150 → 140 (realistic post-cap)
- docs/methodology/country-resilience-index.mdx: prose update
- docs/methodology/indicator-sources.yaml: reviewNotes update
- tests/seed-recovery-fiscal-space.test.mts: cap-pin assertion 25 → 10
  with provenance comment

Tests: 149/149 affected resilience tests pass.
Lint + tsc clean.

No change to: formula, goalposts, weights, validation floors, schemaVersion.
Backward-compatible — the next seeder run repopulates the canonical blob
with the new cap; old scorer code reads the new blob fine.

…9 regression) (koala73#3835)

The 2026-05-19 Pro brief shipped "How nuclear war would impact the
global food system" from Bulletin of Atomic Scientists as CRITICAL
story #6, sitting alongside breaking news about the Iran-Israel war
and the WHO Ebola declaration. The brief promises event-driven
intelligence — a Bulletin analysis essay is not an event.

The existing classifier missed it on all three current signals:
- STRONG #1 (URL section /opinion/): Bulletin URLs have no opinion-
  style path because the WHOLE SITE is commentary, no hard-news
  section to distinguish from.
- STRONG #2 (headline prefix "Opinion:"): hard-news-shaped title.
- CORROBORATING (quote-wrap + columnist framing): description reads
  like a news lede.

The signal IS the publisher. Add STRONG #3: a hand-curated allowlist
of commentary-only publishers, matched by hostname (suffix-anchored to
permit `newsletter.<host>` / `m.<host>` while rejecting typo-domains
like `evilthebulletin.org`).

Initial list (per docs/plans/2026-05-19-001 U1):
- thebulletin.org      — Bulletin of Atomic Scientists
- project-syndicate.org — op-eds from world leaders / academics
- foreignaffairs.com   — CFR's analysis quarterly
- foreignpolicy.com    — Foreign Policy magazine
- warontherocks.com    — defense analysis blog

Maintenance commitment: quarterly review against `droppedOpinion`
telemetry to catch (a) new commentary publishers to add, (b) listed
publishers that launched a hard-news section. Rollback path is
remove-from-Set, never per-URL exceptions (cruft).

Pattern mirrors the existing safePathname/STRONG_URL_SEGMENTS shape;
new safeHostname helper parses URL().hostname so the match is on
the parsed host (not raw .includes() on the link string — closes the
tracking-param injection vector documented in PR koala73#3748).

7 new test cases cover the May 19 regression, the 4 other publishers,
subdomain matching, typo-domain rejection, tracking-param / fragment
spoofing, malformed URLs, and the "allowlisted host PLUS hard-news
content → still opinion" rule.

This is PR-1 of the 4-PR Phase 1 wave from
docs/plans/2026-05-19-001-fix-brief-content-quality-regressions-plan.md.