Skip to content

fix(feeds): remove dead RSS feeds, fix broken URLs#1596

Merged
koala73 merged 1 commit into
mainfrom
fix/dead-rss-feeds
Mar 14, 2026
Merged

fix(feeds): remove dead RSS feeds, fix broken URLs#1596
koala73 merged 1 commit into
mainfrom
fix/dead-rss-feeds

Conversation

@koala73

@koala73 koala73 commented Mar 14, 2026

Copy link
Copy Markdown
Owner

Summary

  • Remove 3 feeds permanently blocked (403): Breaking Defense, My Modern Met, AEI
  • Fix Infobae RSS URL (/feeds/rss/ -> /arc/outboundfeeds/rss/) in both feed config files
  • Fix CSIS RSS URL (/feed -> /rss.xml) in server _feeds.ts
  • Drop active from CelesTrak satellite groups (>2MB payload, always rejected by 2MB guard)

Context

From relay log analysis: 13 RSS feeds returning 403/404, OpenSky proxy down, and CelesTrak active group always exceeding the 2MB limit. Several ghost feeds (old URLs from a stale deployment) will also stop appearing after this deploys.

Test plan

  • TypeScript type checks pass
  • Edge function tests pass (104/104)
  • Markdown lint clean
  • Verify relay logs show no more 403/404 for removed/fixed feeds after deploy

…t group

- Remove 403-blocked feeds: Breaking Defense, My Modern Met, AEI
- Fix Infobae URL: /feeds/rss/ -> /arc/outboundfeeds/rss/ (both files)
- Fix CSIS URL: /feed -> /rss.xml (server _feeds.ts)
- Drop 'active' from CelesTrak SAT_GROUPS (>2MB, always rejected)
@vercel

vercel Bot commented Mar 14, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
worldmonitor Ignored Ignored Mar 14, 2026 5:11pm

Request Review

@koala73
koala73 merged commit 6e8f303 into main Mar 14, 2026
4 checks passed
@koala73
koala73 deleted the fix/dead-rss-feeds branch March 14, 2026 17:14

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0e7efd5ffe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/config/feeds.ts
// Defense & Security (Tier 1)
{ name: 'Defense One', url: rss('https://www.defenseone.com/rss/all/'), type: 'defense' },
{ name: 'Breaking Defense', url: rss('https://breakingdefense.com/feed/'), type: 'defense' },
{ name: 'The War Zone', url: rss('https://www.twz.com/feed'), type: 'defense' },

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep default intel sources aligned with removed feed

Removing Breaking Defense from INTEL_SOURCES here leaves DEFAULT_ENABLED_INTEL still referencing that source, which creates a dangling default entry: dev mode now emits the DEFAULT_ENABLED name "Breaking Defense" not found error and the intel panel initializes with one fewer real default source than configured. Please remove or replace the stale default entry when deleting this feed.

Useful? React with 👍 / 👎.

koala73 added a commit that referenced this pull request Mar 15, 2026
PR #1596 removed the feeds but left the domains in the allowlist.
The relay still accepted proxy requests for these 403-blocked domains
from clients with cached old bundles. Removed:
- breakingdefense.com (403)
- www.arabnews.com (403)
- www.aei.org (403)
- mymodernmet.com (403)

Updated all 3 copies: shared/, scripts/shared/, api/
koala73 added a commit that referenced this pull request Mar 15, 2026
…1626)

PR #1596 removed the feeds but left the domains in the allowlist.
The relay still accepted proxy requests for these 403-blocked domains
from clients with cached old bundles. Removed:
- breakingdefense.com (403)
- www.arabnews.com (403)
- www.aei.org (403)
- mymodernmet.com (403)

Updated all 3 copies: shared/, scripts/shared/, api/
stevenjmiklovic added a commit to stevenjmiklovic/worldmonitor that referenced this pull request Mar 20, 2026
…13)

* fix(seeds): increase forecast TTL from 3600s to 4200s (#1625)

TTL (1h) equaled cron interval (1h), leaving zero buffer for container
cold starts. Health saw EMPTY records=0 during the gap between key
expiry and next seed run. 70min TTL covers the cold start window.

* fix(feeds): remove dead feed domains from RSS allowed-domains list (#1626)

PR #1596 removed the feeds but left the domains in the allowlist.
The relay still accepted proxy requests for these 403-blocked domains
from clients with cached old bundles. Removed:
- breakingdefense.com (403)
- www.arabnews.com (403)
- www.aei.org (403)
- mymodernmet.com (403)

Updated all 3 copies: shared/, scripts/shared/, api/

* refactor(seeds): extract USNI fleet seed from relay with proxy support (#1627)

* refactor(seeds): extract USNI fleet seed from relay with proxy support

USNI's Cloudflare blocks Railway IPs (JA3 fingerprinting), causing
the relay's USNI seed to fail with HTML response instead of JSON.

Changes:
- New seed-usni-fleet.mjs: standalone script with HTTP CONNECT proxy
  support via RESIDENTIAL_PROXY_AUTH (falls back to direct fetch)
- Removed ~240 lines of USNI code from ais-relay.cjs
- TTL bumped from 21600s (6h) to 25200s (7h) for cold start buffer
- Exact same parsing logic (hull types, region coords, vessel extraction,
  strike group detection, battle force summary)
- runSeed pattern with lock, validation, seed-meta, verification

Deploy: Railway cron service, 6h interval, needs RESIDENTIAL_PROXY_AUTH

* fix(seeds): only use RESIDENTIAL_PROXY_AUTH for USNI proxy

* fix(seeds): use OREF_PROXY_AUTH for USNI proxy (the one proxy available)

* fix(supply-chain): remount transit chart on no-op renders (#1630)

* refactor: share edge Turnstile helper (#1628)

* refactor: dedupe signals upstream fetch headers and timeout (#1629)

* fix(seeds): use curl for USNI fetch (JA3 fingerprint bypass) (#1631)

Node.js TLS fingerprint gets blocked by Cloudflare. curl's fingerprint
passes. Strategy: curl direct → curl+proxy → Node.js direct.
Same pattern as relay's orefCurlFetch.

* perf(ci): skip Vercel deploy when only scripts/ change on main (#1633)

Previously, every merge to main triggered a Vercel build even for
scripts-only changes (seed scripts, relay updates). Now checks if
any web-relevant files changed on main too, skipping the build when
only scripts/, docs/, .github/, etc. are modified.

* test(seeds): cover warm-ping origin header (#1632)

* fix(usni-fleet): add Node.js proxy fallback for Cloudflare bypass (#1635)

* fix(usni-fleet): add Node.js HTTP CONNECT proxy fallback, detect Cloudflare HTML

curl is not available in Railway's Railpack v0.18.0 containers. The seed
was failing with ENOENT on curl, then getting Cloudflare-blocked on
Node.js direct.

- Add fetchViaHttpProxy: Node.js HTTP CONNECT tunnel through residential
  proxy (no curl dependency). Uses the same RESIDENTIAL_PROXY_AUTH env.
- Add Cloudflare HTML detection: reject early when response starts with
  <!DOCTYPE instead of passing HTML to JSON.parse.
- Fallback chain: curl direct -> curl+proxy -> Node.js+proxy -> Node.js direct
- Add nixpacks.toml with curl for future Railpack builds

* fix: use ESM import for node:http (require breaks in .mjs)

* test: rewrite transit chart test as structural contract verification (#1636)

Replace fragile source-string extraction + new Function() compilation
with structural pattern checks on the source code. Tests verify:
- render() clears chart before content change
- clearTransitChart() cancels timer, disconnects observer, destroys chart
- MutationObserver setup for DOM readiness detection
- Fallback timer for no-op renders (100-500ms range)
- Both callbacks (observer + timer) clean up each other
- Tab switch and collapse clear chart state
- Mount function guards against missing element/data

Replaces PR #1634's approach which was brittle (method body extraction,
TypeScript cast stripping, sandboxed execution).

* fix: log fetch error cause in seed retry/FATAL handlers (#1638)

* test: rewrite transit chart test as structural contract verification

Replace fragile source-string extraction + new Function() compilation
with structural pattern checks on the source code. Tests verify:
- render() clears chart before content change
- clearTransitChart() cancels timer, disconnects observer, destroys chart
- MutationObserver setup for DOM readiness detection
- Fallback timer for no-op renders (100-500ms range)
- Both callbacks (observer + timer) clean up each other
- Tab switch and collapse clear chart state
- Mount function guards against missing element/data

Replaces PR #1634's approach which was brittle (method body extraction,
TypeScript cast stripping, sandboxed execution).

* fix: log fetch error cause in seed retry and FATAL handlers

Node 20 fetch() throws TypeError('fetch failed') with the real error
hidden in err.cause (DNS, TLS, timeout). The current logging only shows
'fetch failed' which is useless for diagnosis. Now logs err.cause.message
in both withRetry() retries and FATAL catch blocks.

* fix(railway): keep Node 20 + force IPv4 DNS ordering (#1641)

Node 20's fetch() (undici) tries IPv6 first. Railway containers don't
support IPv6 (IPV6_NDISC failures in network trace), causing all seed
services to crash.

Fix: set NODE_OPTIONS=--dns-result-order=ipv4first via nixpacks.toml
so all Railway services prefer IPv4. Keeps Node 20 for import attributes.

* refactor: dedupe contact api json responses (#1639)

* refactor(llm): consolidate provider chain to single source of truth (#1640)

* fix(relay): add LLM fallback chain to ais-relay classify

Replace single Groq-only LLM call with provider fallback chain
(Groq → OpenRouter → Ollama) matching seed-insights.mjs pattern.
If Groq fails or is unavailable, classify falls through to the
next configured provider automatically.

* refactor(llm): consolidate provider chain to single source of truth

- Fix OpenRouter model: openrouter/free → google/gemini-2.5-flash in canonical llm.ts
- Migrate 4 intelligence handlers (classify-event, batch-classify, deduct-situation,
  get-country-intel-brief) from hardcoded Groq-only to callLlm() with full
  ollama → groq → openrouter fallback chain
- Remove duplicate getProviderCredentials from news/v1/_shared.ts, re-export canonical
- Remove orphaned GROQ_API_URL/GROQ_MODEL from intelligence/v1/_shared.ts
- Reorder script provider chains (ais-relay.cjs, seed-insights.mjs) to canonical
  ollama → groq → openrouter order
- Net -161 lines: eliminated duplicated provider logic across 9 files

* fix: eliminate double JSON parse in classify-event, throw on runSeed verification failure

* fix(tests): add llm module alias to country-intel-brief test fixture

* fix: preserve generic LLM_API_* fallback, add retry to seed verification

- Add 'generic' provider to callLlm() chain for LLM_API_URL/LLM_API_KEY/LLM_MODEL
  (preserves existing OpenAI-compatible endpoint contract)
- Change seed verification to warn-only with 1 retry instead of fatal throw
  (write already succeeded, transient read failure shouldn't fail the job)
- Update docs to reflect new provider fallback chain

* feat(advisories): gold standard migration for security advisories (#1637)

* feat(advisories): gold standard migration for security advisories

Move security advisories from client-side RSS fetching (24 feeds per
page load) to Railway cron seed with Redis-read-only Vercel handler.

- Add seed script fetching via relay RSS proxy with domain allowlist
- Add ListSecurityAdvisories proto, handler, and RPC cache tier
- Add bootstrap hydration key for instant page load
- Rewrite client service: bootstrap -> RPC fallback, no browser RSS
- Wire health.js, seed-health.js, and dataSize tracking

* fix(advisories): empty RPC returns ok:true, use full country map

P1 fixes from Codex review:
- Return ok:true for empty-but-successful RPC responses so the panel
  clears to empty instead of stuck loading on cold environments
- Replace 50-entry hardcoded country map with 251-entry shared config
  generated from the project GeoJSON + aliases, matching coverage of
  the old client-side nameToCountryCode matcher

* fix(advisories): add Cote d'Ivoire and other missing country aliases

Adds 14 missing aliases including "cote d ivoire" (US State Dept
title format), common article-prefixed names (the Bahamas, the
Gambia), and alternative official names (Czechia, Eswatini, Cabo
Verde, Timor-Leste).

* fix(proto): inject @ts-nocheck via Makefile generate target

buf generate does not emit @ts-nocheck, but tsc strict mode rejects
the generated code. Adding a post-generation sed step in the Makefile
ensures both CI proto-freshness (make generate + diff) and CI
typecheck (tsc --noEmit) pass consistently.

* fix(predictions): remove Kalshi API key, enforce Redis-only on Vercel (#1642)

Kalshi public market data endpoints require no authentication. Remove
the KALSHI_API_KEY gate that was disabling Kalshi entirely when the
env var was missing, and drop the Authorization header.

Rewrite the Vercel RPC handler to read from Railway-seeded Redis only
(gold standard), removing the fallback that fetched directly from
Gamma/Kalshi APIs on Vercel edge. Handler goes from 330 to 85 lines.

Double all prediction timing values to reduce Railway cron cost:
- Redis TTL: 15min -> 30min
- Health maxStaleMin: 15min -> 30min
- Client hydration freshness: 20min -> 40min
- Railway cron: 10min -> 20min (requires dashboard update)

* fix: add fetch error cause logging to all remaining seed scripts (#1643)

* fix(seeds): improve resilience and fix dead APIs across seed scripts (#1644)

* fix(seeds): improve resilience and fix dead APIs across seed scripts

- Fix wrong domain in seed-service-statuses (worldmonitor.app to api.worldmonitor.app)
- Fix Kalshi API domain migration (trading-api.kalshi.com to api.elections.kalshi.com)
- Replace dead trending APIs (gitterapp.com, herokuapp.com) with OSSInsight + GitHub Search
- Fix case-sensitive HTML detection in seed-usni-fleet (lowercase doctype not matched)
- Add Promise.allSettled rejection logging across 8 seed scripts
- Wrap fetch loops in try-catch (seed-supply-chain-trade, seed-economy) so a single
  network error no longer kills the entire function
- Update list-trending-repos.ts RPC handler to match seed changes

* fix(seeds): correct OSSInsight response parsing and period-aware GitHub Search fallback

- OSSInsight returns {data: {rows: [...]}} not {data: [...]}, fix both seed and handler
- GitHub Search fallback now respects period parameter (daily=1d, weekly=7d, monthly=30d)

* fix(seeds): correct OSSInsight period values (past_week/past_month, not past_7_days/past_28_days)

* Add Greek news channels & feed (#1602)

* Add Greek news channels

* Add ERT and SKAI hlsUrl to LiveNewsPanel.ts

---------

Co-authored-by: Elie Habib <[email protected]>

* feat(scoring): consume live advisory data from Redis with hardcoded fallback (#1620)

Read live travel advisory levels from intelligence:advisories:v1 Redis
key (populated by seed-security-advisories.mjs). Fall back to the
original hardcoded ADVISORY_LEVELS_FALLBACK map when Redis data is
unavailable.

Closes #1359.

* fix(sentry): tighten noise filters for 6 unresolved issues (#1645)

Fix 2 regexes that let errors slip through:
- AbortError filter now handles value without type prefix
- Qt() filter now matches Qt(...) with arguments

Add 4 new filters for browser-specific noise:
- ES module syntax errors (Pale Moon, QQ Browser)
- ucConfig (UC Browser V8 format)
- getShaderPrecisionFormat (WebGL context loss on iOS)

* refactor: dedupe register-interest json responses (#1647)

* fix(usni): move USNI fleet seed back to AIS relay (fixed IP for Froxy proxy) (#1648)

The standalone seed-usni-fleet.mjs cannot reach USNI because:
1. USNI Cloudflare blocks Node.js TLS fingerprint (JA3)
2. curl is not installed on Railway cron containers
3. Froxy residential proxy is IP-whitelisted to the relay fixed IP

Move the USNI seed loop back into ais-relay.cjs where it has access to
curl + the whitelisted proxy. Uses orefCurlFetch for the fetch, same
pattern as the OREF alerts loop. Writes to the same Redis keys
(usni-fleet:sebuf:v1, stale:v1, seed-meta:military:usni-fleet).

6h seed interval, 7h TTL, 7d stale TTL (unchanged from standalone).

* fix(seeds): allow conflict-intel seed to succeed without ACLED keys (#1651)

Validation now accepts empty ACLED events array when humanitarian or
pizzint data was fetched. Previously the seed wrote extra keys
(humanitarian, pizzint) but skipped the canonical key because
validateFn required non-empty events.

* fix(data): restore bootstrap and cache test coverage (#1649)

* fix(data): restore bootstrap and cache test coverage

* fix: resolve linting and test failures

- Remove dead writeSeedMeta/estimateRecordCount functions from redis.ts
  (intentionally removed from cachedFetchJson; seed-meta now written
  only by explicit seed flows, not generic cache reads)
- Fix globe dayNight test to match actual code (forces dayNight: false
  + hideLayerToggle, not catalog-based exclusion)
- Fix country-geometry test mock URL from CDN to /data/countries.geojson
  (source changed to use local bundled file)

* fix(lint): remove duplicate llm-health key in redis-caching test

Duplicate object key '../../../_shared/llm-health' caused the stub
to be overwritten by the real module. Removed the second entry so
the test correctly uses the stub.

* feat(forecast): add structured scenario pipeline and trace export (#1646)

* feat(forecast): add AI Forecasts prediction module (Pro-tier)

MiroFish-inspired prediction engine that generates structured forecasts
across 6 domains (conflict, market, supply chain, political, military,
infrastructure) using existing WorldMonitor data streams.

- Proto definitions for ForecastService with GetForecasts RPC
- Dedicated seed script (seed-forecasts.mjs) with 6 domain detectors,
  cross-domain cascade resolver, prediction market calibration, and
  trend detection via prior snapshot comparison
- Premium-gated RPC handler (PREMIUM_RPC_PATHS enforcement)
- Lazy-loaded ForecastPanel with domain filters, probability bars,
  trend arrows, signal evidence, and cascade links
- Health monitoring integration (seed-meta freshness tracking)
- Refresh scheduler with API key guard

* test(forecast): add 47 unit tests for forecast detectors and utilities

Covers forecastId, normalize, resolveCascades, calibrateWithMarkets,
computeTrends, and smoke tests for all 6 domain detectors. Exports
testable functions from seed script with direct-run guard.

* fix(forecast): domain mismatch 'infra' vs 'infrastructure', add panel category

- Seed script used 'infra' but ForecastPanel filtered on 'infrastructure',
  causing Infra tab to show zero results
- Added 'forecast' to intelligence category in PANEL_CATEGORY_MAP

* fix(forecast): move CSS to one-time injection, improve type safety

- P2: Move style block from setContent to one-time document.head injection
  to prevent CSS accumulation on repeated renders
- P3: Replace +toFixed(3) with Math.round for readability in seed script
- P3: Use Forecast type instead of any[] in RPC handler filter

* fix(forecast): handle sebuf proto data shapes from Redis

Detectors now normalize CII scores from server-side proto format
(combinedScore, TREND_DIRECTION_RISING, region) to uniform shape.
Outage severity handles proto enum format (SEVERITY_LEVEL_HIGH).
Added confidence floor of 0.3 for single-source predictions.

Verified against live Redis: 2 predictions generated (Iran infra
shutdown, IL political instability).

* feat(forecast): unlock AI Forecasts on web, lock desktop only (trial)

- Remove forecast RPC from PREMIUM_RPC_PATHS (web access is free)
- Panel locked on desktop only (same as oref-sirens/telegram-intel)
- Remove API key guards from data-loader and refresh scheduler
- Web users get full access during trial period

* chore: regenerate proto types with make generate

Re-ran make generate after rebasing on main. Plugin v0.7.0 dropped
@ts-nocheck from output, added it back to all 50 generated files.
Fixed 4 type errors from proto codegen changes:
- MarketSource enum -> string union type
- TemporalAnomalyProto -> TemporalAnomaly rename
- webcam lastUpdated number -> string

* chore: add proto freshness check to pre-push hook

Runs make generate before push and compares checksums of generated files.
If proto types are stale, blocks push with instructions to regenerate.
Skips gracefully if buf CLI is not installed.

* fix(forecast): use chokepoints v4 key, include ciiContribution in unrest

- P1: Switch chokepoints input from stale v2 to active v4 Redis key,
  matching bootstrap.js and cache-keys.ts
- P2: Add ciiContribution to unrest component fallback chain in
  normalizeCiiEntry so political detector reads the correct sebuf field

* feat(forecast): Phase 2 LLM scenario enrichment + confidence model

MiroFish-inspired enhancements:
- LLM scenario narratives via Groq/OpenRouter (narrative-only, no numeric
  adjustment). Evidence-grounded prompts with mandatory signal citation
  and few-shot examples from MiroFish's SECTION_SYSTEM_PROMPT_TEMPLATE.
- Top-4 predictions batched into single LLM call for cost efficiency.
- News context from newsInsights attached to all predictions for LLM
  prompt grounding (NOT in signals, cannot affect confidence).
- Deterministic confidence model: source diversity via SIGNAL_TO_SOURCE
  mapping (deduplicates cii+cii_delta, theater+indicators) + calibration
  agreement from prediction market drift. Floor 0.2, ceiling 1.0.
- Output validation: rejects scenarios without signal references.
- Truncated JSON repair for small model output.
- Structured JSON logging for LLM calls.
- Redis cache for LLM scenarios (1h TTL).
- 23 new tests (70 total), all passing.
- Live-tested: OpenRouter gemini-2.5-flash produces evidence-grounded
  scenario narratives from real WorldMonitor data.

* feat(forecast): Phase 3 multi-perspective scenarios, projections, data-driven cascades

MiroFish-inspired enhancements:
- Multi-perspective LLM analysis: top-2 predictions get strategic,
  regional, and contrarian viewpoints via combined LLM call
- Probability projections: domain-specific decay curves (h24/d7/d30)
  anchored to timeHorizon so probability equals projections[timeHorizon]
- Data-driven cascade rules: moved from hardcoded array to JSON config
  (scripts/data/cascade-rules.json) with schema validation, named
  predicate evaluators, unknown key rejection, and fallback to defaults
- 4 new cascade paths: infrastructure->supply_chain, infrastructure->market
  (both requiresSeverity:total), conflict->political, political->market
- Proto: added Perspectives and Projections messages to Forecast
- ForecastPanel: renders projections row and conditional perspectives toggle
- 89 tests (19 new), all passing
- Live-tested: OpenRouter produces perspectives from real data

* feat(forecast): Phase 4 data utilization + entity graph

Fixes data gaps that prevented 4 of 6 detectors from firing:
- Input normalizers: chokepoint v4 shape + GPS hexes-to-zones mapping
- Chokepoint warm-ping (production-only, requires WM_API_BASE_URL)
- Lowered CII conflict threshold from 70 to 60, gated on level=high|critical

4 new standalone detectors:
- UCDP conflict zones (10+ events per country)
- Cyber threat concentration (5+ threats per country)
- GPS jamming in maritime shipping zones (5 regions)
- Prediction markets as signals (60-90% probability markets)

Entity-relationship graph (file-based, 38 nodes):
- Countries, theaters, commodities, chokepoints, alliances
- Alias table resolves both ISO codes and display names
- Graph cascade discovery links predictions across entities

Result: 51 predictions (up from 1-2), spanning conflict, infrastructure,
and supply chain domains. 112 tests, all passing.

* fix(forecast): redis cache format, signal source mapping, type safety

Fresh-eyes audit fixes:
- BUG: redisSet used wrong Upstash API format (POST body with {value,ex}
  instead of command array ['SET',key,value,'EX',ttl]). LLM cache writes
  were silently failing, causing fresh LLM calls every run.
- BUG: prediction_market signal type missing from SIGNAL_TO_SOURCE,
  inflating confidence for market-derived predictions.
- CLEANUP: Remove unnecessary (f as any) casts in ForecastPanel since
  generated Forecast type already has projections/perspectives fields.
- CLEANUP: Bump health maxStaleMin from 60 to 90 to avoid false STALE
  alerts when LLM calls add latency to seed runs.

* feat(forecast): headline-entity matching with news corroboration signals

Uses entity graph aliases to match headlines to predictions by
country/theater (excludes commodity/infrastructure nodes to prevent
false positives). Predictions with matching headlines get a
news_corroboration signal visible in the panel.

Also fixes buildUserPrompt to merge unique headlines from ALL
predictions in the LLM batch (was only reading preds[0].newsContext).

Live-tested: 13 of 51 predictions now have corroborating headlines
(Iran, Israel, Syria, Ukraine, etc). 116 tests, all passing.

* feat(forecast): add country-codes.json for headline-entity matching

56 countries with ISO codes, full names, and scoring keywords (extracted
from src/config/countries.ts + UCDP-relevant additions). Used by
attachNewsContext for richer headline matching via getSearchTermsForRegion
which combines country-codes + entity graph + keyword aliases.

14/57 predictions now have news corroboration (limited by headline
coverage, not matching quality: only 8 headlines currently available).

* feat(forecast): read 300 headlines from news digest instead of 8

Read news:digest:v1:full:en (300 headlines across 16 categories) instead
of just news:insights:v1 topStories (8 headlines). Fallback to topStories
if digest is unavailable.

Result: news corroboration jumped from 25% to 64% (38/59 predictions).

* fix(forecast): handle parenthetical country names in headline matching

Strip suffixes like '(Zaire)', '(Burma)', '(Soviet Union)' from UCDP
region names before matching against country-codes.json. Also use
includes() for reverse name lookup to catch partial matches.

Corroboration: 64% -> 69% (41/59). Remaining 18 unmatched are countries
with no current English-language news coverage.

* fix(forecast): cache validated LLM output, add digest test, log cache errors

Fresh-eyes audit fixes:
- Combined LLM cache now stores only validated items (was caching raw
  unvalidated output, serving potentially invalid scenarios on cache hit)
- redisSet logs warnings on failure (was silently swallowing all errors)
- Added digest-based test for attachNewsContext (primary path was untested)
- Fixed test arity: attachNewsContext(preds, news, digest) with 3 params

* fix(forecast): remove dead confidenceFromSources, reduce warm-ping timeout

- P2: Remove confidenceFromSources (dead code, computeConfidence overwrites
  all initial confidence values). Inline the formula in original detectors.
- P3: Reduce warm-ping timeout from 30s to 15s (non-critical step)
- P3: Add trial status comment on forecast panel config

* fix(forecast): resolve ISO codes to country names, fix market detector, safe pre-push

P1 fixes from code review:
- CII ISO codes (IL, IR) now resolved to full country names (Israel, Iran)
  via country-codes.json. Prevents substring false positives (IL matching
  Chile) in event correlation. Uses word-boundary regex for matching.
- Market detector CII-to-theater mapping now uses entity graph traversal
  instead of broken theater-name substring matching. Iran correctly maps
  to Middle East theater via graph links.
- Pre-push hook no longer runs destructive git checkout on proto freshness
  failure. Reports mismatch and exits without modifying worktree.

* feat(forecast): add structured scenario pipeline and trace export

* fix(forecast): hydrate bootstrap and trim generated drift

* fix(forecast): keep required supply-chain contract updates

* fix(ci): add forecasts to cache-keys registry and regenerate proto

Add forecasts entry to BOOTSTRAP_CACHE_KEYS and BOOTSTRAP_TIERS in
cache-keys.ts to match api/bootstrap.js. Regenerate SupplyChain proto
to fix duplicate TransitDayCount and add riskSummary/riskReportAction.

* fix: restore desktop bootstrap timeouts and SW NetworkOnly (#1653)

- Bootstrap: keep 1.2s/1.8s for web, restore 5s/8s for desktop
  (sidecar IPC + token resolution needs the headroom)
- SW: revert navigation handler to NetworkOnly to prevent stale
  HTML caching that causes 404 storms on deploy

* feat(supply-chain): restructure panel, add corridor disruption to shipping tab (#1652)

* feat(supply-chain): restructure panel for scannable metrics and richer shipping tab

Chokepoints tab: replace text blob description with structured metric rows
showing warnings, vessels, WoW change, disruption %, and risk level. Move
routing advisory into styled callout box on expand. Remove duplicated
riskSummary and warning counts from server description field.

Shipping Rates tab: add corridor disruption snapshot table showing per-corridor
vessel counts, WoW traffic changes, disruption %, and risk levels. Tab now
renders useful data even without FRED API key.

Add CSS for expanded card state (previously no-op), metric rows, disruption
color coding, routing advisory callout, and disruption table.

* test(supply-chain): add restructure tests, update description assertion

Add 7 structural tests for panel restructure:
- activeHasData accepts chokepointData without FRED
- renderShipping delegates to renderDisruptionSnapshot
- null/empty loading states
- data-cp-id and data-chart-cp preservation
- conditional description rendering
- server description no longer duplicates warning counts

Update existing test: description now asserts counts are NOT in text
(they moved to structured fields).

* fix(seeds): lazy-load @aws-sdk/client-s3 in R2 storage helper (#1654)

The top-level import crashes seed-forecasts on Railway when the
package isn't installed. Dynamic import defers the load to when
S3 mode is actually used, allowing the seed to run without the
SDK when R2 is not configured.

* test: cover runtime env guardrails (#1650)

* fix(data): restore bootstrap and cache test coverage

* test: cover runtime env guardrails

* fix(test): align security header tests with current vercel.json

Update catch-all source pattern, geolocation policy value, and
picture-in-picture origins to match current production config.

* debug(seeds): add R2 trace logging to seed-forecasts (#1655)

Adds [Trace] log lines before/after R2 export and [R2] config
resolution debug output to diagnose missing trace artifacts.

* feat(search): add all missing panels to CMD+K command palette (#1656)

24 panels were not discoverable via CMD+K. Added commands for:
map, webcams, AI insights/posture/forecasts, correlation panels
(force posture, escalation, economic warfare, disaster cascade),
fires, gulf economies, giving, UCDP events, displacement, climate,
population exposure, security advisories, Israel sirens, telegram
intel, airline intel, tech readiness, world clock, and layoffs.

Coverage: 55/55 full-variant panels now searchable.

* fix(health): close corridorrisk health monitoring gap (#1658)

The corridorrisk raw key (2h TTL) expires between hourly seed cycles,
causing health to report EMPTY even though data flows correctly through
transit-summaries:v1.

- Increase CORRIDOR_RISK_TTL from 2h to 4h (3 retries before expiry)
- Add corridorrisk to ON_DEMAND_KEYS (WARN instead of CRIT when empty)

* refactor: dedupe error mapper json responses (#1657)

Extract jsonMessageResponse helper to replace 4 identical
Response construction patterns in error-mapper.ts.

* fix: use data-font attribute instead of inline style for font toggle (#1552)

* fix: use data-font attribute instead of inline style for font toggle

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

* fix: use inline style for font-body-base to win cascade over variant themes

Copilot review flagged that [data-font="system"] in @layer base loses to
unlayered variant themes (e.g. happy-theme sets --font-body: Nunito).
Switch back to inline style on --font-body-base so user font preference
always wins the cascade, while keeping the RTL/CJK composition via
var(--font-body-base) fallback.

---------

Co-authored-by: Copilot Autofix powered by AI <[email protected]>
Co-authored-by: Elie Habib <[email protected]>

* feat(predictions): redesign panel with gradient bars, source accents, conviction labels (#1661)

Visual redesign of prediction market cards:
- Source badge (KALSHI/POLYMARKET) moved to card header with matching
  left accent stripe (blue for Kalshi, purple for Polymarket)
- Gradient bars replace flat colors, with inner glow on strong signals
- Conviction labels (Lean Yes / Lean No / Toss-up) based on threshold
- Rounded bar corners (6px), thicker bars (28px), better spacing
- Hover state on cards for interactivity feel

* fix(predictions): prepend event title to short Kalshi outcome labels (#1659)

Kalshi multi-outcome events return market titles like "Before 2035",
"Rhode Island", "Johnny Depp" which are meaningless without the parent
event context. Now combines event title with market title when the
market title is short and doesn't contain a question mark.

Before: "Before 2035" (KALSHI)
After:  "Will AGI be achieved?: Before 2035" (KALSHI)

* fix(forecast): restore missing domains and split cyber (#1660)

* fix(pwa): skip reload on first service worker registration (#1662)

skipWaiting + clientsClaim fires controllerchange when the SW claims
the page for the first time on a new session. The controllerchange
listener was reloading unconditionally, causing a useless full page
reload (double load of all assets and API calls) on every first visit.

Now checks navigator.serviceWorker.controller at page load. If null
(no prior SW), the controllerchange from initial claim is ignored.
Deploy updates (new SW replacing existing) still trigger reload.

* fix(health): prevent CF/CDN caching of health endpoint (#1665)

CF was caching health responses despite no-cache, no-store because
the Cache Rule marks /api/ as "eligible for cache" and CF interprets
no-cache as "revalidate" not "don't cache". Changed to:
- Cache-Control: private, no-store, max-age=0 (tells CF not cacheable)
- CDN-Cache-Control: no-store (CF-specific override)

* fix(health): add explicit seed-meta for chokepoints, move stale RPC keys to ON_DEMAND (#1667)

- Chokepoints handler now writes seed-meta explicitly on fresh fetch
  (not on cache hits, per PR #1649 design)
- Move riskScores and serviceStatuses to ON_DEMAND_KEYS (WARN not
  CRIT when stale; these are RPC-populated, not seeded)
- cableHealth already in ON_DEMAND_KEYS

* feat(trade): add US Treasury customs revenue to Trade Policy panel (#1663)

* feat(trade): add US Treasury customs revenue to Trade Policy panel

US customs duties revenue spiked 4-5x under Trump tariffs (from
$7B/month to $27-31B/month) but the WTO tariff data only goes to
2024. Adds Treasury MTS data showing monthly customs revenue.

- Add GetCustomsRevenue RPC (proto, handler, cache tier)
- Add Treasury fetch to seed-supply-chain-trade.mjs (free API, no key)
- Add Revenue tab to TradePolicyPanel with FYTD YoY comparison
- Fix WTO gate: per-tab gating so Revenue works without WTO key
- Wire bootstrap hydration, health, seed-health tracking

* test(trade): add customs revenue feature tests

22 structural tests covering:
- Handler: raw key mode, empty-cache behavior, correct Redis key
- Seed: Treasury API URL, classification filter, timeout, row
  validation, amount conversion, sort order, seed-meta naming
- Panel: WTO gate fix (per-tab not panel-wide), revenue tab
  defaults when WTO key missing, dynamic FYTD comparison
- Client: no WTO feature gate, bootstrap hydration, type exports

* fix(trade): align FYTD comparison by fiscal month count

Prior FY comparison was filtering by calendar month, which compared
5 months of FY2026 (Oct-Feb) against only 2 months of FY2025
(Jan-Feb), inflating the YoY percentage. Now takes the first N
months of the prior FY matching the current FY month count.

* fix(trade): register treasury_revenue DataSourceId and localize revenue tab

- Add treasury_revenue to DataSourceId union type so freshness
  tracking actually works (was silently ignored)
- Register in data-freshness.ts source config + gap messages
- Add i18n keys: revenue tab label, empty state, unavailable banner
- Update infoTooltip to include Revenue tab description

* fix(trade): complete revenue tab localization

Use t() for all remaining hardcoded strings: footer source labels,
FYTD summary headline, prior-year comparison, and table column
headers. Wire the fytdLabel/vsPriorFy keys that were added but
not used.

* fix(test): update revenue source assertion for localized string

* feat(supply-chain): add SCFI, CCFI, and BDI freight indices (#1666)

* feat(supply-chain): add SCFI, CCFI, and BDI freight indices to shipping tab

Transform the Shipping Rates tab from 2 lagging monthly FRED indices into
a real-time freight cost dashboard with container and bulk shipping rates.

Seed script: add fetchSCFI/fetchCCFI (SSE JSON API) and fetchBDI (HandyBulk
HTML scrape) with inline history accumulation using source observation dates.
Handler: make cache-only (seed is sole aggregator, no FRED fallback on miss).
Panel: group indices into Container Rates, Bulk Shipping, Economic Indicators.
Tests: 26 functional tests with fixture data for parsers, history, and handler.

* fix(supply-chain): use raw Redis read and correct SCFI composite unit

- Handler: switch from cachedFetchJson (env-prefixed) to getCachedJson(key, true)
  so preview deployments read the unprefixed seed key correctly
- Seed: SCFI composite is a dimensionless index, not USD/TEU (route-level unit)
- Tests: update assertions to match both fixes

* fix(forecast): bump TTL from 3600s to 4800s to cover cron gap (#1668)

Hourly cron + 3600s TTL = data expires right as the next seed starts,
causing a ~30s EMPTY window. Bumped to 4800s (80min) so old data
persists while the new seed runs.

* fix(health): remove stale SEED_META for RPC-populated keys, bump to v2.6.5 (#1669)

riskScores and serviceStatuses have data but permanently stale seed-meta
(no longer written by cachedFetchJson after PR #1649). ON_DEMAND_KEYS
only affects EMPTY status, not STALE_SEED. Removing their SEED_META
entries so health doesn't check freshness for keys that can't update it.

Also bumps version to 2.6.5.

* fix(panels): refresh CMD+K active panels on open, add missing info tooltips (#1670)

- Search: refresh activePanelIds in updateSearchIndex() so lazy-loaded
  panels (forecast, oref-sirens, telegram-intel, etc.) appear in CMD+K
- Add ? info tooltips to AI Forecasts, Force Posture, and Disaster Cascade

* fix(health): remove cableHealth SEED_META (RPC-populated, stale after PR #1649) (#1671)

* fix(seed): bump BDI body size limit to 1MB, add gzip encoding (#1672)

HandyBulk page is ~662KB, exceeding the 500KB guard. Also adds
Accept-Encoding header to reduce transfer size.

* fix(forecast): store full traces and tighten matching (#1673)

* fix(forecast): restore missing domains and split cyber

* fix(forecast): store full traces and tighten matching

* fix(aviation): stop Vercel from calling AviationStack directly (#1674)

* fix(aviation): stop Vercel from calling AviationStack directly

- get-airport-ops-summary: read from relay seed cache (aviation:delays:intl:v3)
  instead of calling fetchAviationStackDelays() on every cache miss
- list-airport-flights + get-flight-status: proxy through Railway relay
  /aviationstack endpoint instead of calling AviationStack from Vercel edge
- Add /aviationstack proxy endpoint to ais-relay with 2min in-memory cache

Vercel should NEVER call external paid APIs directly. Railway relay is
the sole egress point for AviationStack (gold standard).

* fix(config): update aviationStack feature to require WS_RELAY_URL

Aviation handlers now proxy through Railway relay instead of calling
AviationStack directly. Update runtime-config to reflect the actual
dependency.

* fix(supply-chain): force rebuild to deploy freight indices UI (#1680)

Vercel build cache served stale bundle missing PR #1666 frontend changes
(SCFI, CCFI, BDI grouping in SupplyChainPanel). Trivial whitespace change
to invalidate Vite cache and force rebuild.

* fix(trade): align flows cache key with seed (#1677)

* fix(trade): align flows cache key with seed (US vs World, not China)

The seed writes trade:flows:v1:840:000:10 (US vs World) but the
data-loader requested trade:flows:v1:840:156:10 (US vs China),
causing perpetual cache misses and a hidden Flows tab.

* feat(seed): add bilateral trade flow pairs (US-China, US-Germany, etc.)

Seed now writes both reporter-vs-World AND key bilateral pairs
so switching between global and bilateral views hits warm cache.

* feat(seed): add World-China and World-US bilateral flow pairs

* fix(trade): revert flows to US-China (840/156), seed now covers this key

The bilateral seed entries now write trade:flows:v1:840:156:10,
so the original US-China request hits cache. Keeps the panel
showing bilateral data consistent with the tariffs partner.

* fix(forecast): log and honor full trace count (#1683)

* fix(bootstrap): move shippingRates to FAST_KEYS for shorter CDN TTL (#1682)

shippingRates was in SLOW_KEYS (2h CDN cache). After seed populates new
freight indices (SCFI, CCFI, BDI), CF served stale bootstrap with only
2 FRED indices for up to 2.5h. Moving to FAST_KEYS (10min CDN TTL)
ensures new data appears promptly.

Also update BOOTSTRAP_TIERS in cache-keys.ts to match.

* fix(handlers): pure Redis reads for 31 seed-backed Vercel handlers (#1684)

* fix(handlers): convert 31 Vercel RPC handlers to pure Redis reads

Enforce gold standard: Vercel reads Redis ONLY, Railway makes ALL external
API calls. Replace cachedFetchJson (which fetches from external APIs on
cache miss) with getCachedJson (pure Redis read from seed data).

Converted domains: economic (8), market (7), trade (4), research (3),
infrastructure (2), aviation (1), climate (1), cyber (1), natural (1),
seismology (1), unrest (1), wildfire (1).

Net -3,678 lines of external API call code removed from Vercel edge.

* test: update structural tests for seed-only handlers

Climate, wildfire, and unrest handlers no longer have TTL constants
or ACLED imports since they are now pure cache readers. Update tests
to verify getCachedJson usage instead of removed patterns.

* fix(handlers): revert service-statuses, fix fred-batch key prefixing

P1: Revert list-service-statuses.ts to original cachedFetchJson form.
Both seeders (relay + cron) warm-ping the RPC which is the only writer.
Making it read-only would break the data pipeline. This handler stays
as-is until a proper seed script is built.

P2: Replace getCachedJsonBatch with individual getCachedJson(key, true)
calls in get-fred-series-batch.ts. getCachedJsonBatch uses prefixKey()
which adds env prefix, but seed data is written without prefix. Using
getCachedJson with raw=true bypasses the prefix.

* fix(trade): prevent circuit breaker from caching empty results (#1685)

All 5 trade fetch functions now pass shouldCache to skip caching
when the RPC returns empty data. Previously, empty/unavailable
responses were cached for 30 min in IndexedDB, causing Flows and
Barriers tabs to stay hidden across page reloads even after the
seed populated Redis with real data.

* fix(supply-chain): move Economic Indicators to own tab (#1686)

With 7 new freight index cards (SCFI, CCFI, BDI, BCI, BPI, BSI, BHSI)
on the Shipping Rates tab, the FRED Economic Indicators were pushed too
far down. Split into a 4th tab for better visibility.

* fix(forecast): filter low-signal panel forecasts (#1687)

* refactor: dedupe map URL param parsing helpers (#1675)

Extract parseEnumParam and parseClampedFloatParam helpers in
urlState.ts to replace 4 inline parsing blocks for view, zoom,
lat, lon, and timeRange.

* feat(trade): rename Revenue tab to US Revenue, add monthly bar chart (#1689)

- Renamed "Revenue" tab to "US Revenue" for clarity
- Added 12-month bar chart above the table showing customs revenue trend
- Bars highlighted in red when revenue exceeds 1.5x prior year average
- Makes the tariff impact (308% YoY increase) visually obvious

* blog: add post on trade routes, chokepoints, and freight costs (#1692)

Covers the new Supply Chain panel features: chokepoint monitoring (8
corridors), freight indices (SCFI, CCFI, BDI), trade policy (WTO data),
and critical minerals. Opens with the current Hormuz crisis (94.4%
traffic drop) as a real-world hook.

* docs: update Mintlify docs for recent features (#1693)

- finance-data.mdx: Add US Treasury customs revenue to Trade Policy
  section (Revenue tab, free API, FYTD comparison, desktop support)
- features.mdx: Expand CMD+K section from 5 categories to full 55
  panel coverage with organized subcategories
- data-sources.mdx: Update advisory section for gold standard
  migration (Railway seed, 265-entry country map, relay proxy)
- architecture.mdx: Add 12 missing seed scripts to Railway pipeline
  table (forecasts, conflict-intel, economy, supply-chain-trade,
  security-advisories, usni-fleet, gdelt-intel, research, etc.)
- changelog.mdx + CHANGELOG.md: Add unreleased entries for customs
  revenue, advisory migration, CMD+K coverage, R2 traces, and 6 fixes

* fix(ui): render heatmap with neutral 0% when sector change is null (#1691)

Sectors with null change (market closed / bootstrap race) now render
as 0% (neutral gray) instead of showing "temporarily unavailable".
Error only shows when the data array is truly empty.

* fix(gateway): skip CDN cache for upstreamUnavailable responses (#1690)

When an RPC handler returns { upstreamUnavailable: true }, the gateway
now sets Cache-Control: no-store instead of the normal CDN cache tier.
This prevents CF from caching empty/error responses for hours (e.g.
trade flows returning 0 results got cached for 4h with static tier).

Successful responses continue to use their configured cache tier.

* fix(blog): correct heroImage path and remove duplicate inline image (#1694)

heroImage used /images/blog/ but blog base is /blog/, so the correct
path is /blog/images/blog/. Also removed duplicate inline ![...] image
reference since the hero image already shows it.

* perf(news): skip client-side AI reclassification for digest items (#1695)

* perf(news): skip client-side AI reclassification for digest items

The server digest already runs enrichWithAiCache() which checks the
same Redis keys that classifyEvent writes to. Re-firing classifyEvent
from every client wastes edge requests even on cache hits.

Removes the aiCandidates/classifyWithAI loop from the digest branch.
Per-feed fallback path is unchanged (it never had classification).
Saves up to ~48 classify-event edge requests per page load.

* test(news): add regression test for digest branch no-reclassification guard

6 structural assertions ensuring digest-backed items never trigger
client-side classifyWithAI calls, and that the imports are removed.
Prevents accidental reintroduction of redundant edge requests.

* fix(forecast): seed military surge signals (#1696)

* fix(forecast): filter low-signal panel forecasts

* fix(forecast): seed military surge signals

* perf: gate non-critical startup loads and refresh intervals by viewport (#1697)

* perf: gate non-critical startup loads and refresh intervals by viewport

Scheduler (App.ts):
- firms: gate by satellite-fires panel viewport or natural layer
- intelligence: gate by cii/strategic-risk/strategic-posture viewport
  or country brief open
- temporalBaseline: gate by same intelligence consumers
- correlation-engine: gate by correlation panel viewport
- strategic-posture: viewport check instead of panel-exists
- strategic-risk: viewport check instead of panel-exists

Startup (data-loader.ts):
- giving: only load when panel near viewport
- firms: only load when satellite-fires panel near viewport or
  natural layer active
- intelligence: only load when cii/strategic panels near viewport
- iranAttacks: only load when intelligence consumers near viewport

Adds 3 centralized helpers: shouldRefreshIntelligence(),
shouldRefreshFirms(), shouldRefreshCorrelation().

* fix(perf): add iranAttacks map layer guard, remove redundant natural check

P2: iranAttacks startup now also loads when mapLayers.iranAttacks is
enabled, preventing missing map pins on first load when the layer is
on but strategic panels are off-screen.

P3: Remove mapLayers.natural from shouldRefreshFirms() since natural
events have their own independent scheduler entry.

* fix(seeds): extend existing cache TTL on validation failure (#1705)

When a seed fetches data but validation rejects it (e.g. FIRMS API
returns 0 fires due to timeout), extend the existing key's TTL
instead of letting it expire. Old data survives until the next
successful fetch. Applies to all seeds using runSeed().

* refactor: dedupe upstash json reads across edge endpoints (#1708)

* fix: unblock geolocation and fix stale CSP hash (#1709)

* fix: unblock geolocation and fix stale CSP hash for SW nuke script

Permissions-Policy had geolocation=() which blocked navigator.geolocation
used by user-location.ts. Changed to geolocation=(self).

CSP script-src had a stale SHA-256 hash (903UI9my...) that didn't match the
current SW nuke script content. The script was silently blocked in production,
preventing recovery from stale service workers after deploys. Replaced with
the correct hash (4Z2xtr1B...) in both vercel.json and index.html meta tag.

* test: update permissions-policy test for geolocation=(self)

Move geolocation from "disabled" list to "delegated" assertions since
it now allows self-origin access for user-location.ts.

* fix(forecast): bundle military surge inputs (#1706)

* chore: clear baseline lint debt (173 warnings → 49) (#1712)

Mechanical fixes across 13 files:
- isNaN() → Number.isNaN() (all values already numeric from parseFloat/parseInt)
- let → const where never reassigned
- Math.pow() → ** operator
- Unnecessary continue in for loop
- Useless string escape in test description
- Missing parseInt radix parameter
- Remove unused private class member (write-only counter)
- Prefix unused function parameter with _

Config: suppress noImportantStyles (CSS !important is intentional) and
useLiteralKeys (bracket notation used for computed/dynamic keys) in
biome.json. Remaining 49 warnings are all noExcessiveCognitiveComplexity
(already configured as warn, safe to address incrementally).

* refactor(enrichment): dedupe domain/org normalization helpers (#1707)

* refactor: dedupe github latest release fetch wiring (#1704)

* fix(military): tighten flight classification (#1713)

* fix(forecast): bundle military surge inputs

* fix(military): tighten flight classification

* refactor: dedupe edge api json response assembly (#1702)

* refactor: dedupe edge api json response assembly

* refactor: expand jsonResponse helper to all edge functions

Roll out jsonResponse() from _json-response.js to 16 files (14 handlers
+ 2 shared helpers), eliminating 55 instances of the
new Response(JSON.stringify(...)) boilerplate.

Only exception: health.js uses JSON.stringify(body, null, indent) for
pretty-print mode, which is incompatible with the helper signature.

Replaced local jsonResponse/json() definitions in contact.js,
register-interest.js, and cache-purge.js with the shared import.

* fix(seeds): skip transient redis lock timeouts (#1714)

* fix(seeds): skip transient redis lock timeouts

* docs(seeds): clarify transient redis error matching

* test: expand transient redis error coverage

Add tests for ECONNRESET, DNS failure (EAI_AGAIN), ETIMEDOUT, and
negative cases (HTTP 403, payload size) to confirm isTransientRedisError
only matches network-level failures, not app-level Redis errors.

* refactor(economic): dedupe FRED series fetch/parse helper (#1715)

* fix(ucdp): page error logging, page-0 fallback, TTL extension (#1717)

* fix(ucdp): add page error logging, page-0 fallback, and TTL extension on empty

Three resilience improvements for UCDP seed loop:

1. Log actual error messages on page fetch failures instead of silently
   swallowing them. Enables diagnosing API outages vs rate limits.

2. Fall back to page 0 data when all newest-page fetches fail. Page 0
   is already fetched during version discovery, so this is free. Provides
   partial (older) data instead of writing 0 events.

3. When 0 events remain after processing, extend existing Redis key TTL
   instead of overwriting with empty payload. Preserves stale-but-valid
   data for the next cycle rather than causing EMPTY_DATA CRIT in health.

* fix(ucdp): remove page-0 fallback, stop seed-meta on failed fetches

P1 fixes from review:
- Remove page-0 fallback that overwrote last known good cache with
  stale historical data. Extend existing key TTL instead.
- Stop writing fresh seed-meta timestamps when no new payload is
  written (both all-pages-failed and empty-after-filtering branches).
  Health checks should reflect actual data freshness, not failed attempts.

Add 6 targeted source-analysis tests verifying:
- Error logging on page failures
- No page-0 data injection
- TTL extension on failure branches
- seed-meta only written on successful publish

* fix(military): improve source-backed flight inference (#1716)

* fix(military): improve source-backed flight inference

* fix(military): tighten operator metadata matching

* fix(military): tighten source hint inference

* fix(blog): apply SEO and content improvements to supply chain post (#1728)

- Shorten meta description to 155 chars (was truncating in SERPs)
- Shorten metaTitle to under 60 chars
- Add Key Takeaways box for skimmers and AI citation
- Move brand name into first paragraph
- Add "Data as of" timestamp note
- Rename H2 with target keyword "maritime chokepoints"
- Add 3 internal links to related blog posts
- Add FAQ section (BDI, Hormuz oil impact, critical chokepoints)
- Strengthen CTA with specific panel actions

* feat(military): add enrichment audit waterfall (#1730)

* fix(military): improve source-backed flight inference

* fix(military): tighten operator metadata matching

* fix(military): tighten source hint inference

* feat(military): add enrichment audit waterfall

* feat(military): capture live source shape gaps

* fix(blog): SEO improvements across all 16 blog posts (#1729)

- Trim metaTitles to under 58 chars (were 63-96, all truncated in SERPs)
- Adjust meta descriptions to 150-160 char range
- Add 2-3 internal blog post links per article for cross-linking
- Add FAQ sections (2-3 questions each) for long-tail keyword capture
- No content changes to article body text

* fix(sidecar): block cloud fallback in Docker mode (#1726)

Self-hosted Docker instances must not proxy unhandled routes to
api.worldmonitor.app. When LOCAL_API_MODE=docker, cloudFallback is
forced to false regardless of LOCAL_API_CLOUD_FALLBACK env var.
Logs a warning if the user explicitly requested fallback.

Prevents self-hosted users from unknowingly sending traffic to the
production Vercel deployment.

* fix(military): use opensky oauth in seed (#1738)

* fix(military): use opensky oauth in seed

* feat(military): log opensky source usage

* fix(seeds): extend seed-meta TTL alongside data keys on fetch failure (#1724)

When upstream APIs fail and seeds extend existing data key TTLs, the
seed-meta key was left untouched. Health checks use seed-meta fetchedAt
to determine staleness, so preserved data still triggered STALE_SEED
warnings even though the data was valid.

Now all TTL extension paths include the corresponding seed-meta key:
- _seed-utils.mjs runSeed() (fetch failure + validation skip)
- fetch-gpsjam.mjs (Wingbits 500 fallback)
- seed-airport-delays.mjs (FAA fetch failure)
- seed-military-flights.mjs (OpenSky fetch failure)
- seed-service-statuses.mjs (RPC fetch failure)

* feat(economic): add macro stress signals (clean cherry-pick) (#1719)

Cherry-pick economic changes from codex/signal-architecture-phase1 onto
clean main. Removed gitignored docs/internal/ file and 23 unrelated
files from scope drift.

Changes:
- Add HY spread (BAMLH0A0HYM2), jobless claims (ICSA), 30Y mortgage
  (MORTGAGE30US), and GSCPI to ALLOWED_SERIES
- Fix display math: toDisplayValue scales units correctly for WALCL
  (trillions) and basis-point series
- Bump batch limit from 10 to 20 series
- Preserve Redis-read-only pattern (no live FRED fetch on Vercel)

* perf(sw): throttled event-driven SW updates (#1718)

* perf(sw): replace 5-min polling with throttled event-driven updates

Replace aggressive 5-minute registration.update() loop with throttled
event-driven checks: on tab visibility, online reconnect, and an hourly
visible-only fallback. Cross-tab throttling via shared localStorage
timestamp prevents multiple tabs from checking simultaneously.

Reduces /sw.js request volume from ~12/hr/tab to ~1/hr across all tabs
while maintaining fast deploy pickup on tab return or reconnect.

* fix(sw): stateful first-visit guard + failure-aware throttle

Two fixes from review:

1. controllerchange guard: hadController was const, so first-visit tabs
   never reloaded on later deploys. Now let + set true after skipping
   the initial control-taking event.

2. Throttle writes timestamp after successful check (not before). On
   failure, retries every 5min instead of waiting the full 1hr cooldown.
   Uses separate localStorage keys for last-check vs last-success.

* fix(sw): 15min poll interval + warn on update check failure

- Change setInterval from 5min to 15min to reduce unnecessary ticks
  (throttle still gates to 1hr on success, 5min on failure)
- Log update check failures with console.warn instead of silently
  swallowing them

* fix(military): prefer direct opensky over proxy (#1742)

* fix(forecast): improve ranking and enrichment coverage (#1745)

* feat(forecast): add trace quality summary (#1746)

* feat(forecast): add trace quality summary

* fix(forecast): split traced and full-run quality metrics

* refactor(military): dedupe Wingbits aircraft-details fetch helper (#1737)

* refactor(military): dedupe Wingbits aircraft-details fetch helper

* fix(military): correct cached details type in batch endpoint

* refactor(api): dedupe in-memory IP rate limiter (#1740)

* feat: add Radiation Watch with seeded anomaly intelligence, map layers, and country exposure (#1735)

* feat(widgets): AI widget builder with live WorldMonitor data (#1732)

* fix(docs): exclude /docs from CSP that blocks Mintlify (#1750)

* fix(docs): exclude /docs from CSP header that blocks Mintlify scripts

The catch-all /(.*) header rule applied Content-Security-Policy with
SHA-based script-src to all routes including /docs/*. Mintlify generates
dozens of inline scripts that don't match those hashes, causing 71 CSP
errors and a completely blank docs page.

Fix: change catch-all to /((?!docs).*) so /docs paths inherit only
their own lightweight headers (nosniff, HSTS, referrer-policy).

* fix(tests): update deploy-config test for docs CSP exclusion

Test was looking for exact source '/(.*)', updated to match the new
'/((?!docs).*)' pattern that excludes /docs from the strict CSP.

* feat(sanctions): add OFAC sanctions pressure intelligence (#1739)

* feat(sanctions): add OFAC sanctions pressure intelligence

* fix(sanctions): strip _state from API response, fix code/name alignment, cap seed limit

- trimResponse now destructures _state before spreading to prevent seed
  internals leaking to API clients during the atomicPublish→afterPublish window
- buildLocationMap and extractPartyCountries now sort (code, name) as aligned
  pairs instead of calling uniqueSorted independently on each array; fixes
  code↔name mispairing for OFAC-specific codes like XC (Crimea) where
  alphabetic order of codes and names diverges
- DEFAULT_RECENT_LIMIT reduced from 120 to 60 to match MAX_ITEMS_LIMIT so
  seeded entries beyond the handler cap are not written unnecessarily
- Add tests/sanctions-pressure.test.mjs covering all three invariants

* fix(sanctions): register sanctions:pressure:v1 in health.js BOOTSTRAP_KEYS and SEED_META

Adds sanctionsPressure to health.js so the health endpoint monitors the
seeded key for emptiness (CRIT) and freshness via seed-meta:sanctions:pressure
(maxStaleMin: 720 matches 12h seed TTL). Without this, health was blind to
stale or missing sanctions data.

* feat(llm): support forecast model overrides (#1751)

* refactor(sanctions): simplify handler to Redis-read-only, fix seed OOM risk (#1753)

* refactor(sanctions): simplify handler to Redis-read-only, fix seed OOM risk

Handler (424→56 lines):
- Remove live OFAC fetch fallback from Vercel Edge handler: XMLParser,
  OFAC_SOURCES, fetchSource, collectPressure, cachedFetchJson fallback.
  Vercel reads Redis only; Railway makes all external API calls.
- On seed miss/empty, return emptyResponse() matching the radiation pattern.

Seed:
- Fetch SDN then Consolidated sequentially instead of Promise.all.
  Combined parallel parse peaks at ~150MB, tight against 512MB heap limit.

Tests:
- Add gold standard compliance assertions (no XMLParser, no OFAC_SOURCES).
- Add memory safety assertion (no Promise.all on OFAC sources).
- Replace handler XML-function tests (removed code) with Redis-read assertions.

* chore: exclude DMCA-TAKEDOWN-NOTICE.md from markdownlint

* fix(csp): add missing Vercel inline script hash (#1756)

Add sha256-903UI9my1I7mqHoiVeZSc56yd50YoRJTB2269QqL76w= to script-src
to allow Vercel-injected inline script that was being blocked by CSP.

* fix(sanctions): add fast-xml-parser to Railway scripts deps (#1755)

seed-sanctions-pressure.mjs imports fast-xml-parser to parse OFAC SDN
XML feeds, but the package was never added to scripts/package.json.
Railway deploys crash with ERR_MODULE_NOT_FOUND on startup.

* fix(widget): add @anthropic-ai/sdk to root deps for Railway (#1757)

The relay dynamically imports @anthropic-ai/sdk but it was only in
scripts/package.json. Railway Nixpacks only installs root deps, so the
package was missing at runtime (ERR_MODULE_NOT_FOUND).

* chore(forecast): log llm stage routing (#1754)

* fix(sanctions): add progress logging to seed (fetch size, entry count, new count) (#1758)

* fix(gdelt): make RPC handler Redis-read-only (gold standard) (#1759)

* fix(relay): add root nixpacks.toml to install scripts/ deps on Railway (#1760)

Railway builds from repo root (root_dir=""), so scripts/nixpacks.toml is
skipped. Root packages are installed via npm ci but scripts/package.json
deps (@anthropic-ai/sdk, fast-xml-parser, etc.) never land in
/app/node_modules, causing ERR_MODULE_NOT_FOUND at runtime.

Add a root nixpacks.toml that:
- Preserves existing scripts/nixpacks.toml settings (curl, NODE_OPTIONS)
- Adds `npm install --prefix scripts` to the build phase so all
  scripts/package.json deps are installed alongside root deps

* fix(forecast): tighten quality and enrichment balance (#1761)

* fix: enforce gold standard on UCDP and theater posture handlers (#1763)

Make list-ucdp-events.ts Redis-read-only (was calling ucdpapi.pcr.uu.se
directly from Vercel Edge). Reduces 160 lines to 28.

Make get-theater-posture.ts Redis-read-only (was calling OpenSky +
Wingbits directly from Vercel Edge). Reduces 273 lines to 38. Keeps
live/stale/backup Redis fallback chain.

Update theater posture tests to verify read-only behavior instead of
testing upstream coalescing and cache writes.

All UCDP/theater data assembly now exclusively on Railway
(ais-relay.cjs + seed-military-flights.mjs).

* test(sanctions): add 74 unit tests for seed parsing/transformation functions (#1762)

Tests cover all pure helper functions in scripts/seed-sanctions-pressure.mjs:
listify, textValue, buildEpoch, uniqueSorted, compactNote,
extractDocumentedName, normalizeDateOfIssue, buildReferenceMaps,
buildLocationMap, resolveEntityType, extractPartyName, extractPrograms,
extractEffectiveAt, extractNote, sortEntries, buildCountryPressure,
buildProgramPressure, and an end-to-end buildEntriesForDocument fixture.

Uses node:vm to load pure functions in an isolated context, bypassing
the loadEnvFile + runSeed side effects that fire on module import.

* Add thermal escalation seeded service (#1747)

* feat(thermal): add thermal escalation seeded service

Cherry-picked from codex/thermal-escalation-phase1 and retargeted
to main. Includes thermal escalation seed script, RPC handler,
proto definitions, bootstrap/health/seed-health wiring, gateway
cache tier, client service, and tests.

* fix(thermal): wire data-loader, fix typing, recalculate summary

Wire fetchThermalEscalations into data-loader.ts with panel forwarding,
freshness tracking, and variant gating. Fix seed-health intervalMin from
90 to 180 to match 3h TTL. Replace 8 as-any casts with typed interface.
Recalculate summary counts after maxItems slice.

* fix(thermal): enforce maxItems on hydrated data + fix bootstrap keys

Codex P2: hydration branch now slices clusters to maxItems before
mapping, matching the RPC fallback behavior.

Also add thermalEscalation to bootstrap.js BOOTSTRAP_CACHE_KEYS and
SLOW_KEYS (was lost during conflict resolution).

* fix(thermal): recalculate summary on sliced hydrated clusters

When maxItems truncates the cluster array from bootstrap hydration,
the summary was still using the original full-set counts. Now
recalculates clusterCount, elevatedCount, spikeCount, etc. on the
sliced array, matching the handler's behavior.

* fix(forecast): address P2/P3 code review findings from PR #1761 (#1765)

- Strip enrichmentMeta from bootstrap.js forecasts payload (seed-internal, not for clients)
- Rename quietDomainBonus -> priorityDomainBonus (it applies to priority domains, not quiet ones)
- Extract cyber score formula magic numbers into named constants (CYBER_SCORE_TYPE_MULTIPLIER, etc.)
- Pre-compute analysisPriority in rankForecastsForAnalysis to avoid double-call per comparison
- Log when filterPublishedForecasts weak-fallback gate suppresses forecasts
- Log how many fallback narratives populateFallbackNarratives applies
- Add // penalties comment header in computeAnalysisPriority

* feat(ui): separate macro stress and energy complex panels (#1749)

* feat(ui): separate macro stress and energy complex panels

Cherry-picked from chained branch onto clean main. Splits the Economic
panel into focused Macro Stress (FRED indicators) and Energy Complex
(oil analytics + market tape) panels.

Also fixes negative dollar-B changes now display with minus sign (was
rendering as positive due to Math.abs without sign prefix).

* fix: restore BIS/spending loading, split commodity/energy flags

Addresses 2 blockers from PR review:

1. Restore BIS and USASpending data loading in all 3 call sites
   (prime tasks, periodic refresh, initial load). Restore full
   spending and centralBanks tab rendering in EconomicPanel while
   keeping the new macro stress indicators view.

2. Split shared commoditiesLoaded flag into separate metalsLoaded
   and energyLoaded flags so each panel falls back independently.
   Energy data arriving cannot suppress metals fallback fetch.

EconomicPanel now has 3 tabs: Indicators (macro stress), Spending,
Central Banks. Oil tab removed (lives in EnergyComplexPanel).

* fix(i18n): restore economic panel translation keys for BIS/spending tabs

PR removed i18n keys needed by the restored spending and centralBanks
tabs (gov, centralBanks, awards, policyRate, etc.). Also updated the
infoTooltip to accurately describe the mixed surface (macro + gov +
central banks) instead of claiming macro-only.

* fix(seeds): add empty-data guards and fix health semantics (#1767)

Health semantics:
- Add faaDelays + gpsjam to EMPTY_DATA_OK_KEYS (0 records = calm, not error)
- Fix EMPTY_DATA_OK_KEYS branch to still check seed-meta freshness
  (prevents stale empty caches from staying green indefinitely)

Seed guards:
- seed-airport-delays: fix meta key in fetch-failure path
  (seed-meta:aviation:delays -> seed-meta:aviation:faa + seed-meta:aviation:notam)
- seed-military-flights: add full TTL extension on zero-flights branch
  (was exiting without preserving any derived data TTLs)
- seed-wb-indicators: add percentage-drop guard (new count < 50% of cached
  = likely partial API failure, extend TTL instead of overwriting)
- ais-relay.cjs: same percentage-drop guard for WB dual writer

Codex-reviewed plan (5 rounds, approved).

* Add 7 new tier-1 countries to risk scoring and monitoring (#1768)

* fix: include all curated countries in CII scoring (LB, IQ, AF, KR, EG, JP, QA)

The CII was only scoring 24 "tier-1" countries while 7 curated countries
(Lebanon, Iraq, Afghanistan, South Korea, Egypt, Japan, Qatar) had full
configurations but were excluded from instability scoring. This expands
TIER1_COUNTRIES to all 31 curated countries so every monitored country
gets proper CII scores with baselines, keywords, bounding boxes, zone
mappings, and travel advisory fallbacks.

https://claude.ai/code/sessio…
nichm pushed a commit to nichm/worldmonitor-private that referenced this pull request Jul 1, 2026
…t group (koala73#1596)

- Remove 403-blocked feeds: Breaking Defense, My Modern Met, AEI
- Fix Infobae URL: /feeds/rss/ -> /arc/outboundfeeds/rss/ (both files)
- Fix CSIS URL: /feed -> /rss.xml (server _feeds.ts)
- Drop 'active' from CelesTrak SAT_GROUPS (>2MB, always rejected)
nichm pushed a commit to nichm/worldmonitor-private that referenced this pull request Jul 1, 2026
…oala73#1626)

PR koala73#1596 removed the feeds but left the domains in the allowlist.
The relay still accepted proxy requests for these 403-blocked domains
from clients with cached old bundles. Removed:
- breakingdefense.com (403)
- www.arabnews.com (403)
- www.aei.org (403)
- mymodernmet.com (403)

Updated all 3 copies: shared/, scripts/shared/, api/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant