fix(feeds): remove dead RSS feeds, fix broken URLs#1596
Conversation
…t group - Remove 403-blocked feeds: Breaking Defense, My Modern Met, AEI - Fix Infobae URL: /feeds/rss/ -> /arc/outboundfeeds/rss/ (both files) - Fix CSIS URL: /feed -> /rss.xml (server _feeds.ts) - Drop 'active' from CelesTrak SAT_GROUPS (>2MB, always rejected)
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0e7efd5ffe
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // Defense & Security (Tier 1) | ||
| { name: 'Defense One', url: rss('https://www.defenseone.com/rss/all/'), type: 'defense' }, | ||
| { name: 'Breaking Defense', url: rss('https://breakingdefense.com/feed/'), type: 'defense' }, | ||
| { name: 'The War Zone', url: rss('https://www.twz.com/feed'), type: 'defense' }, |
There was a problem hiding this comment.
Keep default intel sources aligned with removed feed
Removing Breaking Defense from INTEL_SOURCES here leaves DEFAULT_ENABLED_INTEL still referencing that source, which creates a dangling default entry: dev mode now emits the DEFAULT_ENABLED name "Breaking Defense" not found error and the intel panel initializes with one fewer real default source than configured. Please remove or replace the stale default entry when deleting this feed.
Useful? React with 👍 / 👎.
PR #1596 removed the feeds but left the domains in the allowlist. The relay still accepted proxy requests for these 403-blocked domains from clients with cached old bundles. Removed: - breakingdefense.com (403) - www.arabnews.com (403) - www.aei.org (403) - mymodernmet.com (403) Updated all 3 copies: shared/, scripts/shared/, api/
…1626) PR #1596 removed the feeds but left the domains in the allowlist. The relay still accepted proxy requests for these 403-blocked domains from clients with cached old bundles. Removed: - breakingdefense.com (403) - www.arabnews.com (403) - www.aei.org (403) - mymodernmet.com (403) Updated all 3 copies: shared/, scripts/shared/, api/
…13) * fix(seeds): increase forecast TTL from 3600s to 4200s (#1625) TTL (1h) equaled cron interval (1h), leaving zero buffer for container cold starts. Health saw EMPTY records=0 during the gap between key expiry and next seed run. 70min TTL covers the cold start window. * fix(feeds): remove dead feed domains from RSS allowed-domains list (#1626) PR #1596 removed the feeds but left the domains in the allowlist. The relay still accepted proxy requests for these 403-blocked domains from clients with cached old bundles. Removed: - breakingdefense.com (403) - www.arabnews.com (403) - www.aei.org (403) - mymodernmet.com (403) Updated all 3 copies: shared/, scripts/shared/, api/ * refactor(seeds): extract USNI fleet seed from relay with proxy support (#1627) * refactor(seeds): extract USNI fleet seed from relay with proxy support USNI's Cloudflare blocks Railway IPs (JA3 fingerprinting), causing the relay's USNI seed to fail with HTML response instead of JSON. Changes: - New seed-usni-fleet.mjs: standalone script with HTTP CONNECT proxy support via RESIDENTIAL_PROXY_AUTH (falls back to direct fetch) - Removed ~240 lines of USNI code from ais-relay.cjs - TTL bumped from 21600s (6h) to 25200s (7h) for cold start buffer - Exact same parsing logic (hull types, region coords, vessel extraction, strike group detection, battle force summary) - runSeed pattern with lock, validation, seed-meta, verification Deploy: Railway cron service, 6h interval, needs RESIDENTIAL_PROXY_AUTH * fix(seeds): only use RESIDENTIAL_PROXY_AUTH for USNI proxy * fix(seeds): use OREF_PROXY_AUTH for USNI proxy (the one proxy available) * fix(supply-chain): remount transit chart on no-op renders (#1630) * refactor: share edge Turnstile helper (#1628) * refactor: dedupe signals upstream fetch headers and timeout (#1629) * fix(seeds): use curl for USNI fetch (JA3 fingerprint bypass) (#1631) Node.js TLS fingerprint gets blocked by Cloudflare. curl's fingerprint passes. Strategy: curl direct → curl+proxy → Node.js direct. Same pattern as relay's orefCurlFetch. * perf(ci): skip Vercel deploy when only scripts/ change on main (#1633) Previously, every merge to main triggered a Vercel build even for scripts-only changes (seed scripts, relay updates). Now checks if any web-relevant files changed on main too, skipping the build when only scripts/, docs/, .github/, etc. are modified. * test(seeds): cover warm-ping origin header (#1632) * fix(usni-fleet): add Node.js proxy fallback for Cloudflare bypass (#1635) * fix(usni-fleet): add Node.js HTTP CONNECT proxy fallback, detect Cloudflare HTML curl is not available in Railway's Railpack v0.18.0 containers. The seed was failing with ENOENT on curl, then getting Cloudflare-blocked on Node.js direct. - Add fetchViaHttpProxy: Node.js HTTP CONNECT tunnel through residential proxy (no curl dependency). Uses the same RESIDENTIAL_PROXY_AUTH env. - Add Cloudflare HTML detection: reject early when response starts with <!DOCTYPE instead of passing HTML to JSON.parse. - Fallback chain: curl direct -> curl+proxy -> Node.js+proxy -> Node.js direct - Add nixpacks.toml with curl for future Railpack builds * fix: use ESM import for node:http (require breaks in .mjs) * test: rewrite transit chart test as structural contract verification (#1636) Replace fragile source-string extraction + new Function() compilation with structural pattern checks on the source code. Tests verify: - render() clears chart before content change - clearTransitChart() cancels timer, disconnects observer, destroys chart - MutationObserver setup for DOM readiness detection - Fallback timer for no-op renders (100-500ms range) - Both callbacks (observer + timer) clean up each other - Tab switch and collapse clear chart state - Mount function guards against missing element/data Replaces PR #1634's approach which was brittle (method body extraction, TypeScript cast stripping, sandboxed execution). * fix: log fetch error cause in seed retry/FATAL handlers (#1638) * test: rewrite transit chart test as structural contract verification Replace fragile source-string extraction + new Function() compilation with structural pattern checks on the source code. Tests verify: - render() clears chart before content change - clearTransitChart() cancels timer, disconnects observer, destroys chart - MutationObserver setup for DOM readiness detection - Fallback timer for no-op renders (100-500ms range) - Both callbacks (observer + timer) clean up each other - Tab switch and collapse clear chart state - Mount function guards against missing element/data Replaces PR #1634's approach which was brittle (method body extraction, TypeScript cast stripping, sandboxed execution). * fix: log fetch error cause in seed retry and FATAL handlers Node 20 fetch() throws TypeError('fetch failed') with the real error hidden in err.cause (DNS, TLS, timeout). The current logging only shows 'fetch failed' which is useless for diagnosis. Now logs err.cause.message in both withRetry() retries and FATAL catch blocks. * fix(railway): keep Node 20 + force IPv4 DNS ordering (#1641) Node 20's fetch() (undici) tries IPv6 first. Railway containers don't support IPv6 (IPV6_NDISC failures in network trace), causing all seed services to crash. Fix: set NODE_OPTIONS=--dns-result-order=ipv4first via nixpacks.toml so all Railway services prefer IPv4. Keeps Node 20 for import attributes. * refactor: dedupe contact api json responses (#1639) * refactor(llm): consolidate provider chain to single source of truth (#1640) * fix(relay): add LLM fallback chain to ais-relay classify Replace single Groq-only LLM call with provider fallback chain (Groq → OpenRouter → Ollama) matching seed-insights.mjs pattern. If Groq fails or is unavailable, classify falls through to the next configured provider automatically. * refactor(llm): consolidate provider chain to single source of truth - Fix OpenRouter model: openrouter/free → google/gemini-2.5-flash in canonical llm.ts - Migrate 4 intelligence handlers (classify-event, batch-classify, deduct-situation, get-country-intel-brief) from hardcoded Groq-only to callLlm() with full ollama → groq → openrouter fallback chain - Remove duplicate getProviderCredentials from news/v1/_shared.ts, re-export canonical - Remove orphaned GROQ_API_URL/GROQ_MODEL from intelligence/v1/_shared.ts - Reorder script provider chains (ais-relay.cjs, seed-insights.mjs) to canonical ollama → groq → openrouter order - Net -161 lines: eliminated duplicated provider logic across 9 files * fix: eliminate double JSON parse in classify-event, throw on runSeed verification failure * fix(tests): add llm module alias to country-intel-brief test fixture * fix: preserve generic LLM_API_* fallback, add retry to seed verification - Add 'generic' provider to callLlm() chain for LLM_API_URL/LLM_API_KEY/LLM_MODEL (preserves existing OpenAI-compatible endpoint contract) - Change seed verification to warn-only with 1 retry instead of fatal throw (write already succeeded, transient read failure shouldn't fail the job) - Update docs to reflect new provider fallback chain * feat(advisories): gold standard migration for security advisories (#1637) * feat(advisories): gold standard migration for security advisories Move security advisories from client-side RSS fetching (24 feeds per page load) to Railway cron seed with Redis-read-only Vercel handler. - Add seed script fetching via relay RSS proxy with domain allowlist - Add ListSecurityAdvisories proto, handler, and RPC cache tier - Add bootstrap hydration key for instant page load - Rewrite client service: bootstrap -> RPC fallback, no browser RSS - Wire health.js, seed-health.js, and dataSize tracking * fix(advisories): empty RPC returns ok:true, use full country map P1 fixes from Codex review: - Return ok:true for empty-but-successful RPC responses so the panel clears to empty instead of stuck loading on cold environments - Replace 50-entry hardcoded country map with 251-entry shared config generated from the project GeoJSON + aliases, matching coverage of the old client-side nameToCountryCode matcher * fix(advisories): add Cote d'Ivoire and other missing country aliases Adds 14 missing aliases including "cote d ivoire" (US State Dept title format), common article-prefixed names (the Bahamas, the Gambia), and alternative official names (Czechia, Eswatini, Cabo Verde, Timor-Leste). * fix(proto): inject @ts-nocheck via Makefile generate target buf generate does not emit @ts-nocheck, but tsc strict mode rejects the generated code. Adding a post-generation sed step in the Makefile ensures both CI proto-freshness (make generate + diff) and CI typecheck (tsc --noEmit) pass consistently. * fix(predictions): remove Kalshi API key, enforce Redis-only on Vercel (#1642) Kalshi public market data endpoints require no authentication. Remove the KALSHI_API_KEY gate that was disabling Kalshi entirely when the env var was missing, and drop the Authorization header. Rewrite the Vercel RPC handler to read from Railway-seeded Redis only (gold standard), removing the fallback that fetched directly from Gamma/Kalshi APIs on Vercel edge. Handler goes from 330 to 85 lines. Double all prediction timing values to reduce Railway cron cost: - Redis TTL: 15min -> 30min - Health maxStaleMin: 15min -> 30min - Client hydration freshness: 20min -> 40min - Railway cron: 10min -> 20min (requires dashboard update) * fix: add fetch error cause logging to all remaining seed scripts (#1643) * fix(seeds): improve resilience and fix dead APIs across seed scripts (#1644) * fix(seeds): improve resilience and fix dead APIs across seed scripts - Fix wrong domain in seed-service-statuses (worldmonitor.app to api.worldmonitor.app) - Fix Kalshi API domain migration (trading-api.kalshi.com to api.elections.kalshi.com) - Replace dead trending APIs (gitterapp.com, herokuapp.com) with OSSInsight + GitHub Search - Fix case-sensitive HTML detection in seed-usni-fleet (lowercase doctype not matched) - Add Promise.allSettled rejection logging across 8 seed scripts - Wrap fetch loops in try-catch (seed-supply-chain-trade, seed-economy) so a single network error no longer kills the entire function - Update list-trending-repos.ts RPC handler to match seed changes * fix(seeds): correct OSSInsight response parsing and period-aware GitHub Search fallback - OSSInsight returns {data: {rows: [...]}} not {data: [...]}, fix both seed and handler - GitHub Search fallback now respects period parameter (daily=1d, weekly=7d, monthly=30d) * fix(seeds): correct OSSInsight period values (past_week/past_month, not past_7_days/past_28_days) * Add Greek news channels & feed (#1602) * Add Greek news channels * Add ERT and SKAI hlsUrl to LiveNewsPanel.ts --------- Co-authored-by: Elie Habib <[email protected]> * feat(scoring): consume live advisory data from Redis with hardcoded fallback (#1620) Read live travel advisory levels from intelligence:advisories:v1 Redis key (populated by seed-security-advisories.mjs). Fall back to the original hardcoded ADVISORY_LEVELS_FALLBACK map when Redis data is unavailable. Closes #1359. * fix(sentry): tighten noise filters for 6 unresolved issues (#1645) Fix 2 regexes that let errors slip through: - AbortError filter now handles value without type prefix - Qt() filter now matches Qt(...) with arguments Add 4 new filters for browser-specific noise: - ES module syntax errors (Pale Moon, QQ Browser) - ucConfig (UC Browser V8 format) - getShaderPrecisionFormat (WebGL context loss on iOS) * refactor: dedupe register-interest json responses (#1647) * fix(usni): move USNI fleet seed back to AIS relay (fixed IP for Froxy proxy) (#1648) The standalone seed-usni-fleet.mjs cannot reach USNI because: 1. USNI Cloudflare blocks Node.js TLS fingerprint (JA3) 2. curl is not installed on Railway cron containers 3. Froxy residential proxy is IP-whitelisted to the relay fixed IP Move the USNI seed loop back into ais-relay.cjs where it has access to curl + the whitelisted proxy. Uses orefCurlFetch for the fetch, same pattern as the OREF alerts loop. Writes to the same Redis keys (usni-fleet:sebuf:v1, stale:v1, seed-meta:military:usni-fleet). 6h seed interval, 7h TTL, 7d stale TTL (unchanged from standalone). * fix(seeds): allow conflict-intel seed to succeed without ACLED keys (#1651) Validation now accepts empty ACLED events array when humanitarian or pizzint data was fetched. Previously the seed wrote extra keys (humanitarian, pizzint) but skipped the canonical key because validateFn required non-empty events. * fix(data): restore bootstrap and cache test coverage (#1649) * fix(data): restore bootstrap and cache test coverage * fix: resolve linting and test failures - Remove dead writeSeedMeta/estimateRecordCount functions from redis.ts (intentionally removed from cachedFetchJson; seed-meta now written only by explicit seed flows, not generic cache reads) - Fix globe dayNight test to match actual code (forces dayNight: false + hideLayerToggle, not catalog-based exclusion) - Fix country-geometry test mock URL from CDN to /data/countries.geojson (source changed to use local bundled file) * fix(lint): remove duplicate llm-health key in redis-caching test Duplicate object key '../../../_shared/llm-health' caused the stub to be overwritten by the real module. Removed the second entry so the test correctly uses the stub. * feat(forecast): add structured scenario pipeline and trace export (#1646) * feat(forecast): add AI Forecasts prediction module (Pro-tier) MiroFish-inspired prediction engine that generates structured forecasts across 6 domains (conflict, market, supply chain, political, military, infrastructure) using existing WorldMonitor data streams. - Proto definitions for ForecastService with GetForecasts RPC - Dedicated seed script (seed-forecasts.mjs) with 6 domain detectors, cross-domain cascade resolver, prediction market calibration, and trend detection via prior snapshot comparison - Premium-gated RPC handler (PREMIUM_RPC_PATHS enforcement) - Lazy-loaded ForecastPanel with domain filters, probability bars, trend arrows, signal evidence, and cascade links - Health monitoring integration (seed-meta freshness tracking) - Refresh scheduler with API key guard * test(forecast): add 47 unit tests for forecast detectors and utilities Covers forecastId, normalize, resolveCascades, calibrateWithMarkets, computeTrends, and smoke tests for all 6 domain detectors. Exports testable functions from seed script with direct-run guard. * fix(forecast): domain mismatch 'infra' vs 'infrastructure', add panel category - Seed script used 'infra' but ForecastPanel filtered on 'infrastructure', causing Infra tab to show zero results - Added 'forecast' to intelligence category in PANEL_CATEGORY_MAP * fix(forecast): move CSS to one-time injection, improve type safety - P2: Move style block from setContent to one-time document.head injection to prevent CSS accumulation on repeated renders - P3: Replace +toFixed(3) with Math.round for readability in seed script - P3: Use Forecast type instead of any[] in RPC handler filter * fix(forecast): handle sebuf proto data shapes from Redis Detectors now normalize CII scores from server-side proto format (combinedScore, TREND_DIRECTION_RISING, region) to uniform shape. Outage severity handles proto enum format (SEVERITY_LEVEL_HIGH). Added confidence floor of 0.3 for single-source predictions. Verified against live Redis: 2 predictions generated (Iran infra shutdown, IL political instability). * feat(forecast): unlock AI Forecasts on web, lock desktop only (trial) - Remove forecast RPC from PREMIUM_RPC_PATHS (web access is free) - Panel locked on desktop only (same as oref-sirens/telegram-intel) - Remove API key guards from data-loader and refresh scheduler - Web users get full access during trial period * chore: regenerate proto types with make generate Re-ran make generate after rebasing on main. Plugin v0.7.0 dropped @ts-nocheck from output, added it back to all 50 generated files. Fixed 4 type errors from proto codegen changes: - MarketSource enum -> string union type - TemporalAnomalyProto -> TemporalAnomaly rename - webcam lastUpdated number -> string * chore: add proto freshness check to pre-push hook Runs make generate before push and compares checksums of generated files. If proto types are stale, blocks push with instructions to regenerate. Skips gracefully if buf CLI is not installed. * fix(forecast): use chokepoints v4 key, include ciiContribution in unrest - P1: Switch chokepoints input from stale v2 to active v4 Redis key, matching bootstrap.js and cache-keys.ts - P2: Add ciiContribution to unrest component fallback chain in normalizeCiiEntry so political detector reads the correct sebuf field * feat(forecast): Phase 2 LLM scenario enrichment + confidence model MiroFish-inspired enhancements: - LLM scenario narratives via Groq/OpenRouter (narrative-only, no numeric adjustment). Evidence-grounded prompts with mandatory signal citation and few-shot examples from MiroFish's SECTION_SYSTEM_PROMPT_TEMPLATE. - Top-4 predictions batched into single LLM call for cost efficiency. - News context from newsInsights attached to all predictions for LLM prompt grounding (NOT in signals, cannot affect confidence). - Deterministic confidence model: source diversity via SIGNAL_TO_SOURCE mapping (deduplicates cii+cii_delta, theater+indicators) + calibration agreement from prediction market drift. Floor 0.2, ceiling 1.0. - Output validation: rejects scenarios without signal references. - Truncated JSON repair for small model output. - Structured JSON logging for LLM calls. - Redis cache for LLM scenarios (1h TTL). - 23 new tests (70 total), all passing. - Live-tested: OpenRouter gemini-2.5-flash produces evidence-grounded scenario narratives from real WorldMonitor data. * feat(forecast): Phase 3 multi-perspective scenarios, projections, data-driven cascades MiroFish-inspired enhancements: - Multi-perspective LLM analysis: top-2 predictions get strategic, regional, and contrarian viewpoints via combined LLM call - Probability projections: domain-specific decay curves (h24/d7/d30) anchored to timeHorizon so probability equals projections[timeHorizon] - Data-driven cascade rules: moved from hardcoded array to JSON config (scripts/data/cascade-rules.json) with schema validation, named predicate evaluators, unknown key rejection, and fallback to defaults - 4 new cascade paths: infrastructure->supply_chain, infrastructure->market (both requiresSeverity:total), conflict->political, political->market - Proto: added Perspectives and Projections messages to Forecast - ForecastPanel: renders projections row and conditional perspectives toggle - 89 tests (19 new), all passing - Live-tested: OpenRouter produces perspectives from real data * feat(forecast): Phase 4 data utilization + entity graph Fixes data gaps that prevented 4 of 6 detectors from firing: - Input normalizers: chokepoint v4 shape + GPS hexes-to-zones mapping - Chokepoint warm-ping (production-only, requires WM_API_BASE_URL) - Lowered CII conflict threshold from 70 to 60, gated on level=high|critical 4 new standalone detectors: - UCDP conflict zones (10+ events per country) - Cyber threat concentration (5+ threats per country) - GPS jamming in maritime shipping zones (5 regions) - Prediction markets as signals (60-90% probability markets) Entity-relationship graph (file-based, 38 nodes): - Countries, theaters, commodities, chokepoints, alliances - Alias table resolves both ISO codes and display names - Graph cascade discovery links predictions across entities Result: 51 predictions (up from 1-2), spanning conflict, infrastructure, and supply chain domains. 112 tests, all passing. * fix(forecast): redis cache format, signal source mapping, type safety Fresh-eyes audit fixes: - BUG: redisSet used wrong Upstash API format (POST body with {value,ex} instead of command array ['SET',key,value,'EX',ttl]). LLM cache writes were silently failing, causing fresh LLM calls every run. - BUG: prediction_market signal type missing from SIGNAL_TO_SOURCE, inflating confidence for market-derived predictions. - CLEANUP: Remove unnecessary (f as any) casts in ForecastPanel since generated Forecast type already has projections/perspectives fields. - CLEANUP: Bump health maxStaleMin from 60 to 90 to avoid false STALE alerts when LLM calls add latency to seed runs. * feat(forecast): headline-entity matching with news corroboration signals Uses entity graph aliases to match headlines to predictions by country/theater (excludes commodity/infrastructure nodes to prevent false positives). Predictions with matching headlines get a news_corroboration signal visible in the panel. Also fixes buildUserPrompt to merge unique headlines from ALL predictions in the LLM batch (was only reading preds[0].newsContext). Live-tested: 13 of 51 predictions now have corroborating headlines (Iran, Israel, Syria, Ukraine, etc). 116 tests, all passing. * feat(forecast): add country-codes.json for headline-entity matching 56 countries with ISO codes, full names, and scoring keywords (extracted from src/config/countries.ts + UCDP-relevant additions). Used by attachNewsContext for richer headline matching via getSearchTermsForRegion which combines country-codes + entity graph + keyword aliases. 14/57 predictions now have news corroboration (limited by headline coverage, not matching quality: only 8 headlines currently available). * feat(forecast): read 300 headlines from news digest instead of 8 Read news:digest:v1:full:en (300 headlines across 16 categories) instead of just news:insights:v1 topStories (8 headlines). Fallback to topStories if digest is unavailable. Result: news corroboration jumped from 25% to 64% (38/59 predictions). * fix(forecast): handle parenthetical country names in headline matching Strip suffixes like '(Zaire)', '(Burma)', '(Soviet Union)' from UCDP region names before matching against country-codes.json. Also use includes() for reverse name lookup to catch partial matches. Corroboration: 64% -> 69% (41/59). Remaining 18 unmatched are countries with no current English-language news coverage. * fix(forecast): cache validated LLM output, add digest test, log cache errors Fresh-eyes audit fixes: - Combined LLM cache now stores only validated items (was caching raw unvalidated output, serving potentially invalid scenarios on cache hit) - redisSet logs warnings on failure (was silently swallowing all errors) - Added digest-based test for attachNewsContext (primary path was untested) - Fixed test arity: attachNewsContext(preds, news, digest) with 3 params * fix(forecast): remove dead confidenceFromSources, reduce warm-ping timeout - P2: Remove confidenceFromSources (dead code, computeConfidence overwrites all initial confidence values). Inline the formula in original detectors. - P3: Reduce warm-ping timeout from 30s to 15s (non-critical step) - P3: Add trial status comment on forecast panel config * fix(forecast): resolve ISO codes to country names, fix market detector, safe pre-push P1 fixes from code review: - CII ISO codes (IL, IR) now resolved to full country names (Israel, Iran) via country-codes.json. Prevents substring false positives (IL matching Chile) in event correlation. Uses word-boundary regex for matching. - Market detector CII-to-theater mapping now uses entity graph traversal instead of broken theater-name substring matching. Iran correctly maps to Middle East theater via graph links. - Pre-push hook no longer runs destructive git checkout on proto freshness failure. Reports mismatch and exits without modifying worktree. * feat(forecast): add structured scenario pipeline and trace export * fix(forecast): hydrate bootstrap and trim generated drift * fix(forecast): keep required supply-chain contract updates * fix(ci): add forecasts to cache-keys registry and regenerate proto Add forecasts entry to BOOTSTRAP_CACHE_KEYS and BOOTSTRAP_TIERS in cache-keys.ts to match api/bootstrap.js. Regenerate SupplyChain proto to fix duplicate TransitDayCount and add riskSummary/riskReportAction. * fix: restore desktop bootstrap timeouts and SW NetworkOnly (#1653) - Bootstrap: keep 1.2s/1.8s for web, restore 5s/8s for desktop (sidecar IPC + token resolution needs the headroom) - SW: revert navigation handler to NetworkOnly to prevent stale HTML caching that causes 404 storms on deploy * feat(supply-chain): restructure panel, add corridor disruption to shipping tab (#1652) * feat(supply-chain): restructure panel for scannable metrics and richer shipping tab Chokepoints tab: replace text blob description with structured metric rows showing warnings, vessels, WoW change, disruption %, and risk level. Move routing advisory into styled callout box on expand. Remove duplicated riskSummary and warning counts from server description field. Shipping Rates tab: add corridor disruption snapshot table showing per-corridor vessel counts, WoW traffic changes, disruption %, and risk levels. Tab now renders useful data even without FRED API key. Add CSS for expanded card state (previously no-op), metric rows, disruption color coding, routing advisory callout, and disruption table. * test(supply-chain): add restructure tests, update description assertion Add 7 structural tests for panel restructure: - activeHasData accepts chokepointData without FRED - renderShipping delegates to renderDisruptionSnapshot - null/empty loading states - data-cp-id and data-chart-cp preservation - conditional description rendering - server description no longer duplicates warning counts Update existing test: description now asserts counts are NOT in text (they moved to structured fields). * fix(seeds): lazy-load @aws-sdk/client-s3 in R2 storage helper (#1654) The top-level import crashes seed-forecasts on Railway when the package isn't installed. Dynamic import defers the load to when S3 mode is actually used, allowing the seed to run without the SDK when R2 is not configured. * test: cover runtime env guardrails (#1650) * fix(data): restore bootstrap and cache test coverage * test: cover runtime env guardrails * fix(test): align security header tests with current vercel.json Update catch-all source pattern, geolocation policy value, and picture-in-picture origins to match current production config. * debug(seeds): add R2 trace logging to seed-forecasts (#1655) Adds [Trace] log lines before/after R2 export and [R2] config resolution debug output to diagnose missing trace artifacts. * feat(search): add all missing panels to CMD+K command palette (#1656) 24 panels were not discoverable via CMD+K. Added commands for: map, webcams, AI insights/posture/forecasts, correlation panels (force posture, escalation, economic warfare, disaster cascade), fires, gulf economies, giving, UCDP events, displacement, climate, population exposure, security advisories, Israel sirens, telegram intel, airline intel, tech readiness, world clock, and layoffs. Coverage: 55/55 full-variant panels now searchable. * fix(health): close corridorrisk health monitoring gap (#1658) The corridorrisk raw key (2h TTL) expires between hourly seed cycles, causing health to report EMPTY even though data flows correctly through transit-summaries:v1. - Increase CORRIDOR_RISK_TTL from 2h to 4h (3 retries before expiry) - Add corridorrisk to ON_DEMAND_KEYS (WARN instead of CRIT when empty) * refactor: dedupe error mapper json responses (#1657) Extract jsonMessageResponse helper to replace 4 identical Response construction patterns in error-mapper.ts. * fix: use data-font attribute instead of inline style for font toggle (#1552) * fix: use data-font attribute instead of inline style for font toggle * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <[email protected]> * fix: use inline style for font-body-base to win cascade over variant themes Copilot review flagged that [data-font="system"] in @layer base loses to unlayered variant themes (e.g. happy-theme sets --font-body: Nunito). Switch back to inline style on --font-body-base so user font preference always wins the cascade, while keeping the RTL/CJK composition via var(--font-body-base) fallback. --------- Co-authored-by: Copilot Autofix powered by AI <[email protected]> Co-authored-by: Elie Habib <[email protected]> * feat(predictions): redesign panel with gradient bars, source accents, conviction labels (#1661) Visual redesign of prediction market cards: - Source badge (KALSHI/POLYMARKET) moved to card header with matching left accent stripe (blue for Kalshi, purple for Polymarket) - Gradient bars replace flat colors, with inner glow on strong signals - Conviction labels (Lean Yes / Lean No / Toss-up) based on threshold - Rounded bar corners (6px), thicker bars (28px), better spacing - Hover state on cards for interactivity feel * fix(predictions): prepend event title to short Kalshi outcome labels (#1659) Kalshi multi-outcome events return market titles like "Before 2035", "Rhode Island", "Johnny Depp" which are meaningless without the parent event context. Now combines event title with market title when the market title is short and doesn't contain a question mark. Before: "Before 2035" (KALSHI) After: "Will AGI be achieved?: Before 2035" (KALSHI) * fix(forecast): restore missing domains and split cyber (#1660) * fix(pwa): skip reload on first service worker registration (#1662) skipWaiting + clientsClaim fires controllerchange when the SW claims the page for the first time on a new session. The controllerchange listener was reloading unconditionally, causing a useless full page reload (double load of all assets and API calls) on every first visit. Now checks navigator.serviceWorker.controller at page load. If null (no prior SW), the controllerchange from initial claim is ignored. Deploy updates (new SW replacing existing) still trigger reload. * fix(health): prevent CF/CDN caching of health endpoint (#1665) CF was caching health responses despite no-cache, no-store because the Cache Rule marks /api/ as "eligible for cache" and CF interprets no-cache as "revalidate" not "don't cache". Changed to: - Cache-Control: private, no-store, max-age=0 (tells CF not cacheable) - CDN-Cache-Control: no-store (CF-specific override) * fix(health): add explicit seed-meta for chokepoints, move stale RPC keys to ON_DEMAND (#1667) - Chokepoints handler now writes seed-meta explicitly on fresh fetch (not on cache hits, per PR #1649 design) - Move riskScores and serviceStatuses to ON_DEMAND_KEYS (WARN not CRIT when stale; these are RPC-populated, not seeded) - cableHealth already in ON_DEMAND_KEYS * feat(trade): add US Treasury customs revenue to Trade Policy panel (#1663) * feat(trade): add US Treasury customs revenue to Trade Policy panel US customs duties revenue spiked 4-5x under Trump tariffs (from $7B/month to $27-31B/month) but the WTO tariff data only goes to 2024. Adds Treasury MTS data showing monthly customs revenue. - Add GetCustomsRevenue RPC (proto, handler, cache tier) - Add Treasury fetch to seed-supply-chain-trade.mjs (free API, no key) - Add Revenue tab to TradePolicyPanel with FYTD YoY comparison - Fix WTO gate: per-tab gating so Revenue works without WTO key - Wire bootstrap hydration, health, seed-health tracking * test(trade): add customs revenue feature tests 22 structural tests covering: - Handler: raw key mode, empty-cache behavior, correct Redis key - Seed: Treasury API URL, classification filter, timeout, row validation, amount conversion, sort order, seed-meta naming - Panel: WTO gate fix (per-tab not panel-wide), revenue tab defaults when WTO key missing, dynamic FYTD comparison - Client: no WTO feature gate, bootstrap hydration, type exports * fix(trade): align FYTD comparison by fiscal month count Prior FY comparison was filtering by calendar month, which compared 5 months of FY2026 (Oct-Feb) against only 2 months of FY2025 (Jan-Feb), inflating the YoY percentage. Now takes the first N months of the prior FY matching the current FY month count. * fix(trade): register treasury_revenue DataSourceId and localize revenue tab - Add treasury_revenue to DataSourceId union type so freshness tracking actually works (was silently ignored) - Register in data-freshness.ts source config + gap messages - Add i18n keys: revenue tab label, empty state, unavailable banner - Update infoTooltip to include Revenue tab description * fix(trade): complete revenue tab localization Use t() for all remaining hardcoded strings: footer source labels, FYTD summary headline, prior-year comparison, and table column headers. Wire the fytdLabel/vsPriorFy keys that were added but not used. * fix(test): update revenue source assertion for localized string * feat(supply-chain): add SCFI, CCFI, and BDI freight indices (#1666) * feat(supply-chain): add SCFI, CCFI, and BDI freight indices to shipping tab Transform the Shipping Rates tab from 2 lagging monthly FRED indices into a real-time freight cost dashboard with container and bulk shipping rates. Seed script: add fetchSCFI/fetchCCFI (SSE JSON API) and fetchBDI (HandyBulk HTML scrape) with inline history accumulation using source observation dates. Handler: make cache-only (seed is sole aggregator, no FRED fallback on miss). Panel: group indices into Container Rates, Bulk Shipping, Economic Indicators. Tests: 26 functional tests with fixture data for parsers, history, and handler. * fix(supply-chain): use raw Redis read and correct SCFI composite unit - Handler: switch from cachedFetchJson (env-prefixed) to getCachedJson(key, true) so preview deployments read the unprefixed seed key correctly - Seed: SCFI composite is a dimensionless index, not USD/TEU (route-level unit) - Tests: update assertions to match both fixes * fix(forecast): bump TTL from 3600s to 4800s to cover cron gap (#1668) Hourly cron + 3600s TTL = data expires right as the next seed starts, causing a ~30s EMPTY window. Bumped to 4800s (80min) so old data persists while the new seed runs. * fix(health): remove stale SEED_META for RPC-populated keys, bump to v2.6.5 (#1669) riskScores and serviceStatuses have data but permanently stale seed-meta (no longer written by cachedFetchJson after PR #1649). ON_DEMAND_KEYS only affects EMPTY status, not STALE_SEED. Removing their SEED_META entries so health doesn't check freshness for keys that can't update it. Also bumps version to 2.6.5. * fix(panels): refresh CMD+K active panels on open, add missing info tooltips (#1670) - Search: refresh activePanelIds in updateSearchIndex() so lazy-loaded panels (forecast, oref-sirens, telegram-intel, etc.) appear in CMD+K - Add ? info tooltips to AI Forecasts, Force Posture, and Disaster Cascade * fix(health): remove cableHealth SEED_META (RPC-populated, stale after PR #1649) (#1671) * fix(seed): bump BDI body size limit to 1MB, add gzip encoding (#1672) HandyBulk page is ~662KB, exceeding the 500KB guard. Also adds Accept-Encoding header to reduce transfer size. * fix(forecast): store full traces and tighten matching (#1673) * fix(forecast): restore missing domains and split cyber * fix(forecast): store full traces and tighten matching * fix(aviation): stop Vercel from calling AviationStack directly (#1674) * fix(aviation): stop Vercel from calling AviationStack directly - get-airport-ops-summary: read from relay seed cache (aviation:delays:intl:v3) instead of calling fetchAviationStackDelays() on every cache miss - list-airport-flights + get-flight-status: proxy through Railway relay /aviationstack endpoint instead of calling AviationStack from Vercel edge - Add /aviationstack proxy endpoint to ais-relay with 2min in-memory cache Vercel should NEVER call external paid APIs directly. Railway relay is the sole egress point for AviationStack (gold standard). * fix(config): update aviationStack feature to require WS_RELAY_URL Aviation handlers now proxy through Railway relay instead of calling AviationStack directly. Update runtime-config to reflect the actual dependency. * fix(supply-chain): force rebuild to deploy freight indices UI (#1680) Vercel build cache served stale bundle missing PR #1666 frontend changes (SCFI, CCFI, BDI grouping in SupplyChainPanel). Trivial whitespace change to invalidate Vite cache and force rebuild. * fix(trade): align flows cache key with seed (#1677) * fix(trade): align flows cache key with seed (US vs World, not China) The seed writes trade:flows:v1:840:000:10 (US vs World) but the data-loader requested trade:flows:v1:840:156:10 (US vs China), causing perpetual cache misses and a hidden Flows tab. * feat(seed): add bilateral trade flow pairs (US-China, US-Germany, etc.) Seed now writes both reporter-vs-World AND key bilateral pairs so switching between global and bilateral views hits warm cache. * feat(seed): add World-China and World-US bilateral flow pairs * fix(trade): revert flows to US-China (840/156), seed now covers this key The bilateral seed entries now write trade:flows:v1:840:156:10, so the original US-China request hits cache. Keeps the panel showing bilateral data consistent with the tariffs partner. * fix(forecast): log and honor full trace count (#1683) * fix(bootstrap): move shippingRates to FAST_KEYS for shorter CDN TTL (#1682) shippingRates was in SLOW_KEYS (2h CDN cache). After seed populates new freight indices (SCFI, CCFI, BDI), CF served stale bootstrap with only 2 FRED indices for up to 2.5h. Moving to FAST_KEYS (10min CDN TTL) ensures new data appears promptly. Also update BOOTSTRAP_TIERS in cache-keys.ts to match. * fix(handlers): pure Redis reads for 31 seed-backed Vercel handlers (#1684) * fix(handlers): convert 31 Vercel RPC handlers to pure Redis reads Enforce gold standard: Vercel reads Redis ONLY, Railway makes ALL external API calls. Replace cachedFetchJson (which fetches from external APIs on cache miss) with getCachedJson (pure Redis read from seed data). Converted domains: economic (8), market (7), trade (4), research (3), infrastructure (2), aviation (1), climate (1), cyber (1), natural (1), seismology (1), unrest (1), wildfire (1). Net -3,678 lines of external API call code removed from Vercel edge. * test: update structural tests for seed-only handlers Climate, wildfire, and unrest handlers no longer have TTL constants or ACLED imports since they are now pure cache readers. Update tests to verify getCachedJson usage instead of removed patterns. * fix(handlers): revert service-statuses, fix fred-batch key prefixing P1: Revert list-service-statuses.ts to original cachedFetchJson form. Both seeders (relay + cron) warm-ping the RPC which is the only writer. Making it read-only would break the data pipeline. This handler stays as-is until a proper seed script is built. P2: Replace getCachedJsonBatch with individual getCachedJson(key, true) calls in get-fred-series-batch.ts. getCachedJsonBatch uses prefixKey() which adds env prefix, but seed data is written without prefix. Using getCachedJson with raw=true bypasses the prefix. * fix(trade): prevent circuit breaker from caching empty results (#1685) All 5 trade fetch functions now pass shouldCache to skip caching when the RPC returns empty data. Previously, empty/unavailable responses were cached for 30 min in IndexedDB, causing Flows and Barriers tabs to stay hidden across page reloads even after the seed populated Redis with real data. * fix(supply-chain): move Economic Indicators to own tab (#1686) With 7 new freight index cards (SCFI, CCFI, BDI, BCI, BPI, BSI, BHSI) on the Shipping Rates tab, the FRED Economic Indicators were pushed too far down. Split into a 4th tab for better visibility. * fix(forecast): filter low-signal panel forecasts (#1687) * refactor: dedupe map URL param parsing helpers (#1675) Extract parseEnumParam and parseClampedFloatParam helpers in urlState.ts to replace 4 inline parsing blocks for view, zoom, lat, lon, and timeRange. * feat(trade): rename Revenue tab to US Revenue, add monthly bar chart (#1689) - Renamed "Revenue" tab to "US Revenue" for clarity - Added 12-month bar chart above the table showing customs revenue trend - Bars highlighted in red when revenue exceeds 1.5x prior year average - Makes the tariff impact (308% YoY increase) visually obvious * blog: add post on trade routes, chokepoints, and freight costs (#1692) Covers the new Supply Chain panel features: chokepoint monitoring (8 corridors), freight indices (SCFI, CCFI, BDI), trade policy (WTO data), and critical minerals. Opens with the current Hormuz crisis (94.4% traffic drop) as a real-world hook. * docs: update Mintlify docs for recent features (#1693) - finance-data.mdx: Add US Treasury customs revenue to Trade Policy section (Revenue tab, free API, FYTD comparison, desktop support) - features.mdx: Expand CMD+K section from 5 categories to full 55 panel coverage with organized subcategories - data-sources.mdx: Update advisory section for gold standard migration (Railway seed, 265-entry country map, relay proxy) - architecture.mdx: Add 12 missing seed scripts to Railway pipeline table (forecasts, conflict-intel, economy, supply-chain-trade, security-advisories, usni-fleet, gdelt-intel, research, etc.) - changelog.mdx + CHANGELOG.md: Add unreleased entries for customs revenue, advisory migration, CMD+K coverage, R2 traces, and 6 fixes * fix(ui): render heatmap with neutral 0% when sector change is null (#1691) Sectors with null change (market closed / bootstrap race) now render as 0% (neutral gray) instead of showing "temporarily unavailable". Error only shows when the data array is truly empty. * fix(gateway): skip CDN cache for upstreamUnavailable responses (#1690) When an RPC handler returns { upstreamUnavailable: true }, the gateway now sets Cache-Control: no-store instead of the normal CDN cache tier. This prevents CF from caching empty/error responses for hours (e.g. trade flows returning 0 results got cached for 4h with static tier). Successful responses continue to use their configured cache tier. * fix(blog): correct heroImage path and remove duplicate inline image (#1694) heroImage used /images/blog/ but blog base is /blog/, so the correct path is /blog/images/blog/. Also removed duplicate inline ![...] image reference since the hero image already shows it. * perf(news): skip client-side AI reclassification for digest items (#1695) * perf(news): skip client-side AI reclassification for digest items The server digest already runs enrichWithAiCache() which checks the same Redis keys that classifyEvent writes to. Re-firing classifyEvent from every client wastes edge requests even on cache hits. Removes the aiCandidates/classifyWithAI loop from the digest branch. Per-feed fallback path is unchanged (it never had classification). Saves up to ~48 classify-event edge requests per page load. * test(news): add regression test for digest branch no-reclassification guard 6 structural assertions ensuring digest-backed items never trigger client-side classifyWithAI calls, and that the imports are removed. Prevents accidental reintroduction of redundant edge requests. * fix(forecast): seed military surge signals (#1696) * fix(forecast): filter low-signal panel forecasts * fix(forecast): seed military surge signals * perf: gate non-critical startup loads and refresh intervals by viewport (#1697) * perf: gate non-critical startup loads and refresh intervals by viewport Scheduler (App.ts): - firms: gate by satellite-fires panel viewport or natural layer - intelligence: gate by cii/strategic-risk/strategic-posture viewport or country brief open - temporalBaseline: gate by same intelligence consumers - correlation-engine: gate by correlation panel viewport - strategic-posture: viewport check instead of panel-exists - strategic-risk: viewport check instead of panel-exists Startup (data-loader.ts): - giving: only load when panel near viewport - firms: only load when satellite-fires panel near viewport or natural layer active - intelligence: only load when cii/strategic panels near viewport - iranAttacks: only load when intelligence consumers near viewport Adds 3 centralized helpers: shouldRefreshIntelligence(), shouldRefreshFirms(), shouldRefreshCorrelation(). * fix(perf): add iranAttacks map layer guard, remove redundant natural check P2: iranAttacks startup now also loads when mapLayers.iranAttacks is enabled, preventing missing map pins on first load when the layer is on but strategic panels are off-screen. P3: Remove mapLayers.natural from shouldRefreshFirms() since natural events have their own independent scheduler entry. * fix(seeds): extend existing cache TTL on validation failure (#1705) When a seed fetches data but validation rejects it (e.g. FIRMS API returns 0 fires due to timeout), extend the existing key's TTL instead of letting it expire. Old data survives until the next successful fetch. Applies to all seeds using runSeed(). * refactor: dedupe upstash json reads across edge endpoints (#1708) * fix: unblock geolocation and fix stale CSP hash (#1709) * fix: unblock geolocation and fix stale CSP hash for SW nuke script Permissions-Policy had geolocation=() which blocked navigator.geolocation used by user-location.ts. Changed to geolocation=(self). CSP script-src had a stale SHA-256 hash (903UI9my...) that didn't match the current SW nuke script content. The script was silently blocked in production, preventing recovery from stale service workers after deploys. Replaced with the correct hash (4Z2xtr1B...) in both vercel.json and index.html meta tag. * test: update permissions-policy test for geolocation=(self) Move geolocation from "disabled" list to "delegated" assertions since it now allows self-origin access for user-location.ts. * fix(forecast): bundle military surge inputs (#1706) * chore: clear baseline lint debt (173 warnings → 49) (#1712) Mechanical fixes across 13 files: - isNaN() → Number.isNaN() (all values already numeric from parseFloat/parseInt) - let → const where never reassigned - Math.pow() → ** operator - Unnecessary continue in for loop - Useless string escape in test description - Missing parseInt radix parameter - Remove unused private class member (write-only counter) - Prefix unused function parameter with _ Config: suppress noImportantStyles (CSS !important is intentional) and useLiteralKeys (bracket notation used for computed/dynamic keys) in biome.json. Remaining 49 warnings are all noExcessiveCognitiveComplexity (already configured as warn, safe to address incrementally). * refactor(enrichment): dedupe domain/org normalization helpers (#1707) * refactor: dedupe github latest release fetch wiring (#1704) * fix(military): tighten flight classification (#1713) * fix(forecast): bundle military surge inputs * fix(military): tighten flight classification * refactor: dedupe edge api json response assembly (#1702) * refactor: dedupe edge api json response assembly * refactor: expand jsonResponse helper to all edge functions Roll out jsonResponse() from _json-response.js to 16 files (14 handlers + 2 shared helpers), eliminating 55 instances of the new Response(JSON.stringify(...)) boilerplate. Only exception: health.js uses JSON.stringify(body, null, indent) for pretty-print mode, which is incompatible with the helper signature. Replaced local jsonResponse/json() definitions in contact.js, register-interest.js, and cache-purge.js with the shared import. * fix(seeds): skip transient redis lock timeouts (#1714) * fix(seeds): skip transient redis lock timeouts * docs(seeds): clarify transient redis error matching * test: expand transient redis error coverage Add tests for ECONNRESET, DNS failure (EAI_AGAIN), ETIMEDOUT, and negative cases (HTTP 403, payload size) to confirm isTransientRedisError only matches network-level failures, not app-level Redis errors. * refactor(economic): dedupe FRED series fetch/parse helper (#1715) * fix(ucdp): page error logging, page-0 fallback, TTL extension (#1717) * fix(ucdp): add page error logging, page-0 fallback, and TTL extension on empty Three resilience improvements for UCDP seed loop: 1. Log actual error messages on page fetch failures instead of silently swallowing them. Enables diagnosing API outages vs rate limits. 2. Fall back to page 0 data when all newest-page fetches fail. Page 0 is already fetched during version discovery, so this is free. Provides partial (older) data instead of writing 0 events. 3. When 0 events remain after processing, extend existing Redis key TTL instead of overwriting with empty payload. Preserves stale-but-valid data for the next cycle rather than causing EMPTY_DATA CRIT in health. * fix(ucdp): remove page-0 fallback, stop seed-meta on failed fetches P1 fixes from review: - Remove page-0 fallback that overwrote last known good cache with stale historical data. Extend existing key TTL instead. - Stop writing fresh seed-meta timestamps when no new payload is written (both all-pages-failed and empty-after-filtering branches). Health checks should reflect actual data freshness, not failed attempts. Add 6 targeted source-analysis tests verifying: - Error logging on page failures - No page-0 data injection - TTL extension on failure branches - seed-meta only written on successful publish * fix(military): improve source-backed flight inference (#1716) * fix(military): improve source-backed flight inference * fix(military): tighten operator metadata matching * fix(military): tighten source hint inference * fix(blog): apply SEO and content improvements to supply chain post (#1728) - Shorten meta description to 155 chars (was truncating in SERPs) - Shorten metaTitle to under 60 chars - Add Key Takeaways box for skimmers and AI citation - Move brand name into first paragraph - Add "Data as of" timestamp note - Rename H2 with target keyword "maritime chokepoints" - Add 3 internal links to related blog posts - Add FAQ section (BDI, Hormuz oil impact, critical chokepoints) - Strengthen CTA with specific panel actions * feat(military): add enrichment audit waterfall (#1730) * fix(military): improve source-backed flight inference * fix(military): tighten operator metadata matching * fix(military): tighten source hint inference * feat(military): add enrichment audit waterfall * feat(military): capture live source shape gaps * fix(blog): SEO improvements across all 16 blog posts (#1729) - Trim metaTitles to under 58 chars (were 63-96, all truncated in SERPs) - Adjust meta descriptions to 150-160 char range - Add 2-3 internal blog post links per article for cross-linking - Add FAQ sections (2-3 questions each) for long-tail keyword capture - No content changes to article body text * fix(sidecar): block cloud fallback in Docker mode (#1726) Self-hosted Docker instances must not proxy unhandled routes to api.worldmonitor.app. When LOCAL_API_MODE=docker, cloudFallback is forced to false regardless of LOCAL_API_CLOUD_FALLBACK env var. Logs a warning if the user explicitly requested fallback. Prevents self-hosted users from unknowingly sending traffic to the production Vercel deployment. * fix(military): use opensky oauth in seed (#1738) * fix(military): use opensky oauth in seed * feat(military): log opensky source usage * fix(seeds): extend seed-meta TTL alongside data keys on fetch failure (#1724) When upstream APIs fail and seeds extend existing data key TTLs, the seed-meta key was left untouched. Health checks use seed-meta fetchedAt to determine staleness, so preserved data still triggered STALE_SEED warnings even though the data was valid. Now all TTL extension paths include the corresponding seed-meta key: - _seed-utils.mjs runSeed() (fetch failure + validation skip) - fetch-gpsjam.mjs (Wingbits 500 fallback) - seed-airport-delays.mjs (FAA fetch failure) - seed-military-flights.mjs (OpenSky fetch failure) - seed-service-statuses.mjs (RPC fetch failure) * feat(economic): add macro stress signals (clean cherry-pick) (#1719) Cherry-pick economic changes from codex/signal-architecture-phase1 onto clean main. Removed gitignored docs/internal/ file and 23 unrelated files from scope drift. Changes: - Add HY spread (BAMLH0A0HYM2), jobless claims (ICSA), 30Y mortgage (MORTGAGE30US), and GSCPI to ALLOWED_SERIES - Fix display math: toDisplayValue scales units correctly for WALCL (trillions) and basis-point series - Bump batch limit from 10 to 20 series - Preserve Redis-read-only pattern (no live FRED fetch on Vercel) * perf(sw): throttled event-driven SW updates (#1718) * perf(sw): replace 5-min polling with throttled event-driven updates Replace aggressive 5-minute registration.update() loop with throttled event-driven checks: on tab visibility, online reconnect, and an hourly visible-only fallback. Cross-tab throttling via shared localStorage timestamp prevents multiple tabs from checking simultaneously. Reduces /sw.js request volume from ~12/hr/tab to ~1/hr across all tabs while maintaining fast deploy pickup on tab return or reconnect. * fix(sw): stateful first-visit guard + failure-aware throttle Two fixes from review: 1. controllerchange guard: hadController was const, so first-visit tabs never reloaded on later deploys. Now let + set true after skipping the initial control-taking event. 2. Throttle writes timestamp after successful check (not before). On failure, retries every 5min instead of waiting the full 1hr cooldown. Uses separate localStorage keys for last-check vs last-success. * fix(sw): 15min poll interval + warn on update check failure - Change setInterval from 5min to 15min to reduce unnecessary ticks (throttle still gates to 1hr on success, 5min on failure) - Log update check failures with console.warn instead of silently swallowing them * fix(military): prefer direct opensky over proxy (#1742) * fix(forecast): improve ranking and enrichment coverage (#1745) * feat(forecast): add trace quality summary (#1746) * feat(forecast): add trace quality summary * fix(forecast): split traced and full-run quality metrics * refactor(military): dedupe Wingbits aircraft-details fetch helper (#1737) * refactor(military): dedupe Wingbits aircraft-details fetch helper * fix(military): correct cached details type in batch endpoint * refactor(api): dedupe in-memory IP rate limiter (#1740) * feat: add Radiation Watch with seeded anomaly intelligence, map layers, and country exposure (#1735) * feat(widgets): AI widget builder with live WorldMonitor data (#1732) * fix(docs): exclude /docs from CSP that blocks Mintlify (#1750) * fix(docs): exclude /docs from CSP header that blocks Mintlify scripts The catch-all /(.*) header rule applied Content-Security-Policy with SHA-based script-src to all routes including /docs/*. Mintlify generates dozens of inline scripts that don't match those hashes, causing 71 CSP errors and a completely blank docs page. Fix: change catch-all to /((?!docs).*) so /docs paths inherit only their own lightweight headers (nosniff, HSTS, referrer-policy). * fix(tests): update deploy-config test for docs CSP exclusion Test was looking for exact source '/(.*)', updated to match the new '/((?!docs).*)' pattern that excludes /docs from the strict CSP. * feat(sanctions): add OFAC sanctions pressure intelligence (#1739) * feat(sanctions): add OFAC sanctions pressure intelligence * fix(sanctions): strip _state from API response, fix code/name alignment, cap seed limit - trimResponse now destructures _state before spreading to prevent seed internals leaking to API clients during the atomicPublish→afterPublish window - buildLocationMap and extractPartyCountries now sort (code, name) as aligned pairs instead of calling uniqueSorted independently on each array; fixes code↔name mispairing for OFAC-specific codes like XC (Crimea) where alphabetic order of codes and names diverges - DEFAULT_RECENT_LIMIT reduced from 120 to 60 to match MAX_ITEMS_LIMIT so seeded entries beyond the handler cap are not written unnecessarily - Add tests/sanctions-pressure.test.mjs covering all three invariants * fix(sanctions): register sanctions:pressure:v1 in health.js BOOTSTRAP_KEYS and SEED_META Adds sanctionsPressure to health.js so the health endpoint monitors the seeded key for emptiness (CRIT) and freshness via seed-meta:sanctions:pressure (maxStaleMin: 720 matches 12h seed TTL). Without this, health was blind to stale or missing sanctions data. * feat(llm): support forecast model overrides (#1751) * refactor(sanctions): simplify handler to Redis-read-only, fix seed OOM risk (#1753) * refactor(sanctions): simplify handler to Redis-read-only, fix seed OOM risk Handler (424→56 lines): - Remove live OFAC fetch fallback from Vercel Edge handler: XMLParser, OFAC_SOURCES, fetchSource, collectPressure, cachedFetchJson fallback. Vercel reads Redis only; Railway makes all external API calls. - On seed miss/empty, return emptyResponse() matching the radiation pattern. Seed: - Fetch SDN then Consolidated sequentially instead of Promise.all. Combined parallel parse peaks at ~150MB, tight against 512MB heap limit. Tests: - Add gold standard compliance assertions (no XMLParser, no OFAC_SOURCES). - Add memory safety assertion (no Promise.all on OFAC sources). - Replace handler XML-function tests (removed code) with Redis-read assertions. * chore: exclude DMCA-TAKEDOWN-NOTICE.md from markdownlint * fix(csp): add missing Vercel inline script hash (#1756) Add sha256-903UI9my1I7mqHoiVeZSc56yd50YoRJTB2269QqL76w= to script-src to allow Vercel-injected inline script that was being blocked by CSP. * fix(sanctions): add fast-xml-parser to Railway scripts deps (#1755) seed-sanctions-pressure.mjs imports fast-xml-parser to parse OFAC SDN XML feeds, but the package was never added to scripts/package.json. Railway deploys crash with ERR_MODULE_NOT_FOUND on startup. * fix(widget): add @anthropic-ai/sdk to root deps for Railway (#1757) The relay dynamically imports @anthropic-ai/sdk but it was only in scripts/package.json. Railway Nixpacks only installs root deps, so the package was missing at runtime (ERR_MODULE_NOT_FOUND). * chore(forecast): log llm stage routing (#1754) * fix(sanctions): add progress logging to seed (fetch size, entry count, new count) (#1758) * fix(gdelt): make RPC handler Redis-read-only (gold standard) (#1759) * fix(relay): add root nixpacks.toml to install scripts/ deps on Railway (#1760) Railway builds from repo root (root_dir=""), so scripts/nixpacks.toml is skipped. Root packages are installed via npm ci but scripts/package.json deps (@anthropic-ai/sdk, fast-xml-parser, etc.) never land in /app/node_modules, causing ERR_MODULE_NOT_FOUND at runtime. Add a root nixpacks.toml that: - Preserves existing scripts/nixpacks.toml settings (curl, NODE_OPTIONS) - Adds `npm install --prefix scripts` to the build phase so all scripts/package.json deps are installed alongside root deps * fix(forecast): tighten quality and enrichment balance (#1761) * fix: enforce gold standard on UCDP and theater posture handlers (#1763) Make list-ucdp-events.ts Redis-read-only (was calling ucdpapi.pcr.uu.se directly from Vercel Edge). Reduces 160 lines to 28. Make get-theater-posture.ts Redis-read-only (was calling OpenSky + Wingbits directly from Vercel Edge). Reduces 273 lines to 38. Keeps live/stale/backup Redis fallback chain. Update theater posture tests to verify read-only behavior instead of testing upstream coalescing and cache writes. All UCDP/theater data assembly now exclusively on Railway (ais-relay.cjs + seed-military-flights.mjs). * test(sanctions): add 74 unit tests for seed parsing/transformation functions (#1762) Tests cover all pure helper functions in scripts/seed-sanctions-pressure.mjs: listify, textValue, buildEpoch, uniqueSorted, compactNote, extractDocumentedName, normalizeDateOfIssue, buildReferenceMaps, buildLocationMap, resolveEntityType, extractPartyName, extractPrograms, extractEffectiveAt, extractNote, sortEntries, buildCountryPressure, buildProgramPressure, and an end-to-end buildEntriesForDocument fixture. Uses node:vm to load pure functions in an isolated context, bypassing the loadEnvFile + runSeed side effects that fire on module import. * Add thermal escalation seeded service (#1747) * feat(thermal): add thermal escalation seeded service Cherry-picked from codex/thermal-escalation-phase1 and retargeted to main. Includes thermal escalation seed script, RPC handler, proto definitions, bootstrap/health/seed-health wiring, gateway cache tier, client service, and tests. * fix(thermal): wire data-loader, fix typing, recalculate summary Wire fetchThermalEscalations into data-loader.ts with panel forwarding, freshness tracking, and variant gating. Fix seed-health intervalMin from 90 to 180 to match 3h TTL. Replace 8 as-any casts with typed interface. Recalculate summary counts after maxItems slice. * fix(thermal): enforce maxItems on hydrated data + fix bootstrap keys Codex P2: hydration branch now slices clusters to maxItems before mapping, matching the RPC fallback behavior. Also add thermalEscalation to bootstrap.js BOOTSTRAP_CACHE_KEYS and SLOW_KEYS (was lost during conflict resolution). * fix(thermal): recalculate summary on sliced hydrated clusters When maxItems truncates the cluster array from bootstrap hydration, the summary was still using the original full-set counts. Now recalculates clusterCount, elevatedCount, spikeCount, etc. on the sliced array, matching the handler's behavior. * fix(forecast): address P2/P3 code review findings from PR #1761 (#1765) - Strip enrichmentMeta from bootstrap.js forecasts payload (seed-internal, not for clients) - Rename quietDomainBonus -> priorityDomainBonus (it applies to priority domains, not quiet ones) - Extract cyber score formula magic numbers into named constants (CYBER_SCORE_TYPE_MULTIPLIER, etc.) - Pre-compute analysisPriority in rankForecastsForAnalysis to avoid double-call per comparison - Log when filterPublishedForecasts weak-fallback gate suppresses forecasts - Log how many fallback narratives populateFallbackNarratives applies - Add // penalties comment header in computeAnalysisPriority * feat(ui): separate macro stress and energy complex panels (#1749) * feat(ui): separate macro stress and energy complex panels Cherry-picked from chained branch onto clean main. Splits the Economic panel into focused Macro Stress (FRED indicators) and Energy Complex (oil analytics + market tape) panels. Also fixes negative dollar-B changes now display with minus sign (was rendering as positive due to Math.abs without sign prefix). * fix: restore BIS/spending loading, split commodity/energy flags Addresses 2 blockers from PR review: 1. Restore BIS and USASpending data loading in all 3 call sites (prime tasks, periodic refresh, initial load). Restore full spending and centralBanks tab rendering in EconomicPanel while keeping the new macro stress indicators view. 2. Split shared commoditiesLoaded flag into separate metalsLoaded and energyLoaded flags so each panel falls back independently. Energy data arriving cannot suppress metals fallback fetch. EconomicPanel now has 3 tabs: Indicators (macro stress), Spending, Central Banks. Oil tab removed (lives in EnergyComplexPanel). * fix(i18n): restore economic panel translation keys for BIS/spending tabs PR removed i18n keys needed by the restored spending and centralBanks tabs (gov, centralBanks, awards, policyRate, etc.). Also updated the infoTooltip to accurately describe the mixed surface (macro + gov + central banks) instead of claiming macro-only. * fix(seeds): add empty-data guards and fix health semantics (#1767) Health semantics: - Add faaDelays + gpsjam to EMPTY_DATA_OK_KEYS (0 records = calm, not error) - Fix EMPTY_DATA_OK_KEYS branch to still check seed-meta freshness (prevents stale empty caches from staying green indefinitely) Seed guards: - seed-airport-delays: fix meta key in fetch-failure path (seed-meta:aviation:delays -> seed-meta:aviation:faa + seed-meta:aviation:notam) - seed-military-flights: add full TTL extension on zero-flights branch (was exiting without preserving any derived data TTLs) - seed-wb-indicators: add percentage-drop guard (new count < 50% of cached = likely partial API failure, extend TTL instead of overwriting) - ais-relay.cjs: same percentage-drop guard for WB dual writer Codex-reviewed plan (5 rounds, approved). * Add 7 new tier-1 countries to risk scoring and monitoring (#1768) * fix: include all curated countries in CII scoring (LB, IQ, AF, KR, EG, JP, QA) The CII was only scoring 24 "tier-1" countries while 7 curated countries (Lebanon, Iraq, Afghanistan, South Korea, Egypt, Japan, Qatar) had full configurations but were excluded from instability scoring. This expands TIER1_COUNTRIES to all 31 curated countries so every monitored country gets proper CII scores with baselines, keywords, bounding boxes, zone mappings, and travel advisory fallbacks. https://claude.ai/code/sessio…
…t group (koala73#1596) - Remove 403-blocked feeds: Breaking Defense, My Modern Met, AEI - Fix Infobae URL: /feeds/rss/ -> /arc/outboundfeeds/rss/ (both files) - Fix CSIS URL: /feed -> /rss.xml (server _feeds.ts) - Drop 'active' from CelesTrak SAT_GROUPS (>2MB, always rejected)
…oala73#1626) PR koala73#1596 removed the feeds but left the domains in the allowlist. The relay still accepted proxy requests for these 403-blocked domains from clients with cached old bundles. Removed: - breakingdefense.com (403) - www.arabnews.com (403) - www.aei.org (403) - mymodernmet.com (403) Updated all 3 copies: shared/, scripts/shared/, api/
Summary
/feeds/rss/->/arc/outboundfeeds/rss/) in both feed config files/feed->/rss.xml) in server_feeds.tsactivefrom CelesTrak satellite groups (>2MB payload, always rejected by 2MB guard)Context
From relay log analysis: 13 RSS feeds returning 403/404, OpenSky proxy down, and CelesTrak
activegroup always exceeding the 2MB limit. Several ghost feeds (old URLs from a stale deployment) will also stop appearing after this deploys.Test plan