Skip to content

fix: infra cost optimizations round 2 — polling, TTLs, ACLED dedup#275

Merged
koala73 merged 6 commits into
koala73:mainfrom
elzalem:fix/infra-cost-optimizations-round2
Feb 24, 2026
Merged

fix: infra cost optimizations round 2 — polling, TTLs, ACLED dedup#275
koala73 merged 6 commits into
koala73:mainfrom
elzalem:fix/infra-cost-optimizations-round2

Conversation

@elzalem

@elzalem elzalem commented Feb 23, 2026

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #273. Three targeted optimizations to reduce unnecessary API calls, Redis reads, and upstream bandwidth:

  • Skip AIS polling on backgrounded tabs — adds document.hidden guard + visibilitychange pause/resume so Railway relay gets zero traffic from inactive tabs
  • Align cache TTLs with upstream refresh rates — climate 30min→3h (ERA5 has 2–7 day lag), fire 30min→1h (FIRMS NRT updates ~every 3h). Sources verified against upstream docs
  • Deduplicate ACLED API calls — extract shared fetchAcledCached() so conflict, unrest, and intelligence handlers share one Redis-backed fetch instead of three independent upstream calls

Estimated savings (at 500K monthly users)

Optimization Before After Reduction
AIS relay calls (backgrounded tabs, ~30% of sessions) ~130K/day ~91K/day ~30%
Climate Redis reads 48/day 8/day 83%
Fire Redis reads 48/day 24/day 50%
ACLED upstream calls 3× per cache miss 1× per cache miss up to 66%

TTL verification sources

Test plan

  • Verify AIS overlay loads on first tab visit
  • Background a tab for >30s, confirm no Railway relay calls in Network tab
  • Refocus tab, verify AIS data resumes within one poll interval
  • Verify climate anomalies panel still loads correctly
  • Verify fire detections panel still loads correctly
  • Verify conflict/unrest/risk-scores panels all return data
  • Check Redis keys: acled:shared:* entries appear after first ACLED fetch

🤖 Generated with Claude Code

elzalem and others added 3 commits February 23, 2026 22:00
Add two layers of defence to avoid wasting Railway relay bandwidth on
hidden tabs:

1. document.hidden guard in pollSnapshot() — skips fetch when tab is
   not visible (interval keeps ticking so first tick after focus polls
   immediately)
2. visibilitychange listener — pauses/resumes the interval timer
   entirely so no callbacks fire while backgrounded

Co-Authored-By: Claude Opus 4.6 <[email protected]>
Climate anomalies: 30min → 3h
  Source: Open-Meteo Archive API uses ERA5 reanalysis data with a
  2–7 day lag (https://open-meteo.com/en/docs/historical-weather-api).
  30-min polling gains nothing against week-old data.

Fire detections: 30min → 1h
  Source: NASA FIRMS VIIRS_SNPP_NRT refreshes approximately every
  3 hours (https://firms.modaps.eosdis.nasa.gov/usfs/active_fire/).
  Polling twice per refresh cycle still catches every update.

Combined: ~66% fewer Redis reads on these two endpoints.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
Three RPC handlers (conflict, unrest, intelligence) each made
independent ACLED API calls with their own auth/timeout/parsing logic.
Extract a shared fetchAcledCached() in server/_shared/acled.ts that:

- Centralises auth, timeout, and error handling
- Caches by query params so overlapping date ranges share results
- Eliminates ~120 lines of duplicated fetch boilerplate
- Reduces ACLED API calls when multiple handlers run concurrently

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@vercel

vercel Bot commented Feb 23, 2026

Copy link
Copy Markdown

@elzalem is attempting to deploy a commit to the Elie Team on Vercel.

A member of the Team first needs to authorize it.

elzalem and others added 3 commits February 23, 2026 23:40
17 tests covering: TTL alignment (climate 3h, fire 1h), ACLED shared
cache layer (fetchAcledCached, cache key derivation, consumer imports,
no duplicated ACLED_API_URL), and maritime AIS visibility guard
(document.hidden check, pausePolling, resumePolling, visibilitychange
listener wiring).

Co-Authored-By: Claude Opus 4.6 <[email protected]>
koala73 pushed a commit that referenced this pull request Feb 24, 2026
Reviewed all 9 changed files across 6 commits. Verified:
- ACLED shared cache layer correctly deduplicates 3 upstream API paths
- Cache TTL alignment matches upstream data source refresh rates
- AIS visibility guard has proper two-layer defense with edge case handling
- All 18 PR-specific tests pass, TypeScript compiles cleanly
- 3 minor behavioral differences documented (event type format, timeout, auth)

Verdict: APPROVE

https://claude.ai/code/session_01DSYor9uV7a18fdpSBr8SZh
@koala73

koala73 commented Feb 24, 2026

Copy link
Copy Markdown
Owner

thank you !

@koala73
koala73 merged commit d9f3933 into koala73:main Feb 24, 2026
0 of 4 checks passed
@elzalem
elzalem deleted the fix/infra-cost-optimizations-round2 branch February 24, 2026 07:29
koala73 added a commit that referenced this pull request Feb 24, 2026
## What's Changed

### Performance
- perf: defer YouTube/map init and stagger data loads to reduce blocking time (#287)

### Features
- feat: universal country detection — CII scoring for all countries (#344)
- feat: add Mexico as CII hotspot for cartel/security monitoring (#327)
- feat: add Mexico and LatAm security feeds for instability coverage (#325)
- feat: add category pills and search filter to Panels tab (#322)
- feat: consolidate settings into unified tabbed modal (#319)
- feat: add Island Times (Palau) RSS feed (#317)
- feat: add AI Flow settings popup for web-only AI provider control (#314)
- feat: optional channels with tab-based region browse UI (#295)
- feat: custom channel management (#282)
- feat: add Bild RSS feed scoped to German locale (#312)

### Bug Fixes
- fix: suppress notification sound when popup alerts are disabled
- fix: prevent entity conflation in pane summarization (#341)
- fix: add Mexico to COUNTRY_BOUNDS and COUNTRY_ALIASES (#338)
- fix: make OpenSky cache TTLs env-configurable (#333)
- fix: serialize OpenSky requests with global 429 cooldown (#332)
- fix: replace RSSHub feeds with native/Google News alternatives (#331)
- fix: OpenSky auth resilience — retry, IPv4, no negative cache on auth fail (#329)
- fix: add CARTO and OpenStreetMap attribution to map (#323)
- fix: add drag cleanup handlers and suppress click after drag-drop (#315)
- fix: replace HTML5 drag API with mouse events for WKWebView (#313)
- fix: open channel settings as inline modal instead of separate window (#311)
- fix: sync YouTube live panel mute state with native player controls (#285)
- fix: strip Ollama reasoning tokens from summaries (#299)
- fix: open external links in system browser on Tauri desktop (#297)
- fix: add User-Agent and Cloudflare 403 detection to secret validation (#296)
- fix: infra cost optimizations round 2 (#275)
- fix: enforce military bbox filtering (#284)
- fix: infrastructure cost optimizations across caching, polling, batching (#283)
- fix: circuit breaker persistent cache with safety fixes (#281)
- fix: immediately refresh stale services when tab regains focus (#277)

### Security
- Security hardening: SSRF protection, auth gating, and token generation (#343)
- Harden Railway relay auth, caching, and proxy routing (#320)
- Build/runtime hardening and dependency security updates (#286)
- fix: harden embed postMessage origin check (#302)
koala73 added a commit that referenced this pull request Feb 25, 2026
## What's Changed

### Performance
- perf: defer YouTube/map init and stagger data loads (#287)

### Features
- feat: universal country detection — CII scoring for all countries (#344)
- feat: add Mexico as CII hotspot (#327)
- feat: add Mexico and LatAm security feeds (#325)
- feat: add category pills and search filter to Panels tab (#322)
- feat: consolidate settings into unified tabbed modal (#319)
- feat: optional channels with tab-based region browse UI (#295)
- feat: custom channel management (#282)

### Bug Fixes
- fix: suppress notification sound when popup alerts are disabled
- fix: prevent entity conflation in pane summarization (#341)
- fix: add Mexico to COUNTRY_BOUNDS and COUNTRY_ALIASES (#338)
- fix: OpenSky cache TTLs, serialization, and auth resilience (#329-#333)
- fix: replace RSSHub feeds with native/Google News alternatives (#331)
- fix: replace HTML5 drag API with mouse events for WKWebView (#313)
- fix: sync YouTube mute state with native player controls (#285)
- fix: strip Ollama reasoning tokens from summaries (#299)
- fix: infra cost optimizations (#275, #283)
- fix: circuit breaker persistent cache (#281)
- fix: immediately refresh stale services on tab focus (#277)

### Security
- Security hardening: SSRF protection, auth gating, token generation (#343)
- Harden Railway relay auth, caching, and proxy routing (#320)
- Build/runtime hardening and dependency security updates (#286)
koala73 added a commit that referenced this pull request Feb 25, 2026
…346)

* fix: suppress notification sound when popup alerts are disabled

Badge playSound() was firing on new findings regardless of the
"Pop up new alerts" toggle. Gate sound on popupEnabled so both
the modal and audio respect the user preference.

* chore: bump version to 2.5.7 with changelog

## What's Changed

### Performance
- perf: defer YouTube/map init and stagger data loads (#287)

### Features
- feat: universal country detection — CII scoring for all countries (#344)
- feat: add Mexico as CII hotspot (#327)
- feat: add Mexico and LatAm security feeds (#325)
- feat: add category pills and search filter to Panels tab (#322)
- feat: consolidate settings into unified tabbed modal (#319)
- feat: optional channels with tab-based region browse UI (#295)
- feat: custom channel management (#282)

### Bug Fixes
- fix: suppress notification sound when popup alerts are disabled
- fix: prevent entity conflation in pane summarization (#341)
- fix: add Mexico to COUNTRY_BOUNDS and COUNTRY_ALIASES (#338)
- fix: OpenSky cache TTLs, serialization, and auth resilience (#329-#333)
- fix: replace RSSHub feeds with native/Google News alternatives (#331)
- fix: replace HTML5 drag API with mouse events for WKWebView (#313)
- fix: sync YouTube mute state with native player controls (#285)
- fix: strip Ollama reasoning tokens from summaries (#299)
- fix: infra cost optimizations (#275, #283)
- fix: circuit breaker persistent cache (#281)
- fix: immediately refresh stale services on tab focus (#277)

### Security
- Security hardening: SSRF protection, auth gating, token generation (#343)
- Harden Railway relay auth, caching, and proxy routing (#320)
- Build/runtime hardening and dependency security updates (#286)
andreteow pushed a commit to andreteow/worldmonitor-a47 that referenced this pull request Feb 25, 2026
…oala73#346)

* fix: suppress notification sound when popup alerts are disabled

Badge playSound() was firing on new findings regardless of the
"Pop up new alerts" toggle. Gate sound on popupEnabled so both
the modal and audio respect the user preference.

* chore: bump version to 2.5.7 with changelog

## What's Changed

### Performance
- perf: defer YouTube/map init and stagger data loads (koala73#287)

### Features
- feat: universal country detection — CII scoring for all countries (koala73#344)
- feat: add Mexico as CII hotspot (koala73#327)
- feat: add Mexico and LatAm security feeds (koala73#325)
- feat: add category pills and search filter to Panels tab (koala73#322)
- feat: consolidate settings into unified tabbed modal (koala73#319)
- feat: optional channels with tab-based region browse UI (koala73#295)
- feat: custom channel management (koala73#282)

### Bug Fixes
- fix: suppress notification sound when popup alerts are disabled
- fix: prevent entity conflation in pane summarization (koala73#341)
- fix: add Mexico to COUNTRY_BOUNDS and COUNTRY_ALIASES (koala73#338)
- fix: OpenSky cache TTLs, serialization, and auth resilience (koala73#329-koala73#333)
- fix: replace RSSHub feeds with native/Google News alternatives (koala73#331)
- fix: replace HTML5 drag API with mouse events for WKWebView (koala73#313)
- fix: sync YouTube mute state with native player controls (koala73#285)
- fix: strip Ollama reasoning tokens from summaries (koala73#299)
- fix: infra cost optimizations (koala73#275, koala73#283)
- fix: circuit breaker persistent cache (koala73#281)
- fix: immediately refresh stale services on tab focus (koala73#277)

### Security
- Security hardening: SSRF protection, auth gating, token generation (koala73#343)
- Harden Railway relay auth, caching, and proxy routing (koala73#320)
- Build/runtime hardening and dependency security updates (koala73#286)
facusturla pushed a commit to facusturla/worldmonitor that referenced this pull request Feb 27, 2026
…oala73#275)

* fix: skip AIS polling when browser tab is backgrounded

Add two layers of defence to avoid wasting Railway relay bandwidth on
hidden tabs:

1. document.hidden guard in pollSnapshot() — skips fetch when tab is
   not visible (interval keeps ticking so first tick after focus polls
   immediately)
2. visibilitychange listener — pauses/resumes the interval timer
   entirely so no callbacks fire while backgrounded

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: align cache TTLs with upstream data source refresh rates

Climate anomalies: 30min → 3h
  Source: Open-Meteo Archive API uses ERA5 reanalysis data with a
  2–7 day lag (https://open-meteo.com/en/docs/historical-weather-api).
  30-min polling gains nothing against week-old data.

Fire detections: 30min → 1h
  Source: NASA FIRMS VIIRS_SNPP_NRT refreshes approximately every
  3 hours (https://firms.modaps.eosdis.nasa.gov/usfs/active_fire/).
  Polling twice per refresh cycle still catches every update.

Combined: ~66% fewer Redis reads on these two endpoints.

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: deduplicate ACLED API calls via shared cached fetch layer

Three RPC handlers (conflict, unrest, intelligence) each made
independent ACLED API calls with their own auth/timeout/parsing logic.
Extract a shared fetchAcledCached() in server/_shared/acled.ts that:

- Centralises auth, timeout, and error handling
- Caches by query params so overlapping date ranges share results
- Eliminates ~120 lines of duplicated fetch boilerplate
- Reduces ACLED API calls when multiple handlers run concurrently

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* test: add structural tests for round 2 infra optimizations

17 tests covering: TTL alignment (climate 3h, fire 1h), ACLED shared
cache layer (fetchAcledCached, cache key derivation, consumer imports,
no duplicated ACLED_API_URL), and maritime AIS visibility guard
(document.hidden check, pausePolling, resumePolling, visibilitychange
listener wiring).

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* refactor: rename round 2 test file to ttl-acled-ais-guards

Co-Authored-By: Claude Opus 4.6 <[email protected]>

* fix: address PR koala73#275 review findings

---------

Co-authored-by: Claude Opus 4.6 <[email protected]>
Co-authored-by: Elie Habib <[email protected]>
facusturla pushed a commit to facusturla/worldmonitor that referenced this pull request Feb 27, 2026
…oala73#346)

* fix: suppress notification sound when popup alerts are disabled

Badge playSound() was firing on new findings regardless of the
"Pop up new alerts" toggle. Gate sound on popupEnabled so both
the modal and audio respect the user preference.

* chore: bump version to 2.5.7 with changelog

## What's Changed

### Performance
- perf: defer YouTube/map init and stagger data loads (koala73#287)

### Features
- feat: universal country detection — CII scoring for all countries (koala73#344)
- feat: add Mexico as CII hotspot (koala73#327)
- feat: add Mexico and LatAm security feeds (koala73#325)
- feat: add category pills and search filter to Panels tab (koala73#322)
- feat: consolidate settings into unified tabbed modal (koala73#319)
- feat: optional channels with tab-based region browse UI (koala73#295)
- feat: custom channel management (koala73#282)

### Bug Fixes
- fix: suppress notification sound when popup alerts are disabled
- fix: prevent entity conflation in pane summarization (koala73#341)
- fix: add Mexico to COUNTRY_BOUNDS and COUNTRY_ALIASES (koala73#338)
- fix: OpenSky cache TTLs, serialization, and auth resilience (koala73#329-koala73#333)
- fix: replace RSSHub feeds with native/Google News alternatives (koala73#331)
- fix: replace HTML5 drag API with mouse events for WKWebView (koala73#313)
- fix: sync YouTube mute state with native player controls (koala73#285)
- fix: strip Ollama reasoning tokens from summaries (koala73#299)
- fix: infra cost optimizations (koala73#275, koala73#283)
- fix: circuit breaker persistent cache (koala73#281)
- fix: immediately refresh stale services on tab focus (koala73#277)

### Security
- Security hardening: SSRF protection, auth gating, token generation (koala73#343)
- Harden Railway relay auth, caching, and proxy routing (koala73#320)
- Build/runtime hardening and dependency security updates (koala73#286)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants