Skip to content

Add LOD canvas markers for dense map overlays#23

Closed
koala73 wants to merge 1 commit into
mainfrom
codex/implement-webgl-clustering-for-markers
Closed

Add LOD canvas markers for dense map overlays#23
koala73 wants to merge 1 commit into
mainfrom
codex/implement-webgl-clustering-for-markers

Conversation

@koala73

@koala73 koala73 commented Jan 15, 2026

Copy link
Copy Markdown
Owner

Motivation

  • The map currently renders many markers as HTML which can cause DOM bloat and layout overhead at low zoom/high density.
  • Use a level-of-detail (LOD) approach to render large numbers of low-detail markers on a GPU/Canvas layer while preserving HTML markers for high-fidelity items.
  • Improve runtime performance and responsiveness when many natural/weather/outage/protest/flight/AIS points are visible.

Description

  • Introduces LOD thresholds and density heuristics (CLUSTER_LOD_ZOOM, CLUSTER_LOD_MAX_ZOOM, CLUSTER_DENSITY_THRESHOLD, etc.) and state fields (clusterCtx, clusterMode) to MapComponent.
  • Adds a 2D canvas rendering pass with renderClusterCanvas, resizeClusterCanvas, shouldUseClusterCanvas, and getClusterDensityCount, and updates clearClusterCanvas to clear the 2D context when present.
  • Switches overlay rendering to selective HTML + canvas LOD: renderOverlays now uses clusterMode to render only high-severity or high-impact HTML markers while the canvas draws dense dots for natural events, weather alerts, outages, protests, flight delays, and AIS disruptions; renderAisDisruptions signature updated to accept a filtered list.
  • Canvas drawing implements lightweight dot rendering and per-layer coloring/sizing to preserve visual cues while massively reducing DOM element count at low zoom/high density.

Testing

  • Launched the dev server with npm run dev and confirmed Vite served the app (local network URL printed).
  • Captured a functional smoke screenshot by running a Playwright script against http://127.0.0.1:4173/, which produced artifacts/map-lod.png demonstrating the LOD canvas pass.
  • Note: no automated unit tests were executed; the dev server logs showed unrelated network/proxy errors but the page loaded sufficiently for the screenshot.

Codex Task

@vercel

vercel Bot commented Jan 15, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
worldmonitor Ready Ready Preview, Comment Jan 15, 2026 9:52am

@koala73 koala73 closed this Jan 23, 2026
@koala73
koala73 deleted the codex/implement-webgl-clustering-for-markers branch January 26, 2026 03:39
@SebastienMelki SebastienMelki added area: map Map, globe, DeckGL visualization performance Performance optimization labels Feb 17, 2026
EleCor79 added a commit to EleCor79/BehavioralHealthPulse that referenced this pull request Mar 2, 2026
Distribute all 32 feeds from feeds-health-italy-eu.csv into the FEEDS
config consumed by loadNews() / fetchCategoryFeeds():

- ministero-salute: +MinSalute News Specifiche (CSV koala73#18)
- iss-epicentro:    +ISS Notizie (CSV koala73#3), +Epicentro Coronavirus (CSV koala73#17)
- aifa-tracker:     +AIFA Feed (CSV koala73#5), +EMA News (CSV koala73#8), +FDA (CSV koala73#12)
- agenas-ospedali:  +AGENAS RSS (CSV koala73#4), +PNRR (CSV koala73#6/koala73#19),
                    +Lombardia (CSV koala73#20), +Lazio (CSV koala73#21)
- ema-europa:       +EMA Clinical Trials (CSV koala73#22)
- ecdc-sorveglianza:+ECDC Weekly Threats (CSV koala73#23), +WHO DON (CSV koala73#9),
                    +ProMED (CSV koala73#10)
- live-news:        +Sanitainformazione (CSV koala73#24), +Humanitas (CSV koala73#26)
- europe:           +EU Core Health Indicators (CSV koala73#31),
                    +GLOBSEC HRI (CSV koala73#32), +ISTAT (CSV koala73#13),
                    +EIN Health Europe (CSV koala73#29)
- rare-diseases:    NEW category — EURORDIS (CSV koala73#14), Orphanet IT (CSV koala73#15),
                    Telethon (CSV koala73#16), CDC FluView (CSV koala73#11)
                    Panel disabled by default, available in settings

Co-Authored-By: Claude Opus 4.6 <[email protected]>
koala73 added a commit that referenced this pull request Mar 24, 2026
… runId mismatch

- Add `error` field to GetSimulationPackageResponse: populated with
  "redis_unavailable" on Redis errors so callers can distinguish a
  healthy not-found (found=false, error="") from a Redis failure
  (found=false, error="redis_unavailable"). Adds console.warn on error.
- Add `note` field: populated when req.runId is supplied but does not
  match the latest package's runId, signalling that per-run filtering
  is not yet active (Phase 3).
- Add proto comment on run_id: "Currently ignored; reserved for Phase 3"
- Add milliseconds annotation to generated_at description.
- Simplify handler: extract NOT_FOUND constant, remove SimulationPackagePointer
  interface, remove || '' / || 0 guards on guaranteed-present fields.
- Regenerate all buf-generated files.

Fixes todos #18 (runId silently ignored) and #19 (error indistinguishable
from not-found). Also resolves todos #22 (simplifications) and #23
(OpenAPI required fields / generatedAt unit annotation).
koala73 added a commit that referenced this pull request Mar 24, 2026
* feat(forecast): Phase 2 simulation package read path (getSimulationPackage RPC + Redis existence key)

- writeSimulationPackage now writes forecast:simulation-package:latest to Redis after
  successful R2 write, containing { runId, pkgKey, schemaVersion, theaterCount, generatedAt }
  with TTL matching TRACE_REDIS_TTL_SECONDS (60 days)
- New getSimulationPackage RPC handler reads Redis key, returns pointer metadata without
  requiring an R2 fetch (zero R2 cost for existence check)
- Wired into ForecastServiceHandler and server/gateway.ts cache tier (medium)
- Proto: GetSimulationPackage RPC + get_simulation_package.proto message definitions
- api/health.js: simulationPackageLatest added to STANDALONE_KEYS + ON_DEMAND_KEYS
- Tests: SIMULATION_PACKAGE_LATEST_KEY constant + writeSimulationPackage null-guard test

Closes todo #17 (Phase 2 prerequisites for MiroFish integration)

* chore(generated): regenerate proto types for GetSimulationPackage RPC

* fix(simulation-rpc): distinguish Redis failure from not-found; signal runId mismatch

- Add `error` field to GetSimulationPackageResponse: populated with
  "redis_unavailable" on Redis errors so callers can distinguish a
  healthy not-found (found=false, error="") from a Redis failure
  (found=false, error="redis_unavailable"). Adds console.warn on error.
- Add `note` field: populated when req.runId is supplied but does not
  match the latest package's runId, signalling that per-run filtering
  is not yet active (Phase 3).
- Add proto comment on run_id: "Currently ignored; reserved for Phase 3"
- Add milliseconds annotation to generated_at description.
- Simplify handler: extract NOT_FOUND constant, remove SimulationPackagePointer
  interface, remove || '' / || 0 guards on guaranteed-present fields.
- Regenerate all buf-generated files.

Fixes todos #18 (runId silently ignored) and #19 (error indistinguishable
from not-found). Also resolves todos #22 (simplifications) and #23
(OpenAPI required fields / generatedAt unit annotation).

* fix(simulation-rpc): change cache tier from medium to slow (aligns with deep-run update frequency)

* fix(simulation-rpc): fix key prefixing, make Redis errors reachable, no-cache not-found

Three P1 regressions caught in external review:

1. Key prefix bug: getCachedJson() applies preview:<sha>: prefix in non-production
   environments, but writeSimulationPackage writes the raw key via a direct Redis
   command. In preview/dev the RPC always returned found:false even when the package
   existed. Fix: new getRawJson() in redis.ts always uses the unprefixed key AND throws
   on failure instead of swallowing errors.

2. redis_unavailable unreachable: getCachedJson swallows fetch failures and missing-
   credentials by returning null, so the catch block for redis_unavailable was dead
   code. getRawJson() throws on HTTP errors and missing credentials, making the
   error: "redis_unavailable" contract actually reachable.

3. Negative-cache stampede: slow tier caches every 200 GET. A request before any deep
   run wrote a package returned { found:false } which the CDN cached for up to 1h,
   breaking post-run discovery. Fix: markNoCacheResponse() on both not-found and
   error paths so they are served fresh on every request.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: map Map, globe, DeckGL visualization codex performance Performance optimization

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants