Add shareable map state URLs and Copy Link button#3
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
koala73
added a commit
that referenced
this pull request
Jan 18, 2026
… freshness Assessment & Documentation: - Add docs/GEOPOLITICAL_ASSESSMENT.md with full platform analysis - Strategic improvement roadmap with prioritized recommendations Quick Win #1 - Data Freshness (intelligence gaps): - Add getIntelligenceGaps() and getIntelligenceGapSummary() to data-freshness.ts - Add human-readable messages explaining what analysts CAN'T see - Add hasCriticalGaps() for alert integration Quick Win #2 - Escalation Scores: - Add escalationScore (1-5), escalationTrend, escalationIndicators to Hotspot type - Update 11 major hotspots with scores (Sahel, Haiti, Horn of Africa, Moscow, Beijing, Kyiv, Taipei, Tehran, Tel Aviv, Pyongyang, Sana'a) Quick Win #3 - Signal Context ("Why It Matters"): - Add SIGNAL_CONTEXT with whyItMatters, actionableInsight, confidenceNote - Add getSignalContext() helper for all 10 signal types - Explains analytical significance of each signal type Quick Win #4 - Historical Context: - Add HistoricalContext interface with lastMajorEvent, precedentCount, cyclicalRisk fields - Add whyItMatters field to Hotspot type - Update major hotspots with historical precedents and geopolitical significance Quick Win #5 - Propaganda Risk Flags: - Add PropagandaRisk type and SourceRiskProfile interface - Add SOURCE_PROPAGANDA_RISK mapping for state media (Xinhua, TASS, RT, CGTN) - Add getSourcePropagandaRisk() and isStateAffiliatedSource() helpers - Flag medium-risk state-affiliated sources (Al Jazeera, France 24, DW, etc.)
facusturla
pushed a commit
to facusturla/worldmonitor
that referenced
this pull request
Feb 27, 2026
… freshness Assessment & Documentation: - Add docs/GEOPOLITICAL_ASSESSMENT.md with full platform analysis - Strategic improvement roadmap with prioritized recommendations Quick Win koala73#1 - Data Freshness (intelligence gaps): - Add getIntelligenceGaps() and getIntelligenceGapSummary() to data-freshness.ts - Add human-readable messages explaining what analysts CAN'T see - Add hasCriticalGaps() for alert integration Quick Win koala73#2 - Escalation Scores: - Add escalationScore (1-5), escalationTrend, escalationIndicators to Hotspot type - Update 11 major hotspots with scores (Sahel, Haiti, Horn of Africa, Moscow, Beijing, Kyiv, Taipei, Tehran, Tel Aviv, Pyongyang, Sana'a) Quick Win koala73#3 - Signal Context ("Why It Matters"): - Add SIGNAL_CONTEXT with whyItMatters, actionableInsight, confidenceNote - Add getSignalContext() helper for all 10 signal types - Explains analytical significance of each signal type Quick Win koala73#4 - Historical Context: - Add HistoricalContext interface with lastMajorEvent, precedentCount, cyclicalRisk fields - Add whyItMatters field to Hotspot type - Update major hotspots with historical precedents and geopolitical significance Quick Win koala73#5 - Propaganda Risk Flags: - Add PropagandaRisk type and SourceRiskProfile interface - Add SOURCE_PROPAGANDA_RISK mapping for state media (Xinhua, TASS, RT, CGTN) - Add getSourcePropagandaRisk() and isStateAffiliatedSource() helpers - Flag medium-risk state-affiliated sources (Al Jazeera, France 24, DW, etc.)
EleCor79
added a commit
to EleCor79/BehavioralHealthPulse
that referenced
this pull request
Mar 2, 2026
Distribute all 32 feeds from feeds-health-italy-eu.csv into the FEEDS config consumed by loadNews() / fetchCategoryFeeds(): - ministero-salute: +MinSalute News Specifiche (CSV koala73#18) - iss-epicentro: +ISS Notizie (CSV koala73#3), +Epicentro Coronavirus (CSV koala73#17) - aifa-tracker: +AIFA Feed (CSV koala73#5), +EMA News (CSV koala73#8), +FDA (CSV koala73#12) - agenas-ospedali: +AGENAS RSS (CSV koala73#4), +PNRR (CSV koala73#6/koala73#19), +Lombardia (CSV koala73#20), +Lazio (CSV koala73#21) - ema-europa: +EMA Clinical Trials (CSV koala73#22) - ecdc-sorveglianza:+ECDC Weekly Threats (CSV koala73#23), +WHO DON (CSV koala73#9), +ProMED (CSV koala73#10) - live-news: +Sanitainformazione (CSV koala73#24), +Humanitas (CSV koala73#26) - europe: +EU Core Health Indicators (CSV koala73#31), +GLOBSEC HRI (CSV koala73#32), +ISTAT (CSV koala73#13), +EIN Health Europe (CSV koala73#29) - rare-diseases: NEW category — EURORDIS (CSV koala73#14), Orphanet IT (CSV koala73#15), Telethon (CSV koala73#16), CDC FluView (CSV koala73#11) Panel disabled by default, available in settings Co-Authored-By: Claude Opus 4.6 <[email protected]>
20 tasks
This was referenced Mar 9, 2026
3 tasks
koala73
added a commit
that referenced
this pull request
Apr 12, 2026
P2 #1 — no consumer surface for GetRegionalBrief Acknowledged. The consumer is the RegionalIntelligenceBoard panel, which will call GetRegionalBrief and render a weekly brief block. This wiring is Phase 3 PR3 (UI) scope — the RPC + Redis key are the delivery mechanism, not the end surface. No code change in this commit; the RPC is ready for the panel to consume. P2 #2 — readRecentTransitions collapses failure to [] readRecentTransitions returned [] on Redis/network failure, which is indistinguishable from a genuinely quiet week. The LLM then generates a brief claiming "no regime transitions" when in reality the upstream is down — fabricating false input. Fix: return null on failure. The seeder skips the region with a clear log message when transitions is null, so the brief is never written with unreliable input. Empty array [] now only means genuinely no transitions in the 7-day window. P2 #3 — parseBriefJson accepts briefs the seeder rejects parseBriefJson treated non-empty key_developments as valid even if situation_recap was empty. The seeder gate only writes when brief.situation_recap is truthy. That mismatch means the validator pass + provider-fallback logic could accept a response that the seeder then silently drops. Fix: require situation_recap in parseBriefJson for valid=true, matching the seeder gate. Now both checks agree on what constitutes a usable brief, and the provider-fallback chain correctly falls through when a provider returns a brief with developments but no recap.
koala73
added a commit
that referenced
this pull request
Apr 12, 2026
…ptile P1+P2 on #2989) P1 — TTL silently not applied (briefs never expire) Upstash REST ignores query-string SET options (?EX=N). The correct form is path-segment: /set/{key}/{value}/EX/{seconds}. Without this fix every brief persists indefinitely and Redis storage grows unboundedly across weekly runs. P2 — seed-meta not written when all regions skipped writeExtraKeyWithMeta was gated on generated > 0. If every region was skipped (no snapshot yet, or LLM failed), seed-meta was never written, making the seeder indistinguishable from "never ran" in health tooling. Now writes seed-meta whenever failed === 0, carrying regionsSkipped count. P2 #3 (validate gate) — already fixed in previous commit (parseBriefJson now requires situation_recap for valid=true).
koala73
added a commit
that referenced
this pull request
Apr 12, 2026
* feat(intelligence): weekly regional briefs (Phase 3 PR2) Phase 3 PR2 of the Regional Intelligence Model. Adds LLM-powered weekly intelligence briefs per region, completing the core feature set. ## New seeder: scripts/seed-regional-briefs.mjs Standalone weekly cron script (not part of the 6h derived-signals bundle). For each non-global region: 1. Read the latest snapshot via two-hop Redis read 2. Read recent regime transitions from the history log (#2981) 3. Call the LLM once per region with regime trajectory + balance + triggers + narrative context 4. Write structured brief to intelligence:regional-briefs:v1:weekly:{region} with 8-day TTL (survives one missed weekly run) Reuses the same injectable-callLlm + parse-validation + provider-chain pattern from narrative.mjs and weekly-brief.mjs. ## New module: scripts/regional-snapshot/weekly-brief.mjs generateWeeklyBrief(region, snapshot, transitions, opts?) -> { region_id, generated_at, period_start, period_end, situation_recap, regime_trajectory, key_developments[], risk_outlook, provider, model } buildBriefPrompt() — pure prompt builder parseBriefJson() — JSON parser with prose-extraction fallback emptyBrief() — canonical empty shape Global region is skipped. Provider chain: Groq -> OpenRouter. Validate callback ensures only parseable responses pass (narrative.mjs PR #2960 review fix pattern). ## Proto + RPC: GetRegionalBrief proto/worldmonitor/intelligence/v1/get_regional_brief.proto - GetRegionalBriefRequest { region_id } - GetRegionalBriefResponse { brief: RegionalBrief } - RegionalBrief { region_id, generated_at, period_start, period_end, situation_recap, regime_trajectory, key_developments[], risk_outlook, provider, model } ## Server handler server/worldmonitor/intelligence/v1/get-regional-brief.ts Simple getCachedJson read + adaptBrief snake->camel adapter. Returns upstreamUnavailable: true on Redis failure so the gateway skips caching (matching the get-regime-history pattern from #2981). ## Premium gating + cache tier src/shared/premium-paths.ts + server/gateway.ts RPC_CACHE_TIER ## Tests — 27 new unit tests buildBriefPrompt (5): region/balance/transitions/narrative rendered, empty transitions handled, missing fields tolerated parseBriefJson (5): valid JSON, garbage, all-empty, cap at 5, prose extraction generateWeeklyBrief (6): success, global skip, LLM fail, garbage, exception, period_start/end delta emptyBrief (2): region_id + empty fields handler (4): key prefix, adapter export, upstreamUnavailable, registration security (2): premium path + cache tier proto (3): RPC declared, import wired, RegionalBrief fields ## Verification - npm run test:data: 4651/4651 pass - npm run typecheck + typecheck:api: clean - biome lint: clean * fix(intelligence): address 3 review findings on #2989 P2 #1 — no consumer surface for GetRegionalBrief Acknowledged. The consumer is the RegionalIntelligenceBoard panel, which will call GetRegionalBrief and render a weekly brief block. This wiring is Phase 3 PR3 (UI) scope — the RPC + Redis key are the delivery mechanism, not the end surface. No code change in this commit; the RPC is ready for the panel to consume. P2 #2 — readRecentTransitions collapses failure to [] readRecentTransitions returned [] on Redis/network failure, which is indistinguishable from a genuinely quiet week. The LLM then generates a brief claiming "no regime transitions" when in reality the upstream is down — fabricating false input. Fix: return null on failure. The seeder skips the region with a clear log message when transitions is null, so the brief is never written with unreliable input. Empty array [] now only means genuinely no transitions in the 7-day window. P2 #3 — parseBriefJson accepts briefs the seeder rejects parseBriefJson treated non-empty key_developments as valid even if situation_recap was empty. The seeder gate only writes when brief.situation_recap is truthy. That mismatch means the validator pass + provider-fallback logic could accept a response that the seeder then silently drops. Fix: require situation_recap in parseBriefJson for valid=true, matching the seeder gate. Now both checks agree on what constitutes a usable brief, and the provider-fallback chain correctly falls through when a provider returns a brief with developments but no recap. * fix(intelligence): TTL path-segment fix + seed-meta always-write (Greptile P1+P2 on #2989) P1 — TTL silently not applied (briefs never expire) Upstash REST ignores query-string SET options (?EX=N). The correct form is path-segment: /set/{key}/{value}/EX/{seconds}. Without this fix every brief persists indefinitely and Redis storage grows unboundedly across weekly runs. P2 — seed-meta not written when all regions skipped writeExtraKeyWithMeta was gated on generated > 0. If every region was skipped (no snapshot yet, or LLM failed), seed-meta was never written, making the seeder indistinguishable from "never ran" in health tooling. Now writes seed-meta whenever failed === 0, carrying regionsSkipped count. P2 #3 (validate gate) — already fixed in previous commit (parseBriefJson now requires situation_recap for valid=true). * fix(intelligence): register regional-briefs in health.js SEED_META + STANDALONE_KEYS (review P2 on #2989) * fix(intelligence): register regional-briefs in api/seed-health.js (review P2 on #2989) * fix(intelligence): raise brief TTL to 15 days to cover missed weekly cycle (review P2 on #2989) * fix(intelligence): distinguish missing-key from Redis-error + coverage-gated health (review P2s on #2989) P2 #1 — false upstreamUnavailable before first seed getCachedJson returns null for both "key missing" and "Redis failed", so the handler was advertising an outage for every region before the first weekly seed ran. Switched to getRawJson (throws on Redis errors) so null = genuinely missing key → clean empty 200, and thrown error = upstream failure → upstreamUnavailable: true for gateway no-store. P2 #2 — partial run hides coverage loss in health The seed-meta was written with generated count even if only 1 of 7 regions produced a brief. /api/health treats any positive recordCount as healthy, so broad regional failure was invisible to operators. Fix: recordCount is set to 0 when generated < ceil(expectedRegions/2). This makes /api/health report EMPTY_DATA for severely partial runs while still writing seed-meta (so the seeder is confirmed to have run). coverageOk flag in the summary payload lets operators drill into the exact coverage state. * fix(intelligence): tighten coverage gate to expectedRegions-1 (review P2 on #2989)
4 tasks
SebastienMelki
added a commit
that referenced
this pull request
Apr 21, 2026
…nt 202→200 wire break (#3207) Commit 7 silently shifted /api/scenario/v1/run-scenario's response contract in two ways that the commit message covered only partially: 1. HTTP 202 Accepted → HTTP 200 OK 2. Dropped `statusUrl` string from the response body The `statusUrl` drop was mentioned as "unused by SupplyChainPanel" but not framed as a contract change. The 202 → 200 shift was not mentioned at all. This is a same-version (v1 → v1) migration, so external callers that key off either signal — `response.status === 202` or `response.body.statusUrl` — silently branch incorrectly. Evaluated options: (a) sebuf per-RPC status-code config — not available. sebuf's HttpConfig only models `path` and `method`; no status annotation. (b) Bump to scenario/v2 — judged heavier than the break itself for a single status-code shift. No in-repo caller uses 202 or statusUrl; the docs-level impact is containable. (c) Accept the break, document explicitly, partially restore. Took option (c): - Restored `statusUrl` in the proto (new field `string status_url = 3` on RunScenarioResponse). Server computes `/api/scenario/v1/get-scenario-status?jobId=<encoded job_id>` and populates it on every successful enqueue. External callers that followed this URL keep working unchanged. - 202 → 200 is not recoverable inside the sebuf generator, so it is called out explicitly in two places: - docs/api-scenarios.mdx now includes a prominent `<Warning>` block documenting the v1→v1 contract shift + the suggested migration (branch on response body shape, not HTTP status). - RunScenarioResponse proto comment explains why 200 is the new success status on enqueue. OpenAPI bundle regenerated to reflect the restored statusUrl field. - Regression test added in tests/scenario-handler.test.mjs pinning `statusUrl` to the exact URL-encoded shape — locks the invariant so a future proto rename or handler refactor can't silently drop it again. From koala73 review (#3242 second-pass, HIGH new #3). Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
15 tasks
SebastienMelki
added a commit
that referenced
this pull request
Apr 22, 2026
… (#3242) * chore(api): enforce sebuf contract via exceptions manifest (#3207) Adds api/api-route-exceptions.json as the single source of truth for non-proto /api/ endpoints, with scripts/enforce-sebuf-api-contract.mjs gating every PR via npm run lint:api-contract. Fixes the root-only blind spot in the prior allowlist (tests/edge-functions.test.mjs), which only scanned top-level *.js files and missed nested paths and .ts endpoints — the gap that let api/supply-chain/v1/country-products.ts and friends drift under proto domain URL prefixes unchallenged. Checks both directions: every api/<domain>/v<N>/[rpc].ts must pair with a generated service_server.ts (so a deleted proto fails CI), and every generated service must have an HTTP gateway (no orphaned generated code). Manifest entries require category + reason + owner, with removal_issue mandatory for temporary categories (deferred, migration-pending) and forbidden for permanent ones. .github/CODEOWNERS pins the manifest to @SebastienMelki so new exceptions don't slip through review. The manifest only shrinks: migration-pending entries (19 today) will be removed as subsequent commits in this PR land each migration. * refactor(maritime): migrate /api/ais-snapshot → maritime/v1.GetVesselSnapshot (#3207) The proto VesselSnapshot was carrying density + disruptions but the frontend also needed sequence, relay status, and candidate_reports to drive the position-callback system. Those only lived on the raw relay passthrough, so the client had to keep hitting /api/ais-snapshot whenever callbacks were registered and fall back to the proto RPC only when the relay URL was gone. This commit pushes all three missing fields through the proto contract and collapses the dual-fetch-path into one proto client call. Proto changes (proto/worldmonitor/maritime/v1/): - VesselSnapshot gains sequence, status, candidate_reports. - GetVesselSnapshotRequest gains include_candidates (query: include_candidates). Handler (server/worldmonitor/maritime/v1/get-vessel-snapshot.ts): - Forwards include_candidates to ?candidates=... on the relay. - Separate 5-min in-memory caches for the candidates=on and candidates=off variants; they have very different payload sizes and should not share a slot. - Per-request in-flight dedup preserved per-variant. Frontend (src/services/maritime/index.ts): - fetchSnapshotPayload now calls MaritimeServiceClient.getVesselSnapshot directly with includeCandidates threaded through. The raw-relay path, SNAPSHOT_PROXY_URL, DIRECT_RAILWAY_SNAPSHOT_URL and LOCAL_SNAPSHOT_FALLBACK are gone — production already routed via Vercel, the "direct" branch only ever fired on localhost, and the proto gateway covers both. - New toLegacyCandidateReport helper mirrors toDensityZone/toDisruptionEvent. api/ais-snapshot.js deleted; manifest entry removed. Only reduced the codegen scope to worldmonitor.maritime.v1 (buf generate --path) — regenerating the full tree drops // @ts-nocheck from every client/server file and surfaces pre-existing type errors across 30+ unrelated services, which is not in scope for this PR. Shape-diff vs legacy payload: - disruptions / density: proto carries the same fields, just with the GeoCoordinates wrapper and enum strings (remapped client-side via existing toDisruptionEvent / toDensityZone helpers). - sequence, status.{connected,vessels,messages}: now populated from the proto response — was hardcoded to 0/false in the prior proto fallback. - candidateReports: same shape; optional numeric fields come through as 0 instead of undefined, which the legacy consumer already handled. * refactor(sanctions): migrate /api/sanctions-entity-search → LookupSanctionEntity (#3207) The proto docstring already claimed "OFAC + OpenSanctions" coverage but the handler only fuzzy-matched a local OFAC Redis index — narrower than the legacy /api/sanctions-entity-search, which proxied OpenSanctions live (the source advertised in docs/api-proxies.mdx). Deleting the legacy without expanding the handler would have been a silent coverage regression for external consumers. Handler changes (server/worldmonitor/sanctions/v1/lookup-entity.ts): - Primary path: live search against api.opensanctions.org/search/default with an 8s timeout and the same User-Agent the legacy edge fn used. - Fallback path: the existing OFAC local fuzzy match, kept intact for when OpenSanctions is unreachable / rate-limiting. - Response source field flips between 'opensanctions' (happy path) and 'ofac' (fallback) so clients can tell which index answered. - Query validation tightened: rejects q > 200 chars (matches legacy cap). Rate limiting: - Added /api/sanctions/v1/lookup-entity to ENDPOINT_RATE_POLICIES at 30/min per IP — matches the legacy createIpRateLimiter budget. The gateway already enforces per-endpoint policies via checkEndpointRateLimit. Docs: - docs/api-proxies.mdx — dropped the /api/sanctions-entity-search row (plus the orphaned /api/ais-snapshot row left over from the previous commit in this PR). - docs/panels/sanctions-pressure.mdx — points at the new RPC URL and describes the OpenSanctions-primary / OFAC-fallback semantics. api/sanctions-entity-search.js deleted; manifest entry removed. * refactor(military): migrate /api/military-flights → ListMilitaryFlights (#3207) Legacy /api/military-flights read a pre-baked Redis blob written by the seed-military-flights cron and returned flights in a flat app-friendly shape (lat/lon, lowercase enums, lastSeenMs). The proto RPC takes a bbox, fetches OpenSky live, classifies server-side, and returns nested GeoCoordinates + MILITARY_*_TYPE_* enum strings + lastSeenAt — same data, different contract. fetchFromRedis in src/services/military-flights.ts was doing nothing sebuf-aware. Renamed it to fetchViaProto and rewrote to: - Instantiate MilitaryServiceClient against getRpcBaseUrl(). - Iterate MILITARY_QUERY_REGIONS (PACIFIC + WESTERN) in parallel — same regions the desktop OpenSky path and the seed cron already use, so dashboard coverage tracks the analytic pipeline. - Dedup by hexCode across regions. - Map proto → app shape via new mapProtoFlight helper plus three reverse enum maps (AIRCRAFT_TYPE_REVERSE, OPERATOR_REVERSE, CONFIDENCE_REVERSE). The seed cron (scripts/seed-military-flights.mjs) stays put: it feeds regional-snapshot mobility, cross-source signals, correlation, and the health freshness check (api/health.js: 'military:flights:v1'). None of those read the legacy HTTP endpoint; they read the Redis key directly. The proto handler uses its own per-bbox cache keys under the same prefix, so dashboard traffic no longer races the seed cron's blob — the two paths diverge by a small refresh lag, which is acceptable. Docs: dropped the /api/military-flights row from docs/api-proxies.mdx. api/military-flights.js deleted; manifest entry removed. Shape-diff vs legacy: - f.location.{latitude,longitude} → f.lat, f.lon - f.aircraftType: MILITARY_AIRCRAFT_TYPE_TANKER → 'tanker' via reverse map - f.operator: MILITARY_OPERATOR_USAF → 'usaf' via reverse map - f.confidence: MILITARY_CONFIDENCE_LOW → 'low' via reverse map - f.lastSeenAt (number) → f.lastSeen (Date) - f.enrichment → f.enriched (with field renames) - Extra fields registration / aircraftModel / origin / destination / firstSeenAt now flow through where proto populates them. * fix(supply-chain): thread includeCandidates through chokepoint status (#3207) Caught by tsconfig.api.json typecheck in the pre-push hook (not covered by the plain tsc --noEmit run that ran before I pushed the ais-snapshot commit). The chokepoint status handler calls getVesselSnapshot internally with a static no-auth request — now required to include the new includeCandidates bool from the proto extension. Passing false: server-internal callers don't need per-vessel reports. * test(maritime): update getVesselSnapshot cache assertions (#3207) The ais-snapshot migration replaced the single cachedSnapshot/cacheTimestamp pair with a per-variant cache so candidates-on and candidates-off payloads don't evict each other. Pre-push hook surfaced that tests/server-handlers still asserted the old variable names. Rewriting the assertions to match the new shape while preserving the invariants they actually guard: - Freshness check against slot TTL. - Cache read before relay call. - Per-slot in-flight dedup. - Stale-serve on relay failure (result ?? slot.snapshot). * chore(proto): restore // @ts-nocheck on regenerated maritime files (#3207) I ran 'buf generate --path worldmonitor/maritime/v1' to scope the proto regen to the one service I was changing (to avoid the toolchain drift that drops @ts-nocheck from 60+ unrelated files — separate issue). But the repo convention is the 'make generate' target, which runs buf and then sed-prepends '// @ts-nocheck' to every generated .ts file. My scoped command skipped the sed step. The proto-check CI enforces the sed output, so the two maritime files need the directive restored. * refactor(enrichment): decomm /api/enrichment/{company,signals} legacy edge fns (#3207) Both endpoints were already ported to IntelligenceService: - getCompanyEnrichment (/api/intelligence/v1/get-company-enrichment) - listCompanySignals (/api/intelligence/v1/list-company-signals) No frontend callers of the legacy /api/enrichment/* paths exist. Removes: - api/enrichment/company.js, signals.js, _domain.js - api-route-exceptions.json migration-pending entries (58 remain) - docs/api-proxies.mdx rows for /api/enrichment/{company,signals} - docs/architecture.mdx reference updated to the IntelligenceService RPCs Verified: typecheck, typecheck:api, lint:api-contract (89 files / 58 entries), lint:boundaries, tests/edge-functions.test.mjs (136 pass), tests/enrichment-caching.test.mjs (14 pass — still guards the intelligence/v1 handlers), make generate is zero-diff. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * refactor(leads): migrate /api/{contact,register-interest} → LeadsService (#3207) New leads/v1 sebuf service with two POST RPCs: - SubmitContact → /api/leads/v1/submit-contact - RegisterInterest → /api/leads/v1/register-interest Handler logic ported 1:1 from api/contact.js + api/register-interest.js: - Turnstile verification (desktop sources bypass, preserved) - Honeypot (website field) silently accepts without upstream calls - Free-email-domain gate on SubmitContact (422 ApiError) - validateEmail (disposable/offensive/typo-TLD/MX) on RegisterInterest - Convex writes via ConvexHttpClient (contactMessages:submit, registerInterest:register) - Resend notification + confirmation emails (HTML templates unchanged) Shared helpers moved to server/_shared/: - turnstile.ts (getClientIp + verifyTurnstile) - email-validation.ts (disposable/offensive/MX checks) Rate limits preserved via ENDPOINT_RATE_POLICIES: - submit-contact: 3/hour per IP (was in-memory 3/hr) - register-interest: 5/hour per IP (was in-memory 5/hr; desktop sources previously capped at 2/hr via shared in-memory map — now 5/hr like everyone else, accepting the small regression in exchange for Upstash-backed global limiting) Callers updated: - pro-test/src/App.tsx contact form → new submit-contact path - src-tauri/sidecar/local-api-server.mjs cloud-fallback rewrites /api/register-interest → /api/leads/v1/register-interest when proxying; keeps local path for older desktop builds - src/services/runtime.ts isKeyFreeApiTarget allows both old and new paths through the WORLDMONITOR_API_KEY-optional gate Tests: - tests/contact-handler.test.mjs rewritten to call submitContact handler directly; asserts on ValidationError / ApiError - tests/email-validation.test.mjs + tests/turnstile.test.mjs point at the new server/_shared/ modules Deleted: api/contact.js, api/register-interest.js, api/_ip-rate-limit.js, api/_turnstile.js, api/_email-validation.js, api/_turnstile.test.mjs. Manifest entries removed (58 → 56). Docs updated (api-platform, api-commerce, usage-rate-limits). Verified: npm run typecheck + typecheck:api + lint:api-contract (88 files / 56 entries) + lint:boundaries pass; full test:data (5852 tests) passes; make generate is zero-diff. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * chore(pro-test): rebuild bundle for leads/v1 contact form (#3207) Updates the enterprise contact form to POST to /api/leads/v1/submit-contact (old path /api/contact removed in the previous commit). Bundle is rebuilt from pro-test/src/App.tsx source change in 9ccd309. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * fix(review): address HIGH review findings 1-3 (#3207) Three review findings from @koala73 on the sebuf-migration PR, all silent bugs that would have shipped to prod: ### 1. Sanctions rate-limit policy was dead code ENDPOINT_RATE_POLICIES keyed the 30/min budget under /api/sanctions/v1/lookup-entity, but the generated route (from the proto RPC LookupSanctionEntity) is /api/sanctions/v1/lookup-sanction-entity. hasEndpointRatePolicy / getEndpointRatelimit are exact-string pathname lookups, so the mismatch meant the endpoint fell through to the generic 600/min global limiter instead of the advertised 30/min. Net effect: the live OpenSanctions proxy endpoint (unauthenticated, external upstream) had 20x the intended rate budget. Fixed by renaming the policy key to match the generated route. ### 2. Lost stale-seed fallback on military-flights Legacy api/military-flights.js cascaded military:flights:v1 → military:flights:stale:v1 before returning empty. The new proto handler went straight to live OpenSky/relay and returned null on miss. Relay or OpenSky hiccup used to serve stale seeded data (24h TTL); under the new handler it showed an empty map. Both keys are still written by scripts/seed-military-flights.mjs on every run — fix just reads the stale key when the live fetch returns null, converts the seed's app-shape flights (flat lat/lon, lowercase enums, lastSeenMs) to the proto shape (nested GeoCoordinates, enum strings, lastSeenAt), and filters to the request bbox. Read via getRawJson (unprefixed) to match the seed cron's writes, which bypass the env-prefix system. ### 3. Hex-code casing mismatch broke getFlightByHex The seed cron writes hexCode: icao24.toUpperCase() (uppercase); src/services/military-flights.ts:getFlightByHex uppercases the lookup input: f.hexCode === hexCode.toUpperCase(). The new proto handler preserved OpenSky's lowercase icao24, and mapProtoFlight is a pass-through. getFlightByHex was silently returning undefined for every call after the migration. Fix: uppercase in the proto handler (live + stale paths), and document the invariant in a comment on MilitaryFlight.hex_code in military_flight.proto so future handlers don't re-break it. ### Verified - typecheck + typecheck:api clean - lint:api-contract (56 entries) / lint:boundaries clean - tests/edge-functions.test.mjs 130 pass - make generate zero-diff (openapi spec regenerated for proto comment) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * fix(review): restore desktop 2/hr rate cap on register-interest (#3207) Addresses HIGH review finding #4 from @koala73. The legacy api/register-interest.js applied a nested 2/hr per-IP cap when `source === 'desktop-settings'`, on top of the generic 5/hr endpoint budget. The sebuf migration lost this — desktop-source requests now enjoy the full 5/hr cap. Since `source` is an unsigned client-supplied field, anyone sending `source: 'desktop-settings'` skips Turnstile AND gets 5/hr. Without the tighter cap the Turnstile bypass is cheaper to abuse. Added `checkScopedRateLimit` to `server/_shared/rate-limit.ts` — a reusable second-stage Upstash limiter keyed on an opaque scope string + caller identifier. Fail-open on Redis errors to match existing checkRateLimit / checkEndpointRateLimit semantics. Handlers that need per-subscope caps on top of the gateway-level endpoint budget use this helper. In register-interest: when `isDesktopSource`, call checkScopedRateLimit with scope `/api/leads/v1/register-interest#desktop`, limit=2, window=1h, IP as identifier. On exceeded → throw ApiError(429). ### What this does not fix This caps the blast radius of the Turnstile bypass but does not close it — an attacker sending `source: 'desktop-settings'` still skips Turnstile (just at 2/hr instead of 5/hr). The proper fix is a signed desktop-secret header that authenticates the bypass; filed as follow-up #3252. That requires coordinated Tauri build + Vercel env changes out of scope for #3207. ### Verified - typecheck + typecheck:api clean - lint:api-contract (56 entries) - tests/edge-functions.test.mjs + contact-handler.test.mjs (147 pass) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * fix(review): MEDIUM + LOW + rate-limit-policy CI check (#3207) Closes out the remaining @koala73 review findings from #3242 that didn't already land in the HIGH-fix commits, plus the requested CI check that would have caught HIGH #1 (dead-code policy key) at review time. ### MEDIUM #5 — Turnstile missing-secret policy default Flip `verifyTurnstile`'s default `missingSecretPolicy` from `'allow'` to `'allow-in-development'`. Dev with no secret = pass (expected local); prod with no secret = reject + log. submit-contact was already explicitly overriding to `'allow-in-development'`; register-interest was silently getting `'allow'`. Safe default now means a future missing-secret misconfiguration in prod gets caught instead of silently letting bots through. Removed the now-redundant override in submit-contact. ### MEDIUM #6 — Silent enum fallbacks in maritime client `toDisruptionEvent` mapped `AIS_DISRUPTION_TYPE_UNSPECIFIED` / unknown enum values → `gap_spike` / `low` silently. Refactored to return null when either enum is unknown; caller filters nulls out of the array. Handler doesn't produce UNSPECIFIED today, but the `gap_spike` default would have mislabeled the first new enum value the proto ever adds — dropping unknowns is safer than shipping wrong labels. ### LOW — Copy drift in register-interest email Email template hardcoded `435+ Sources`; PR #3241 bumped marketing to `500+`. Bumped in the rewritten file to stay consistent. The `as any` on Convex mutation names carried over from legacy and filed as follow-up #3253. ### Rate-limit-policy coverage lint `scripts/enforce-rate-limit-policies.mjs` validates every key in `ENDPOINT_RATE_POLICIES` resolves to a proto-generated gateway route by cross-referencing `docs/api/*.openapi.yaml`. Fails with the sanctions-entity-search incident referenced in the error message so future drift has a paper trail. Wired into package.json (`lint:rate-limit-policies`) and the pre-push hook alongside `lint:boundaries`. Smoke-tested both directions — clean repo passes (5 policies / 175 routes), seeded drift (the exact HIGH #1 typo) fails with the advertised remedy text. ### Verified - `lint:rate-limit-policies` ✓ - `typecheck` + `typecheck:api` ✓ - `lint:api-contract` ✓ (56 entries) - `lint:boundaries` ✓ - edge-functions + contact-handler tests (147 pass) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * refactor(commit 5): decomm /api/eia/* + migrate /api/satellites → IntelligenceService (#3207) Both targets turned out to be decomm-not-migration cases. The original plan called for two new services (economic/v1.GetEiaSeries + natural/v1.ListSatellitePositions) but research found neither was needed: ### /api/eia/[[...path]].js — pure decomm, zero consumers The "catch-all" is a misnomer — only two paths actually worked, /api/eia/health and /api/eia/petroleum, both Redis-only readers. Zero frontend callers in src/. Zero server-side readers. Nothing consumes the `energy:eia-petroleum:v1` key that seed-eia-petroleum.mjs writes daily. The EIA data the frontend actually uses goes through existing typed RPCs in economic/v1: GetEnergyPrices, GetCrudeInventories, GetNatGasStorage, GetEnergyCapacity. None of those touch /api/eia/*. Building GetEiaSeries would have been dead code. Deleted the legacy file + its test (tests/api-eia-petroleum.test.mjs — it only covered the legacy endpoint, no behavior to preserve). Empty api/eia/ dir removed. **Note for review:** the Redis seed cron keeps running daily and nothing consumes it. If that stays unused, seed-eia-petroleum.mjs should be retired too (separate PR). Out of scope for sebuf-migration. ### /api/satellites.js — Learning #2 strikes again IntelligenceService.ListSatellites already exists at /api/intelligence/v1/list-satellites, reads the same Redis key (intelligence:satellites:tle:v1), and supports an optional country filter the legacy didn't have. One frontend caller in src/services/satellites.ts needed to switch from `fetch(toApiUrl('/api/satellites'))` to the typed IntelligenceServiceClient.listSatellites. Shape diff was tiny — legacy `noradId` became proto `id` (handler line 36 already picks either), everything else identical. alt/velocity/inclination in the proto are ignored by the caller since it propagates positions client-side via satellite.js. Kept the client-side cache + failure cooldown + 20s timeout (still valid concerns at the caller level). ### Manifest + docs - api-route-exceptions.json: 56 → 54 entries (both removed) - docs/api-proxies.mdx: dropped the two rows from the Raw-data passthroughs table ### Verified - typecheck + typecheck:api ✓ - lint:api-contract (54 entries) / lint:boundaries / lint:rate-limit-policies ✓ - tests/edge-functions.test.mjs 127 pass (down from 130 — 3 tests were for the deleted eia endpoint) - make generate zero-diff (no proto changes) Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * refactor(commit 6): migrate /api/supply-chain/v1/{country-products,multi-sector-cost-shock} → SupplyChainService (#3207) Both endpoints were hand-rolled TS handlers sitting under a proto URL prefix — the exact drift the manifest guardrail flagged. Promoted both to typed RPCs: - GetCountryProducts → /api/supply-chain/v1/get-country-products - GetMultiSectorCostShock → /api/supply-chain/v1/get-multi-sector-cost-shock Handlers preserve the existing semantics: PRO-gate via isCallerPremium(ctx.request), iso2 / chokepointId validation, raw bilateral-hs4 Redis read (skip env-prefix to match seeder writes), CHOKEPOINT_STATUS_KEY for war-risk tier, and the math from _multi-sector-shock.ts unchanged. Empty-data and non-PRO paths return the typed empty payload (no 403 — the sebuf gateway pattern is empty-payload-on-deny). Client wrapper switches from premiumFetch to client.getCountryProducts/ client.getMultiSectorCostShock. Legacy MultiSectorShock / MultiSectorShockResponse / CountryProductsResponse names remain as type aliases of the generated proto types so CountryBriefPanel + CountryDeepDivePanel callsites compile with zero churn. Manifest 54 → 52. Rate-limit gateway routes 175 → 177. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * fix(gateway): add cache-tier entries for new supply-chain RPCs (#3207) Pre-push tests/route-cache-tier.test.mjs caught the missing entries. Both PRO-gated, request-varying — match the existing supply-chain PRO cohort (get-country-cost-shock, get-bypass-options, etc.) at slow-browser tier. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * refactor(commit 7): migrate /api/scenario/v1/{run,status,templates} → ScenarioService (#3207) Promote the three literal-filename scenario endpoints to a typed sebuf service with three RPCs: POST /api/scenario/v1/run-scenario (RunScenario) GET /api/scenario/v1/get-scenario-status (GetScenarioStatus) GET /api/scenario/v1/list-scenario-templates (ListScenarioTemplates) Preserves all security invariants from the legacy handlers: - 405 for wrong method (sebuf service-config method gate) - scenarioId validation against SCENARIO_TEMPLATES registry - iso2 regex ^[A-Z]{2}$ - JOB_ID_RE path-traversal guard on status - Per-IP 10/min rate limit (moved to gateway ENDPOINT_RATE_POLICIES) - Queue-depth backpressure (>100 → 429) - PRO gating via isCallerPremium - AbortSignal.timeout on every Redis pipeline (runRedisPipeline helper) Wire-level diffs vs legacy: - Per-user RL now enforced at the gateway (same 10/min/IP budget). - Rate-limit response omits Retry-After header; retryAfter is in the body per error-mapper.ts convention. - ListScenarioTemplates emits affectedHs2: [] when the registry entry is null (all-sectors sentinel); proto repeated cannot carry null. - RunScenario returns { jobId, status } (no statusUrl field — unused by SupplyChainPanel, drop from wire). Gateway wiring: - server/gateway.ts RPC_CACHE_TIER: list-scenario-templates → 'daily' (matches legacy max-age=3600); get-scenario-status → 'slow-browser' (premium short-circuit target, explicit entry required by tests/route-cache-tier.test.mjs). - src/shared/premium-paths.ts: swap old run/status for the new run-scenario/get-scenario-status paths. - api/scenario/v1/{run,status,templates}.ts deleted; 3 manifest exceptions removed (63 → 52 → 49 migration-pending). Client: - src/services/scenario/index.ts — typed client wrapper using premiumFetch (injects Clerk bearer / API key). - src/components/SupplyChainPanel.ts — polling loop swapped from premiumFetch strings to runScenario/getScenarioStatus. Hard 20s timeout on run preserved via AbortSignal.any. Tests: - tests/scenario-handler.test.mjs — 18 new handler-level tests covering every security invariant + the worker envelope coercion. - tests/edge-functions.test.mjs — scenario sections removed, replaced with a breadcrumb pointer to the new test file. Docs: api-scenarios.mdx, scenario-engine.mdx, usage-rate-limits.mdx, usage-errors.mdx, supply-chain.mdx refreshed with new paths. Verified: typecheck, typecheck:api, lint:api-contract (49 entries), lint:rate-limit-policies (6/180), lint:boundaries, route-cache-tier (parity), full edge-functions (117) + scenario-handler (18). Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * refactor(commit 8): migrate /api/v2/shipping/{route-intelligence,webhooks} → ShippingV2Service (#3207) Partner-facing endpoints promoted to a typed sebuf service. Wire shape preserved byte-for-byte (camelCase field names, ISO-8601 fetchedAt, the same subscriberId/secret formats, the same SET + SADD + EXPIRE 30-day Redis pipeline). Partner URLs /api/v2/shipping/* are unchanged. RPCs landed: - GET /route-intelligence → RouteIntelligence (PRO, slow-browser) - POST /webhooks → RegisterWebhook (PRO) - GET /webhooks → ListWebhooks (PRO, slow-browser) The existing path-parameter URLs remain on the legacy edge-function layout because sebuf's HTTP annotations don't currently model path params (grep proto/**/*.proto for `path: "{…}"` returns zero). Those endpoints are split into two Vercel dynamic-route files under api/v2/shipping/webhooks/, behaviorally identical to the previous hybrid file but cleanly separated: - GET /webhooks/{subscriberId} → [subscriberId].ts - POST /webhooks/{subscriberId}/rotate-secret → [subscriberId]/[action].ts - POST /webhooks/{subscriberId}/reactivate → [subscriberId]/[action].ts Both get manifest entries under `migration-pending` pointing at #3207. Other changes - scripts/enforce-sebuf-api-contract.mjs: extended GATEWAY_RE to accept api/v{N}/{domain}/[rpc].ts (version-first) alongside the canonical api/{domain}/v{N}/[rpc].ts; first-use of the reversed ordering is shipping/v2 because that's the partner contract. - vite.config.ts: dev-server sebuf interceptor regex extended to match both layouts; shipping/v2 import + allRoutes entry added. - server/gateway.ts: RPC_CACHE_TIER entries for /api/v2/shipping/ route-intelligence + /webhooks (slow-browser; premium-gated endpoints short-circuit to slow-browser but the entries are required by tests/route-cache-tier.test.mjs). - src/shared/premium-paths.ts: route-intelligence + webhooks added. - tests/shipping-v2-handler.test.mjs: 18 handler-level tests covering PRO gate, iso2/cargoType/hs2 coercion, SSRF guards (http://, RFC1918, cloud metadata, IMDS), chokepoint whitelist, alertThreshold range, secret/subscriberId format, pipeline shape + 30-day TTL, cross-tenant owner isolation, `secret` omission from list response. Manifest delta - Removed: api/v2/shipping/route-intelligence.ts, api/v2/shipping/webhooks.ts - Added: api/v2/shipping/webhooks/[subscriberId].ts (migration-pending) - Added: api/v2/shipping/webhooks/[subscriberId]/[action].ts (migration-pending) - Added: api/internal/brief-why-matters.ts (internal-helper) — regression surface from the #3248 main merge, which introduced the file without a manifest entry. Filed here to keep the lint green; not strictly in scope for commit 8 but unblocking. Net result: 49 → 47 `migration-pending` entries (one net-removal even though webhook path-params stay pending, because two files collapsed into two dynamic routes). Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * fix(review HIGH 1): SupplyChainServiceClient must use premiumFetch (#3207) Signed-in browser pro users were silently hitting 401 on 8 supply-chain premium endpoints (country-products, multi-sector-cost-shock, country-chokepoint-index, bypass-options, country-cost-shock, sector-dependency, route-explorer-lane, route-impact). The shared client was constructed with globalThis.fetch, so no Clerk bearer or X-WorldMonitor-Key was injected. The gateway's validateApiKey runs with forceKey=true for PREMIUM_RPC_PATHS and 401s before isCallerPremium is consulted. The generated client's try/catch collapses the 401 into an empty-fallback return, leaving panels blank with no visible error. Fix is one line at the client constructor: swap globalThis.fetch for premiumFetch. The same pattern is already in use for insider-transactions, stock-analysis, stock-backtest, scenario, trade (premiumClient) — this was an omission on this client, not a new pattern. premiumFetch no-ops safely when no credentials are available, so the 5 non-premium methods on this client (shippingRates, chokepointStatus, chokepointHistory, criticalMinerals, shippingStress) continue to work unchanged. This also fixes two panels that were pre-existing latently broken on main (chokepoint-index, bypass-options, etc. — predating #3207, not regressions from it). Commit 6 expanded the surface by routing two more methods through the same buggy client; this commit fixes the class. From koala73 review (#3242 second-pass, HIGH new #1): > Exact class PR #3233 fixed for RegionalIntelligenceBoard / > DeductionPanel / trade / country-intel. Supply-chain was not in > #3233's scope. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * fix(review HIGH 2): restore 400 on input-shape errors for 2 supply-chain handlers (#3207) Commit 6 collapsed all non-happy paths into empty-200 on `get-country-products` and `get-multi-sector-cost-shock`, including caller-bug cases that legacy returned 400 for: - get-country-products: malformed iso2 → empty 200 (was 400) - get-multi-sector-cost-shock: malformed iso2 / missing chokepointId / unknown chokepointId → empty 200 (was 400) The commit message for 6 called out the 403-for-non-pro → empty-200 shift ("sebuf gateway pattern is empty-payload-on-deny") but not the 400 shift. They're different classes: - Empty-payload-200 for PRO-deny: intentional contract change, already documented and applied across the service. Generated clients treat "you lack PRO" as "no data" — fine. - Empty-payload-200 for malformed input: caller bug silently masked. External API consumers can't distinguish "bad wiring" from "genuinely no data", test harnesses lose the signal, bad calling code doesn't surface in Sentry. Fix: `throw new ValidationError(violations)` on the 3 input-shape branches. The generated sebuf server maps ValidationError → HTTP 400 (see src/generated/server/.../service_server.ts and leads/v1 which already uses this pattern). PRO-gate deny stays as empty-200 — that contract shift was intentional and is preserved. Regression tests added at tests/supply-chain-validation.test.mjs (8 cases) pinning the three-way contract: - bad input → 400 (ValidationError) - PRO-gate deny on valid input → 200 empty - valid PRO input, no data in Redis → 200 empty (unchanged) From koala73 review (#3242 second-pass, HIGH new #2). Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * fix(review HIGH 3): restore statusUrl on RunScenarioResponse + document 202→200 wire break (#3207) Commit 7 silently shifted /api/scenario/v1/run-scenario's response contract in two ways that the commit message covered only partially: 1. HTTP 202 Accepted → HTTP 200 OK 2. Dropped `statusUrl` string from the response body The `statusUrl` drop was mentioned as "unused by SupplyChainPanel" but not framed as a contract change. The 202 → 200 shift was not mentioned at all. This is a same-version (v1 → v1) migration, so external callers that key off either signal — `response.status === 202` or `response.body.statusUrl` — silently branch incorrectly. Evaluated options: (a) sebuf per-RPC status-code config — not available. sebuf's HttpConfig only models `path` and `method`; no status annotation. (b) Bump to scenario/v2 — judged heavier than the break itself for a single status-code shift. No in-repo caller uses 202 or statusUrl; the docs-level impact is containable. (c) Accept the break, document explicitly, partially restore. Took option (c): - Restored `statusUrl` in the proto (new field `string status_url = 3` on RunScenarioResponse). Server computes `/api/scenario/v1/get-scenario-status?jobId=<encoded job_id>` and populates it on every successful enqueue. External callers that followed this URL keep working unchanged. - 202 → 200 is not recoverable inside the sebuf generator, so it is called out explicitly in two places: - docs/api-scenarios.mdx now includes a prominent `<Warning>` block documenting the v1→v1 contract shift + the suggested migration (branch on response body shape, not HTTP status). - RunScenarioResponse proto comment explains why 200 is the new success status on enqueue. OpenAPI bundle regenerated to reflect the restored statusUrl field. - Regression test added in tests/scenario-handler.test.mjs pinning `statusUrl` to the exact URL-encoded shape — locks the invariant so a future proto rename or handler refactor can't silently drop it again. From koala73 review (#3242 second-pass, HIGH new #3). Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * fix(review HIGH 1/2): close webhook tenant-isolation gap on shipping/v2 (#3207) Koala flagged this as a merge blocker in PR #3242 review. server/worldmonitor/shipping/v2/{register-webhook,list-webhooks}.ts migrated without reinstating validateApiKey(req, { forceKey: true }), diverging from both the sibling api/v2/shipping/webhooks/[subscriberId] routes and the documented "X-WorldMonitor-Key required" contract in docs/api-shipping-v2.mdx. Attack surface: the gateway accepts Clerk bearer auth as a pro signal. A Clerk-authenticated pro user with no X-WorldMonitor-Key reaches the handler, callerFingerprint() falls back to 'anon', and every such caller collapses into a shared webhook:owner:anon:v1 bucket. The defense-in-depth ownerTag !== ownerHash check in list-webhooks.ts doesn't catch it because both sides equal 'anon' — every Clerk-session holder could enumerate / overwrite every other Clerk-session pro tenant's registered webhook URLs. Fix: reinstate validateApiKey(ctx.request, { forceKey: true }) at the top of each handler, throwing ApiError(401) when absent. Matches the sibling routes exactly and the published partner contract. Tests: - tests/shipping-v2-handler.test.mjs: two existing "non-PRO → 403" tests for register/list were using makeCtx() with no key, which now fails at the 401 layer first. Renamed to "no API key → 401 (tenant-isolation gate)" with a comment explaining the failure mode being tested. 18/18 pass. Verified: typecheck:api, lint:api-contract (no change), lint:boundaries, lint:rate-limit-policies, test:data (6005/6005). Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> * fix(review HIGH 2/2): restore v1 path aliases on scenario + supply-chain (#3207) Koala flagged this as a merge blocker in PR #3242 review. Commits 6 + 7 of #3207 renamed five documented v1 URLs to the sebuf method-derived paths and deleted the legacy edge-function files: POST /api/scenario/v1/run → run-scenario GET /api/scenario/v1/status → get-scenario-status GET /api/scenario/v1/templates → list-scenario-templates GET /api/supply-chain/v1/country-products → get-country-products GET /api/supply-chain/v1/multi-sector-cost-shock → get-multi-sector-cost-shock server/router.ts is an exact static-match table (Map keyed on `METHOD PATH`), so any external caller — docs, partner scripts, grep-the- internet — hitting the old documented URL would 404 on first request after merge. Commit 8 (shipping/v2) preserved partner URLs byte-for- byte; the scenario + supply-chain renames missed that discipline. Fix: add five thin alias edge functions that rewrite the pathname to the canonical sebuf path and delegate to the domain [rpc].ts gateway via a new server/alias-rewrite.ts helper. Premium gating, rate limits, entitlement checks, and cache-tier lookups all fire on the canonical path — aliases are pure URL rewrites, not a duplicate handler pipeline. api/scenario/v1/{run,status,templates}.ts api/supply-chain/v1/{country-products,multi-sector-cost-shock}.ts Vite dev parity: file-based routing at api/ is a Vercel concern, so the dev middleware (vite.config.ts) gets a matching V1_ALIASES rewrite map before the router dispatch. Manifest: 5 new entries under `deferred` with removal_issue=#3282 (tracking their retirement at the next v1→v2 break). lint:api-contract stays green (89 files checked, 55 manifest entries validated). Docs: - docs/api-scenarios.mdx: migration callout at the top with the full old→new URL table and a link to the retirement issue. - CHANGELOG.md + docs/changelog.mdx: Changed entry documenting the rename + alias compat + the 202→200 shift (from commit 23c821a). Verified: typecheck:api, lint:api-contract, lint:rate-limit-policies, lint:boundaries, test:data (6005/6005). Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]> --------- Co-authored-by: Claude Opus 4.7 (1M context) <[email protected]>
8 tasks
13 tasks
koala73
added a commit
that referenced
this pull request
Apr 24, 2026
…#3358) * fix(insights): trust cluster rank, stop LLM from re-picking top story WORLD BRIEF panel published "Iran's new supreme leader was seriously wounded, leading him to delegate power to the Revolutionary Guards. This development comes amid an ongoing war with Israel." to every visitor for 3h. Payload: openrouter / gemini-2.5-flash. Root cause: callLLM sent all 10 clustered headlines with "pick the ONE most significant and summarize ONLY that story". Clustering ranked Lebanon journalist killing #1 (2 corroborating sources); News24 Iran rumor ranked #3 (1 source). Gemini overrode the rank, picked #3, and embellished with war framing from story #4. Objective rank (sourceCount, velocity, isAlert) lost to model vibe. Shrink the LLM's job to phrasing. Clustering already ranks — pass only topStories[0].primaryTitle and instruct the model to rewrite it using ONLY facts from the headline. No name/place/context invention. Also: - temperature 0.3 -> 0.1 (factual summary, not creative) - CACHE_TTL 3h -> 30m so a bad brief ages out in one cron cycle - Drop dead MAX_HEADLINES const Payload shape unchanged; frontend untouched. * fix(insights): corroboration gate + revert TTL + drop unconditional WHERE Follow-up to review feedback on the ranking contract, TTL, and prompt: 1. Corroboration gate (P1a). scoreImportance() in scripts/_clustering.mjs is keyword-heavy (violence +125 on a single word, flashpoint +75, ^1.5 multiplier when both hit), so a single-source sensational rumor can outrank a 2-source lead purely on lexical signals. Blindly trusting topStories[0] would let the ranker's keyword bias still pick bad stories. Walk topStories for sourceCount >= 2 instead — corroboration becomes a hard requirement, not a tiebreaker. If no cluster qualifies, publish status=degraded with no brief (frontend already handles this). 2. CACHE_TTL back to 10800 (P1b). 30m TTL == one cron cadence means the key expires on any missed or delayed run and /api/bootstrap loses insights entirely (api/bootstrap.js reads news:insights:v1 directly, no LKG across TTL-gap). The short TTL was defense-in-depth for bad content; the real safety is now upstream (corroboration gate + grounded prompt), so the LKG window doesn't need to be sacrificed for it. 3. Prompt: location conditional (P2). "Use ONLY facts present" + "Lead with WHAT happened and WHERE" conflicted for headlines without an explicit location and pushed the model toward inferred-place hallucination. Replaced with "Include a location, person, or organization ONLY if it appears in the headline." * test(insights): lock corroboration gate + grounded-prompt invariants Review P2: the corroboration gate and the prompt's no-invention rules had no tests, so future edits to selectTopStories() ordering or prompt text could silently reintroduce the original hallucination. Extract the brief-selection helper and prompt builders into a pure module (scripts/_insights-brief.mjs) so tests can import them without triggering seed-insights.mjs's top-level runSeed() call: - pickBriefCluster(topStories) returns first sourceCount>=2 cluster - briefSystemPrompt(dateISO) returns the system prompt - briefUserPrompt(headline) returns the user prompt Regression tests (tests/seed-insights-brief.test.mjs, 12 cases) lock: - pickBriefCluster skips single-source rumors even when ranked above a multi-sourced lead (explicit regression: News24 Iran supreme leader 2026-04-23 scenario with realistic scores) - pickBriefCluster tolerates missing/null entries - briefSystemPrompt forbids invented facts and proper nouns - briefSystemPrompt's "location" rule is conditional (no unconditional "Lead with WHAT and WHERE" directive that would push the model toward place-inference when the headline has no location) - briefSystemPrompt does not contain "pick the most important" style language (ranking is done by pickBriefCluster upstream) - briefUserPrompt passes the headline verbatim and instructs "only facts from this headline" Also fix a misleading comment on CACHE_TTL: corroboration is gated at brief-selection time, not on the topStories payload itself (which still includes single-source clusters rendered as the headline list). test:data: 6657/6657 pass (was 6645; +12).
4 tasks
koala73
added a commit
that referenced
this pull request
Apr 24, 2026
… B) (#3366) * feat(energy-atlas): promote Atlas map layers to FULL variant (§R #3 = B) Per plan §R/#3 decision B: the Redis-backed evidence registries (75 gas + 75 oil pipelines, 200 storage facilities, 29 fuel shortages) are now toggleable on the main worldmonitor.app map. Previously they were hardcoded energy-variant-only, and FULL users who toggled `pipelines: true` got the ~20-entry legacy static PIPELINES list. Changes: - `src/components/DeckGLMap.ts`: drop the `SITE_VARIANT === 'energy'` gates at :1511-1541. The pipelines layer now always uses `createEnergyPipelinesLayer()` (Redis-backed evidence registry); `createPipelinesLayer` (legacy static) is left in the file as dead code pending a separate cleanup PR that also retires `src/config/pipelines.ts`. Storage and fuel-shortage layers are now gated only on the variant's `mapLayers.storageFacilities` / `mapLayers.fuelShortages` booleans. - `src/config/panels.ts`: add `storageFacilities: false` + `fuelShortages: false` to FULL_MAP_LAYERS (desktop + mobile) so the keys exist for toggle dispatch; default off so users opt in. - `src/config/map-layer-definitions.ts`: extend the `full` variant's VARIANT_LAYER_ORDER to include `storageFacilities` and `fuelShortages`, so `getAllowedLayerKeys('full')` admits them and the layer picker surfaces them. - `src/config/commands.ts`: add CMD+K toggles `layer:storageFacilities` and `layer:fuelShortages` next to the existing `layer:pipelines`. Finance + commodity variants already had `pipelines: true`; they now render the more comprehensive Redis-backed 150-entry dataset instead of the ~20-entry legacy list. If a variant doesn't want this, they set `pipelines: false` in their MAP_LAYERS config. Part of docs/internal/energy-atlas-registry-expansion.md §R. * fix(energy-atlas): restrict storageFacilities + fuelShortages to flat renderer Reviewer (Codex) found two gaps in PR #3366: 1. GlobeMap 3D toggles did nothing. LAYER_REGISTRY declared both new layers with the default ['flat', 'globe'] renderers, so the toggle showed up in globe mode. But GlobeMap.ts has no rendering support: ensureStaticDataForLayer (:2160) only handles cables/pipelines/etc., and the layer-channel map (:2484) has no entries for either. Users in globe mode saw the toggle and got silent no-ops. 2. SVG/mobile fallback (Map.ts fullLayers at :381) also has no render path for these data types. The existing cyberThreats precedent at :387 documents this as an intentional DeckGL-only pattern. Fix: - Restrict both LAYER_REGISTRY entries to ['flat'] explicitly. The layer picker hides the toggle in globe mode instead of exposing a no-op. Comment points to the GlobeMap gap so a future globe-rendering PR knows what to undo. - Extend the existing cyberThreats note in Map.ts:387 to cover storageFacilities + fuelShortages too, noting they're already hidden from globe mode via the LAYER_REGISTRY restriction. This is the smallest possible fix consistent with the pre-existing pattern. Full globe-mode rendering for these layers is out of scope — tracked separately as a follow-up. * fix(energy-atlas): gate layer:* CMD+K by current renderer + DeckGL state Reviewer follow-up on PR #3366: the previous fix restricted LAYER_REGISTRY renderers to ['flat'] so the globe-mode layer picker hides storageFacilities / fuelShortages toggles. But CMD+K was still callable — SearchModal.matchCommands didn't filter `layer:*` commands by renderer, so a user could CMD+K "storage layer" in globe or SVG mode and trigger a silent no-op. Fix — centralize "can this layer render right now?" in one helper: - Add `deckGLOnly?: boolean` to LayerDefinition. `renderers: ['flat']` is not enough because `'flat'` covers both DeckGL-flat and SVG-flat, and the SVG/mobile fallback has no render path for either layer. Mark both as `deckGLOnly: true`. - New `isLayerExecutable(key, renderer, isDeckGLActive)` helper in map-layer-definitions.ts. Returns true iff renderers include the current renderer AND (if deckGLOnly) DeckGL is active. - `SearchModal.setLayerExecutableFn(fn)`: caller-supplied predicate used in both `matchCommands` (search results) and `renderAllCommandsList` (full picker). - `search-manager` wires the predicate using `ctx.map.isGlobeMode()` + `ctx.map.isDeckGLActive()`, and also adds a symmetric guard in the `layer:` dispatch case so direct activations (keyboard accelerator, programmatic invocation) bail the same way. Pre-existing resilienceScore DeckGL gate at search-manager:494 kept as a belt-and-suspenders — the new isLayerExecutable check already covers it since resilienceScore has `renderers: ['flat']` (though it lacks deckGLOnly). Left the specific check in place to avoid scope creep on a working guard. Typecheck clean, 6694/6694 tests pass. * fix(energy-atlas): filter CMD+K layer commands by variant too Greptile P2 on commit 3f7a400: `layer:storageFacilities` and `layer:fuelShortages` still surface in CMD+K on tech / finance / commodity / happy variants (where they're not in VARIANT_LAYER_ORDER). Renderer + DeckGL filter was passing because those variants run flat DeckGL. Dispatch silently failed at the `variantAllowed` guard in handleCommand (:491), producing an invisible no-op from the user's POV. Fix: extend `setLayerExecutableFn` predicate to also check `getAllowedLayerKeys(SITE_VARIANT).has(key)` before the renderer checks. SearchModal now hides these commands on non-full/non-energy variants where they can't execute. This also cleans up the pre-existing pattern for other variant-specific layer commands flagged by Greptile as "consistent with how other variant-specific layer commands (e.g. layer:nuclear on tech variant) already behave today" — they now all route through the same predicate. * fix(energy-atlas): gate layers:* presets + add isLayerExecutable tests (review P2) Two Codex P2 findings on this PR: 1. `layers:*` presets bypassed the renderer/DeckGL gate. `search-manager.ts:481` checked only `allowed.has(layer)` before flipping a preset layer on. A user in globe mode or on SVG fallback who ran `layers:all` or `layers:infra` would silently set `deckGLOnly` layers (storageFacilities, fuelShortages) to true — toggles with no rendered output, and since the picker hides those layers under the current renderer the user had no way to toggle them back off without switching modes. Fix: funnel presets through the same `isLayerExecutable` predicate per-layer CMD+K already uses. `executable(k)` combines the existing `allowed.has` variant check with the renderer + DeckGL gate, so presets now match the per-layer dispatch behavior exactly. 2. No regression tests for the `deckGLOnly` / `isLayerExecutable` contract, despite it being behavior-critical renderer gating. Fix: added `tests/map-layer-executable.test.mts` — 16 cases: - Flag assertions: storageFacilities + fuelShortages carry `deckGLOnly: true` and renderers: ['flat']. Layers without the flag (pipelines, conflicts, cables) have it `undefined`, not accidentally `false`. - Renderer-gate cases: deckGLOnly layers pass only on flat + DeckGL active, not on SVG fallback, not on globe. Flat-only non-deckGLOnly layers (ciiChoropleth) pass on flat regardless of DeckGL status. Dual-renderer layers (pipelines) pass on both flat and globe. Unknown layer keys return false. - Exhaustive 2×2×2 matrix across (renderer, isDeckGL, deckGLOnly) using representative layer keys for each shape. All 16 new tests pass. Full test:data suite still green. Typecheck clean. * fix(energy-atlas): add pipeline-status to finance + commodity panel sets (review P1) Codex P1: FINANCE_MAP_LAYERS and COMMODITY_MAP_LAYERS both carry `pipelines: true`, and PR #3366 unified all variants on `createEnergyPipelinesLayer` which dispatches `energy:open-pipeline-detail` on row click. The listener for that event lives in PipelineStatusPanel. `PanelLayoutManager.createPanel()` only instantiates panels whose keys are present in `panelSettings`, which derives from FULL_PANELS / FINANCE_PANELS / etc. — so on finance and commodity variants the listener never existed, and pipeline clicks were a silent no-op. Fix: add `pipeline-status` to both FINANCE_PANELS and COMMODITY_PANELS with `enabled: false` (panel slot not auto-opened; users invoke it by clicking a pipeline on the map or via CMD+K). The panel now instantiates on both variants and the click-through works end to end. FULL_PANELS + ENERGY_PANELS already had the key from earlier PRs; no change there. Typecheck clean, test:data 6696/6696 pass.
koala73
added a commit
that referenced
this pull request
Apr 24, 2026
…try sleep Greptile review of PR #3385 flagged two P2s in the Comtrade seeder. Finding #3 (parseComtradeFlowResponse double-count risk): `cmdCode=TOTAL` without a partner filter currently returns only world-aggregate rows in practice — but `parseComtradeFlowResponse` summed every row unconditionally. A future refactor adding per- partner querying would silently double-count (world-aggregate row + partner-level rows for the same year), cutting the derived share in half with no test signal. Fix: explicit `partnerCode ∈ {'0', 0, null/undefined}` filter. Matches current empirical behavior (aggregate-only responses) and makes the construct robust to a future partner-level query. Finding #4 (wasted backoff on final retry): 429 and 5xx branches slept `backoffMs` before `continue`, but on `attempt === RETRY_MAX_ATTEMPTS` the loop condition fails immediately after — the sleep was pure waste. Added early-return (parallel to the existing pattern in the network-error catch branch) so the final attempt exits the retry loop at the first non-success response without extra latency. Tests: - 3 new `parseComtradeFlowResponse` variants: world-only filter, numeric-0 partnerCode shape, rows without partnerCode field - Existing tests updated: the double-count assertion replaced with a "per-partner rows must NOT sum into the world-aggregate total" assertion that pins the new contract Rebased onto updated prereq. Full test suite: 6,890 tests (+2 net).
koala73
added a commit
that referenced
this pull request
Apr 24, 2026
…read (#3385) * feat(seed): BUNDLE_RUN_STARTED_AT_MS env + runSeed SIGTERM cleanup Prereq for the re-export-share Comtrade seeder (plan 2026-04-24-003), usable by any cohort seeder whose consumer needs bundle-level freshness. Two coupled changes: 1. `_bundle-runner.mjs` injects `BUNDLE_RUN_STARTED_AT_MS` into every spawned child. All siblings in a single bundle run share one value (captured at `runBundle` start, not spawn time). Consumers use this to detect stale peer keys — if a peer's seed-meta predates the current bundle run, fall back to a hard default rather than read a cohort-peer's last-week output. 2. `_seed-utils.mjs::runSeed` registers a `process.once('SIGTERM')` handler that releases the acquired lock and extends existing-data TTL before exiting 143. `_bundle-runner.mjs` sends SIGTERM on section timeout, then SIGKILL after KILL_GRACE_MS (5s). Without this handler the `finally` path never runs on SIGKILL, leaving the 30-min acquireLock reservation in place until its own TTL expires — the next cron tick silently skips the resource. Regression guard memory: `bundle-runner-sigkill-leaks-child-lock` (PR #3128 root cause). Tests added: - bundle-runner env injection (value within run bounds) - sibling sections share the same timestamp (critical for the consumer freshness guard) - runSeed SIGTERM path: exit 143 + cleanup log - process.once contract: second SIGTERM does not re-enter handler * fix(seed): address P1/P2 review findings on SIGTERM + bundle contracts Addresses PR #3384 review findings (todos 256, 257, 259, 260): #256 (P1) — SIGTERM handler narrowed to fetch phase only. Was installed at runSeed entry and armed through every `process.exit` path; could race `emptyDataIsFailure: true` strict-floor exits (IMF-External, WB-bulk) and extend seed-meta TTL when the contract forbids it — silently re-masking 30-day outages. Now the handler is attached immediately before `withRetry(fetchFn)` and removed in a try/finally that covers all fetch-phase exit branches. #257 (P1) — `BUNDLE_RUN_STARTED_AT_MS` now has a first-class helper. Exported `getBundleRunStartedAtMs()` from `_seed-utils.mjs` with JSDoc describing the bundle-freshness contract. Fleet-wide helper so the next consumer seeder imports instead of rediscovering the idiom. #259 (P2) — SIGTERM cleanup runs `Promise.allSettled` on disjoint-key ops (`releaseLock` + `extendExistingTtl`). Serialising compounded Upstash latency during the exact failure mode (Redis degraded) this handler exists to handle, risking breach of the 5s SIGKILL grace. #260 (P2) — `_bundle-runner.mjs` asserts topological order on optional `dependsOn` section field. Throws on unknown-label refs and on deps appearing at a later index. Fleet-wide contract replacing the previous prose-comment ordering guarantee. Tests added/updated: - New: SIGTERM handler removed after fetchFn completes (narrowed-scope contract — post-fetch SIGTERM must NOT trigger TTL extension) - New: dependsOn unknown-label + out-of-order + happy-path (3 tests) Full test suite: 6,866 tests pass (+4 net). * fix(seed): getBundleRunStartedAtMs returns null outside a bundle run Review follow-up: the earlier `Math.floor(Date.now()/1000)*1000` fallback regressed standalone (non-bundle) runs. A consumer seeder invoked manually just after its peer wrote `fetchedAt = (now - 5s)` would see `bundleStartMs = Date.now()`, reject the perfectly-fresh peer envelope as "stale", and fall back to defaults — defeating the point of the peer-read path outside the bundle. Returning null when `BUNDLE_RUN_STARTED_AT_MS` is unset/invalid keeps the freshness gate scoped to its real purpose (across-bundle-tick staleness) and lets standalone runs skip the gate entirely. Consumers check `bundleStartMs != null` before applying the comparison; see the companion `seed-sovereign-wealth.mjs` change on the stacked PR. * test(seed): SIGTERM cleanup test now verifies Redis DEL + EXPIRE calls Greptile review P2 on PR #3384: the existing test only asserted exit code + log line, not that the Redis ops were actually issued. The log claim was ahead of the test. Fixture now logs every Upstash fetch call's shape (EVAL / pipeline- EXPIRE / other) to stderr. Test asserts: - >=1 EVAL op was issued during SIGTERM cleanup (releaseLock Lua script on the lock key) - >=1 pipeline-EXPIRE op was issued (extendExistingTtl on canonical + seed-meta keys) - The EVAL body carries the runSeed-generated runId (proves it's THIS run's release, not a phantom op) - The EXPIRE pipeline touches both the canonicalKey AND the seed-meta key (proves the keys[] array was built correctly including the extraKeys merge path) Full test suite: 6,866 tests pass, typecheck clean. * feat(resilience): Comtrade-backed re-export-share seeder + SWF Redis read Plan ref: docs/plans/2026-04-24-003-feat-reexport-share-comtrade-seeder-plan.md Motivating case. Before this PR, the SWF `rawMonths` denominator for the `sovereignFiscalBuffer` dimension used GROSS annual imports for every country. For re-export hubs (goods transiting without domestic settlement), this structurally under-reports resilience: UAE's 2023 $941B of imports include $334B of transit flow that never represents domestic consumption. Net imports = gross × (1 − reexport_share). The previous (PR 3A) design flattened a hand-curated YAML into Redis; the YAML shipped empty and never populated, so the correction never applied and the cohort audit showed no movement. Gap #2 (this PR). Two coupled changes to make the correction actually apply: 1. Comtrade-backed seeder (`scripts/seed-recovery-reexport-share.mjs`). Rewritten to fetch UN Comtrade `flowCode=RX` (re-exports) and `flowCode=M` (imports) per cohort member, compute share = RX/M at the latest co-populated year, clamp to [0.05, 0.95], publish the envelope. Header auth (`Ocp-Apim-Subscription-Key`) — subscription key never reaches URL/logs/Redis. `maxRecords=250000` cap with truncation detection. Sequential + retry-on-429 with backoff. Hub cohort resolved by Phase 0 empirical probe (plan §Phase 0): ['AE', 'PA']. Six candidates (SG/HK/NL/BE/MY/LT) return HTTP 200 with zero RX rows — Comtrade doesn't expose RX for those reporters. 2. SWF seeder reads from Redis (`scripts/seed-sovereign-wealth.mjs`). Swaps `loadReexportShareByCountry()` (YAML) for `loadReexportShareFromRedis()` (Redis key written by #1). Guarded by bundle-run freshness: if the sibling Reexport-Share seeder's `seed-meta` predates `BUNDLE_RUN_STARTED_AT_MS` (set by the prereq PR's `_bundle-runner.mjs` env-injection), HARD fallback to gross imports rather than apply last-month's stale share. Health registries. Both new keys registered in BOTH `api/health.js` SEED_META (60-day alert threshold) and `api/seed-health.js` SEED_DOMAINS (43200min interval). feedback_two_health_endpoints_must_match. Bundle wiring. `seed-bundle-resilience-recovery` Reexport-Share timeout bumped 60s → 300s (Comtrade + retry can take 2-3 min worst-case). Ordering preserved: Reexport-Share before Sovereign- Wealth so the SWF seeder reads a freshly-written key in the same cron tick. Deletions. YAML + loader + 7 obsolete loader tests removed; single source of truth is now Comtrade → Redis. Prereq. Stacks on PR #3384 (feat/bundle-runner-env-sigterm) which adds BUNDLE_RUN_STARTED_AT_MS env injection + runSeed SIGTERM cleanup. This PR's bundle-freshness guard depends on that env variable. Tests (19 new, 7 deleted, +12 net): - Pure math: parseComtradeFlowResponse, computeShareFromFlows, clampShare, declareRecords + credential-leak source scan (15) - Integration (Gap #2 regression guards): SWF seeder loadReexport ShareFromRedis — fresh/absent/malformed/stale-meta/missing-meta (5) - Health registry dual-registry drift guard — scoped to this PR's keys, respecting pre-existing asymmetry (4) - Bundle-ordering + timeout assertions (2) Phase 0 cohort validation committed to plan. Full test suite passes: 6,881 tests. * fix(resilience): address P1/P2 review findings — adopt shared helpers, pin freshness boundary Addresses PR #3385 review findings: #257 (P1) consumer — `seed-sovereign-wealth.mjs` imports the shared `getBundleRunStartedAtMs` helper from `_seed-utils.mjs` (added in the prereq commit) instead of its own `getBundleStartMs`. Single source of truth for the bundle-freshness contract. #258 (P2) — `seed-recovery-reexport-share.mjs` isMain guard uses the canonical `pathToFileURL(process.argv[1]).href === import.meta.url` form instead of basename-suffix matching. Handles symlinks, case- different paths on macOS HFS+, and Windows path separators without string munging. #260 (P2) consumer — Sovereign-Wealth declares `dependsOn: ['Reexport-Share']` in the bundle spec. `_bundle-runner.mjs` (prereq commit) now enforces topological order on load and throws on violation — replaces the previous prose-comment ordering contract. #261 (P2) — added a test to `tests/seed-sovereign-wealth-reads-redis- reexport-share.test.mts` pinning the inclusive-boundary semantic: `fetchedAtMs === bundleStartMs` must be treated as FRESH. Guards against a future refactor to `<=` that would silently reject peers writing at the very first millisecond of the bundle run. Rebased onto updated prereq. Full test suite: 6,886 tests pass (+5 net). * fix(resilience): freshness gate skipped in standalone mode; meta still required Review catch: the previous `bundleStartMs = Date.now()` fallback made standalone/manual `seed-sovereign-wealth.mjs` runs ALWAYS reject any previously-seeded re-export-share meta as "stale" — even when the operator ran the Reexport seeder milliseconds beforehand. Defeated the point of the peer-read path outside the bundle. With `getBundleRunStartedAtMs()` now returning null outside a bundle (companion commit on the prereq branch), the consumer only applies the freshness gate when `bundleStartMs != null`. Standalone runs accept any `fetchedAt` — the operator is responsible for ordering. Two guards survive the change: - Meta MUST exist (absence = peer-outage fail-safe, both modes) - In-bundle: meta MUST be at or after `BUNDLE_RUN_STARTED_AT_MS` Two new tests pin both modes: - standalone: accepts meta written 10 min before this process started - standalone: still rejects missing meta (peer-outage fail-safe survives gate bypass) Rebased onto updated prereq. Full test suite: 6,888 tests (+2 net). * fix(resilience): filter world-aggregate Comtrade rows + skip final-retry sleep Greptile review of PR #3385 flagged two P2s in the Comtrade seeder. Finding #3 (parseComtradeFlowResponse double-count risk): `cmdCode=TOTAL` without a partner filter currently returns only world-aggregate rows in practice — but `parseComtradeFlowResponse` summed every row unconditionally. A future refactor adding per- partner querying would silently double-count (world-aggregate row + partner-level rows for the same year), cutting the derived share in half with no test signal. Fix: explicit `partnerCode ∈ {'0', 0, null/undefined}` filter. Matches current empirical behavior (aggregate-only responses) and makes the construct robust to a future partner-level query. Finding #4 (wasted backoff on final retry): 429 and 5xx branches slept `backoffMs` before `continue`, but on `attempt === RETRY_MAX_ATTEMPTS` the loop condition fails immediately after — the sleep was pure waste. Added early-return (parallel to the existing pattern in the network-error catch branch) so the final attempt exits the retry loop at the first non-success response without extra latency. Tests: - 3 new `parseComtradeFlowResponse` variants: world-only filter, numeric-0 partnerCode shape, rows without partnerCode field - Existing tests updated: the double-count assertion replaced with a "per-partner rows must NOT sum into the world-aggregate total" assertion that pins the new contract Rebased onto updated prereq. Full test suite: 6,890 tests (+2 net).
This was referenced Jul 1, 2026
koala73
added a commit
that referenced
this pull request
Jul 3, 2026
Code-review finding: UnauthorizedError was injected unconditionally, so the 4 all-public services (Leads/Natural/Seismology/Unrest) — whose ops all carry security:[] and no 401 — defined the schema but never referenced it (orphaned component, Spectral oas3-unused-component hint). Gate the injection on the spec having >=1 non-public op (the op that carries the 401 which references it), in both injectJson and injectYamlAuthContract, mirroring the existing ForbiddenError conditional. The bundle (which aggregates non-public ops) and all authenticated per-service specs keep it. Test asserts presence iff a non-public op exists AND absence otherwise (regression guard). Removes UnauthorizedError from 4 JSON + 4 YAML specs; src/generated unchanged; make generate idempotent; contract tests pass. Claude-Session: https://claude.ai/code/session_01EsjtheWqt8GxEZfd3DyTMu
koala73
added a commit
that referenced
this pull request
Jul 3, 2026
…doc-truth (adversarial audit) (#4657) * fix(api): correct OpenAPI sample values, missing 403s, servers URL, and doc-truth (adversarial audit) An adversarial audit of the generated OpenAPI docs (vs the actual gateway + handlers) surfaced real defects the mechanical guards can't catch. Fixed via four disjoint source changes, then one `make generate`: U1 — realistic sample values (scripts/openapi-inject-examples.mjs): the field-name heuristic emitted values handlers reject. Curated overrides sourced from the real accepted sets: chokepointId→'suez' (was 'suez-canal', rejected), scenarioId→'hormuz-tanker-blockade' (was 'oil-price-shock', unregistered), FRED series_id→'GDP' (was 'example-id'), icao24→'a835af' (was 'example'), chokepointIds→['suez']. BLS series_id stays 'USPRIV' (enum-resolved). U2 — missing 403s (scripts/openapi-inject-security.mjs): register-interest (public, throws 403 Turnstile/desktop-auth) now documents its 403 via PUBLIC_FORBIDDEN_GATES; the 11 legacy-Pro PREMIUM_RPC_PATHS not covered by ENDPOINT_ENTITLEMENTS now document the gateway's 403 'Pro subscription required' (ForbiddenError). Entitlement wins on overlap (PREMIUM_ONLY = premium − entitlement). U3 — servers URL (new scripts/openapi-inject-servers.mjs): per-service specs had no `servers`, so Mintlify rendered curl with the api.example.com fallback. Now injects servers:[{url:https://api.worldmonitor.app}] into per-service JSON+YAML (bundle already carries it). Wired into `make generate`. U4 — behavioral doc-truth (9 proto comments): runId (drives `note`, not ignored), webcam lastUpdated (ISO-8601 string), disabled intelligence stubs (company-enrichment/signals), and clamps documented to reality (gdelt cap 20, aircraft-batch 10, seismology default 500/no-clamp, tech-events clampInt ranges). Verified: every defect fixed in the regenerated docs; `make generate` idempotent; src/generated unchanged; 500 contract tests pass (incl. new curated-example, servers, and premium/register-interest 403 guards). Follow-up issues to file: webcam int64 field vs ISO handler, gdelt/aircraft-batch/ seismology/tech-events suspected handler bugs, #4604 required-accuracy gaps (BLS series_id / company effectively-required). Claude-Session: https://claude.ai/code/session_01EsjtheWqt8GxEZfd3DyTMu * fix(api): address Greptile review — webcam field type + scenario regex robustness - webcam GetWebcamImageResponse.last_updated: int64 -> string. The field was int64 but the handler emits new Date(...).toISOString() (an ISO string, which is an INVALID int64), so `format: int64` contradicted the description. The generated TS type was already `string`, so this is zero TS/runtime impact and removes the format/description conflict (resolves #4658 item 1, Greptile P2). - openapi-inject-examples.mjs + test: broaden the scenario-template id regex to match single/double/backtick quotes so a future quote-style change in scenario-templates.ts can't silently fall back to the literal (Greptile P2). Regenerated; webcam lastUpdated now {type:string}; scenario example unchanged (hormuz-tanker-blockade); src/generated unchanged; make generate idempotent; contract tests pass. Claude-Session: https://claude.ai/code/session_01EsjtheWqt8GxEZfd3DyTMu * fix(api): align OpenAPI review follow-ups * fix(api): gate UnauthorizedError injection on non-public ops (review #3) Code-review finding: UnauthorizedError was injected unconditionally, so the 4 all-public services (Leads/Natural/Seismology/Unrest) — whose ops all carry security:[] and no 401 — defined the schema but never referenced it (orphaned component, Spectral oas3-unused-component hint). Gate the injection on the spec having >=1 non-public op (the op that carries the 401 which references it), in both injectJson and injectYamlAuthContract, mirroring the existing ForbiddenError conditional. The bundle (which aggregates non-public ops) and all authenticated per-service specs keep it. Test asserts presence iff a non-public op exists AND absence otherwise (regression guard). Removes UnauthorizedError from 4 JSON + 4 YAML specs; src/generated unchanged; make generate idempotent; contract tests pass. Claude-Session: https://claude.ai/code/session_01EsjtheWqt8GxEZfd3DyTMu * feat(api): document the account-state (#4611) 403 on all authenticated ops (review #4) Code-review finding: the gateway returns 403 'API access requires an active subscription' on ANY non-public keyed route when a user API key resolves to an affirmatively inactive/expired entitlement (gateway.ts:1073-1083, #4611) — a global account-state gate orthogonal to the per-route entitlement/premium gates, previously undocumented on plain authed ops. Document it per-operation (parallel to how the 401 is already documented): - Every non-public op now carries a 403 -> ForbiddenError. Entitlement paths keep 'PRO entitlement access denied', premium-only keep 'Pro subscription required', and the remaining ~156 plain authed ops get the account-state description. - ForbiddenError is now gated on hasNonPublicOp (any authed op references it), same gate as UnauthorizedError; all-public specs carry neither (no orphan). - Applied to JSON + per-service YAML + bundle with full parity; test asserts every non-public op has a ForbiddenError 403 and the schema is present iff non-public. Regenerated; 185/185 non-public ops carry a 403; make generate idempotent; src/generated unchanged; 391 contract tests pass. Claude-Session: https://claude.ai/code/session_01EsjtheWqt8GxEZfd3DyTMu * refactor(api): extract shared OpenAPI codegen module; tests import source of truth (review #2) Code-review theme (security + testing + maintainability): the byte-faithful serializer was copy-pasted across the openapi-inject-* injectors, and the contract tests re-derived the injector's helpers / scraped its source with duplicate regexes — a tautology that catches a dropped injection but not a wrong value, plus latent drift (required.mjs's eq had already diverged). Extract scripts/lib/openapi-codegen.mjs (pure node, zero npm deps — safe in the make-generate context): the serializer (sortRec/goEscape/serialize/eq), normalizeKey, the fail-closed gateway/entitlement/premium source-of-truth parsers, and PUBLIC_FORBIDDEN_GATES. Rewire openapi-inject-{security,examples, servers}.mjs to import it, and rewire the security + examples contract tests to import the SAME parsers/gates/normalizeKey instead of re-scraping the injector — so the test can no longer drift from the injector via a divergent private regex. Pure refactor: make generate is byte-identical (0 diff across docs/api + src/generated), all 3 injectors --check idempotent, contract tests pass. Claude-Session: https://claude.ai/code/session_01EsjtheWqt8GxEZfd3DyTMu
This was referenced Jul 4, 2026
koala73
pushed a commit
that referenced
this pull request
Jul 6, 2026
…vider timeout + harden classification (#4980 review) Addresses the ce-code-review findings on #4980: - #1/#4: raise MARKET_IMPLICATIONS_MIN_RUN_BUDGET_MS 20_000 -> 30_000 (= max provider.timeout 25s + FORECAST_LLM_STAGE_BUDGET_GUARD_MS 5s). At 20s the pre-call guard admitted calls that were then timeout-CAPPED below the provider's own timeout, which is indistinguishable from a genuinely hung provider: a real openrouter timeout in the [20s,30s) band was misclassified as a benign budget starve and its SEED_ERROR suppressed, and 20-25s calls were guaranteed to abort mid-flight (~15s wasted). Admitting only at >=30s gives the provider its full window, so any timeout is attributable and a genuine failure still surfaces SEED_ERROR. - #2: getRemainingForecastLlmBudgetMs now delegates the run-remaining calc to getRemainingForecastLlmRunBudgetMs (was a verbatim duplicate formula). - #3: document the failure-reason classification invariant on the sawProviderFailure / sawBudgetCappedTimeout declarations and the budget-capped-timeout heuristic (comments only, no behavior change). - #5: add tests for the 20-30s guard skip (locks the 30s threshold), the retained mid-call budget_exhausted preserve branch, and its provider_failed symmetry (via the __setForecastLlmCallOverrideForTests seam). - #6: test 1 now sets a real provider key + transport counter so its llmCalls===0 tripwire exercises the guard instead of the !apiKey short-circuit; removed the now-unused budgetStarveFetch helper. test 4 deadline bumped 25s -> 35s to keep passing the new 30s guard. Tests: 605 pass across all seed-forecasts.mjs consumers; biome + unicode clean. Claude-Session: https://claude.ai/code/session_017odw3Pf9ue8RZzxYAQ37P8
koala73
added a commit
that referenced
this pull request
Jul 6, 2026
…ved of the shared LLM run budget (#4978) (#4980) * fix(forecast): stop false SEED_ERROR when market_implications is starved of the shared LLM run budget (#4978) market_implications is the LAST forecast LLM stage (afterPublish) and shares the single 150s run budget with every upstream stage. When upstream stages are slow (e.g. deepseek-v4-flash breaching its 25s call timeout on combined/ scenario, #4944), they drain that budget before this tail stage runs; callForecastLLM then throws a budget error and returns null. Pre-fix the caller treated that starve identically to a real LLM failure and wrote a status:'error' seed-meta, so /api/health flipped to SEED_ERROR for benign, self-healing resource contention (observed 2026-07-06 14:03; self-healed 14:15). Two changes: - Distinguish a budget-starve from a real failure. On starve (pre-call guard, or a null result with the run budget now <=0), preserve last-good WITHOUT rewriting seed-meta.fetchedAt, so age-based STALE_SEED (maxStaleMin=120) still escalates if the starve persists past 2h (gpsjam preserve-last-good design). Genuine provider failures with budget remaining still surface SEED_ERROR. - Reorder: run market_implications BEFORE the best-effort telemetry in afterPublish (history + deep-forecast snapshots, ~20s R2 trace export) so that wall-clock can't push the tail stage past the run deadline. This recovers the ~20s that pushed it over the 150s deadline in the failing run. Root trigger (deepseek-v4-flash latency draining the shared budget) is #4944 territory; a complementary combined-stage groq fallback is a 1-line Railway env change (FORECAST_LLM_COMBINED_PROVIDER_ORDER=openrouter,groq), left out of code to respect the intentional #4944 pin. Tests: tests/market-implications-budget-starve.test.mjs (starve preserves last-good + no error meta; genuine failure still errors) + an afterPublish ordering guard in market-implications-seed-health.test.mjs. Failing-test-first per Bug Fix Protocol. Claude-Session: https://claude.ai/code/session_019uhnqAKEHTws3QMaUvq58N * fix(forecast): preserve market implication failure signals Address PR #4980 review feedback by keeping provider failures distinct from run-budget starvation and restoring stale OK market-implications meta from last-good payloads during starved runs. Also updates the budget-starve tests so the provider-failure regression performs a real provider request. * fix(forecast): raise market_implications budget guard to the full provider timeout + harden classification (#4980 review) Addresses the ce-code-review findings on #4980: - #1/#4: raise MARKET_IMPLICATIONS_MIN_RUN_BUDGET_MS 20_000 -> 30_000 (= max provider.timeout 25s + FORECAST_LLM_STAGE_BUDGET_GUARD_MS 5s). At 20s the pre-call guard admitted calls that were then timeout-CAPPED below the provider's own timeout, which is indistinguishable from a genuinely hung provider: a real openrouter timeout in the [20s,30s) band was misclassified as a benign budget starve and its SEED_ERROR suppressed, and 20-25s calls were guaranteed to abort mid-flight (~15s wasted). Admitting only at >=30s gives the provider its full window, so any timeout is attributable and a genuine failure still surfaces SEED_ERROR. - #2: getRemainingForecastLlmBudgetMs now delegates the run-remaining calc to getRemainingForecastLlmRunBudgetMs (was a verbatim duplicate formula). - #3: document the failure-reason classification invariant on the sawProviderFailure / sawBudgetCappedTimeout declarations and the budget-capped-timeout heuristic (comments only, no behavior change). - #5: add tests for the 20-30s guard skip (locks the 30s threshold), the retained mid-call budget_exhausted preserve branch, and its provider_failed symmetry (via the __setForecastLlmCallOverrideForTests seam). - #6: test 1 now sets a real provider key + transport counter so its llmCalls===0 tripwire exercises the guard instead of the !apiKey short-circuit; removed the now-unused budgetStarveFetch helper. test 4 deadline bumped 25s -> 35s to keep passing the new 30s guard. Tests: 605 pass across all seed-forecasts.mjs consumers; biome + unicode clean. Claude-Session: https://claude.ai/code/session_017odw3Pf9ue8RZzxYAQ37P8 --------- Co-authored-by: Elie Habib <[email protected]>
koala73
added a commit
that referenced
this pull request
Jul 6, 2026
…erity coalesce key + test locks (#4985) Addresses the ce-code-review findings on PR #4985: - #2: extract the byte-identical dedup-material ternary from ais-relay.cjs, seed-aviation.mjs, and notification-relay.cjs into a single source of truth scripts/shared/notification-dedup.cjs::buildDedupMaterial (require from CJS, ESM import from the seeder). Behavior-preserving. - #1 + #3: aviation coalesce key aviation:delay:${iata} -> aviation:closure:${iata}:${severity} — renames the prefix to match the aviation_closure eventType and the notam:closure sibling, and folds the severity band into the key so a post-recovery MAJOR->SEVERE re-escalation re-notifies within the 4h window (mirrors marketAlertCoalesceKey). NOTAM key left airport-only; closure-subject coalescing by airport is the PR's stated intent. - #4: lock the marketAlertCoalesceKey stableIdentifier normalization line in the coalesce test, and assert all three publishers use the shared helper. Validation: node --test tests/notification-relay-coalesce-key.test.mjs (21 pass), tests/notification-relay-payload-audit.test.mjs (7 pass), node --check on all four scripts, CJS require + ESM import interop verified. Claude-Session: https://claude.ai/code/session_01MNFwKS7tAevgd7u1v8Dp49
koala73
added a commit
that referenced
this pull request
Jul 6, 2026
* fix(notifications): coalesce repeated alert events * fix(notifications): review fixes — shared dedup helper + aviation severity coalesce key + test locks (#4985) Addresses the ce-code-review findings on PR #4985: - #2: extract the byte-identical dedup-material ternary from ais-relay.cjs, seed-aviation.mjs, and notification-relay.cjs into a single source of truth scripts/shared/notification-dedup.cjs::buildDedupMaterial (require from CJS, ESM import from the seeder). Behavior-preserving. - #1 + #3: aviation coalesce key aviation:delay:${iata} -> aviation:closure:${iata}:${severity} — renames the prefix to match the aviation_closure eventType and the notam:closure sibling, and folds the severity band into the key so a post-recovery MAJOR->SEVERE re-escalation re-notifies within the 4h window (mirrors marketAlertCoalesceKey). NOTAM key left airport-only; closure-subject coalescing by airport is the PR's stated intent. - #4: lock the marketAlertCoalesceKey stableIdentifier normalization line in the coalesce test, and assert all three publishers use the shared helper. Validation: node --test tests/notification-relay-coalesce-key.test.mjs (21 pass), tests/notification-relay-payload-audit.test.mjs (7 pass), node --check on all four scripts, CJS require + ESM import interop verified. Claude-Session: https://claude.ai/code/session_01MNFwKS7tAevgd7u1v8Dp49 * fix(relay): COPY scripts/shared/notification-dedup.cjs into Dockerfile.relay (#4985) The shared buildDedupMaterial helper extracted in the previous commit is require()'d by scripts/ais-relay.cjs (the relay entrypoint), so the relay Docker image must COPY it or the container crashes at startup with ERR_MODULE_NOT_FOUND. Caught by tests/dockerfile-relay-imports.test.mjs in the full CI unit suite (the local pre-push runs only the changed test file). Claude-Session: https://claude.ai/code/session_01MNFwKS7tAevgd7u1v8Dp49 * fix(seed-aviation): diff aviation prev-state by airport+severity so escalations actually re-notify (#4985 review P1) The prior commit made the aviation coalesce key severity-aware (aviation:closure:${iata}:${severity}) so a MAJOR->SEVERE escalation could re-notify. But dispatchAviationNotifications still diffed previous state by airport ALONE (`!prevSet.has(a.iata)`), which filters an already-alerted airport's escalation out upstream — before the severity-aware coalesce key is ever reached. Net: the intended high->critical escalation never published for the common continuous case. The two dedup layers used mismatched identities. Fix: key the prev-alerted set and the newAlerts diff by the SAME airport+severity identity the coalesce key uses (aviationAlertKey = `${iata}:${band}`), via a shared aviationSeverityBand helper reused by the loop's severity too. Behavior: - MAJOR->SEVERE escalation for an already-alerted airport now publishes (goal). - SEVERE->MAJOR de-escalation also re-notifies once (as 'high') — symmetric identity; a downgrade is a distinct, informative state and the 4h publisher TTL still prevents same-band spam. - One-time on deploy: prevSet holds the old iata-only format, so currently-severe airports re-notify once (bounded to slice(0,3)); self-heals next tick (24h TTL). NOTAM is unchanged: its coalesce key and prev-state are both ICAO-only (severity is constant 'high'), so no identity mismatch there. Repro + proof: tests/notification-relay-coalesce-key.test.mjs gains a source guard asserting the airport+severity diff identity (failed before this fix, passes after). node --test: 22 pass; payload-audit 7 pass; node --check clean. Claude-Session: https://claude.ai/code/session_01MNFwKS7tAevgd7u1v8Dp49
koala73
added a commit
that referenced
this pull request
Jul 7, 2026
…arden allSettledWithConcurrency (PR #4995 review) Addresses the PR #4995 review: #1 (P2) — the concurrency regression compared explicit 1 vs 6 and never exercised the production default. Added a deterministic max-in-flight test that calls fetchBigMacPrices with NO concurrency override and asserts exactly 6 EXA calls are in flight, so a regression of the EXA_CONCURRENCY default toward sequential fails the suite. The timing test now also uses the default (no override). #2 (P2) — added a total-EXA-outage test: every row available:false, empty cheapest/most-expensive, and declareRecords(data) === 0 (the recordCount contract that drives runSeed's no-publish/retry path rather than publishing 0 records). #3 (P3) — allSettledWithConcurrency is now exported, but Math.min(concurrency, len) let an invalid concurrency (0 / NaN / negative / float) start zero workers and return a sparse, all-unprocessed array. Clamp to a finite integer >=1 (degrade to sequential) and added invalid-concurrency + empty-input coverage. Backward compatible: fetchCoinPaprikaTickersById already passed clamped ints (its test still passes). Claude-Session: https://claude.ai/code/session_01XKZZy7bzUNTZeJDVWgXbjj
koala73
added a commit
that referenced
this pull request
Jul 7, 2026
…ne crashes (#4994) (#4995) * fix(seed-bigmac): parallelize the 50-country EXA loop to stop 240s-deadline crashes (#4994) The Big Mac seeder fetched all 50 countries SEQUENTIALLY under runSeed's 240s fetch-phase deadline. Whenever EXA latency exceeded ~4.8s/country (240s/50) the run breached the deadline and exited 75 — a spurious "Deploy Crashed!" alert every tick, no data lost but 240s of compute burned. Run the country loop with BOUNDED CONCURRENCY (6) via the existing allSettledWithConcurrency helper (now exported from _seed-utils.mjs). Worst case is ceil(50/6)=9 waves x 15s ~= 135s, comfortably under the deadline. Per-country failures already degrade to available:false, so the run now exits 0 with partial data; only a total EXA outage trips the graceful path. (The old 150ms inter-call throttle is dropped — the concurrency cap of 6 is the rate limiter.) Also guards the seeder's top-level execution behind an isMain check so the core is importable, and adds tests/bigmac-seed.test.mjs (concurrency regression + country-order preservation + single-country-failure isolation). Claude-Session: https://claude.ai/code/session_01XKZZy7bzUNTZeJDVWgXbjj * test(seed-bigmac): pin default concurrency + total-outage contract; harden allSettledWithConcurrency (PR #4995 review) Addresses the PR #4995 review: #1 (P2) — the concurrency regression compared explicit 1 vs 6 and never exercised the production default. Added a deterministic max-in-flight test that calls fetchBigMacPrices with NO concurrency override and asserts exactly 6 EXA calls are in flight, so a regression of the EXA_CONCURRENCY default toward sequential fails the suite. The timing test now also uses the default (no override). #2 (P2) — added a total-EXA-outage test: every row available:false, empty cheapest/most-expensive, and declareRecords(data) === 0 (the recordCount contract that drives runSeed's no-publish/retry path rather than publishing 0 records). #3 (P3) — allSettledWithConcurrency is now exported, but Math.min(concurrency, len) let an invalid concurrency (0 / NaN / negative / float) start zero workers and return a sparse, all-unprocessed array. Clamp to a finite integer >=1 (degrade to sequential) and added invalid-concurrency + empty-input coverage. Backward compatible: fetchCoinPaprikaTickersById already passed clamped ints (its test still passes). Claude-Session: https://claude.ai/code/session_01XKZZy7bzUNTZeJDVWgXbjj * test(seed-bigmac): silence seeder console + drop flaky wall-clock assertion (fix CI unit flake) The prior review-fix commit passed locally under `node --test` but failed the CI `unit` job (tsx --test --test-concurrency=16). Root cause was NOT a logic error: the file was killed mid-run and node:test's own child-process message parser crashed (FileTest.parseMessage). The seeder logs one line per country (×50) plus per-failure warnings; the new total-outage test emits 50 console.warn in a single SYNCHRONOUS burst (allFail throws with no setTimeout yield), and under 16-way concurrency that output flood corrupts the runner's IPC stream. Fix: - Silence console.log/warn/error for the duration of this test file (before/after hooks, restored afterward). Pure test-side change; also de-noises the CI log. - Drop the wall-clock "concurrent < 1/3 of sequential" test. Concurrency is already proven deterministically by the max-in-flight test (asserts exactly 6 in flight at the production default); a timing-ratio assertion is inherently flaky under --test-concurrency=16. Verified: 0 leaked seeder log lines, 10/10 green across repeated + concurrent runs. Claude-Session: https://claude.ai/code/session_01XKZZy7bzUNTZeJDVWgXbjj
koala73
added a commit
that referenced
this pull request
Jul 7, 2026
…-aware (PR #5003 review) Addresses the review of PR #5003 (both P1s + both P2s): #1 (P1) retries strand the fallback: each provider runs under withRetry(3) = 4 attempts, so openrouter's retries drained the run budget and groq was never reached (reviewer reproduced: OpenRouter called twice, Groq never, SEED_ERROR). A static reservation can't cover the retry chain (4×25 + 4×20 = 180s > the 150s budget). Fix: add a per-call `maxRetries` option to callForecastLLM and pass `maxRetries: 0` for market_implications — one attempt per provider, so a slow primary falls straight through to the fallback. New regression asserts openrouter=1, groq=1 (red without it). #2 (P1) reservation ignored provider-order overrides + runnable keys: replaced the static all-provider sum with getMarketImplicationsMinRunBudgetMs(llmOptions), which sums the RESOLVED, key-filtered chain + stage guard. A single-provider override now reserves 30s (not 50s) so a 40s budget admits it; a no-key chain reserves only the guard so a genuine outage surfaces SEED_ERROR instead of hiding as a starve. New regression covers the single-provider-admitted case. #3 (P2) test hygiene: ENV_KEYS now saves/restores GROQ_API_KEY and the global FORECAST_LLM_PROVIDER_ORDER; default-chain tests clear both order envs. #4 (P2) updated the stale "MIN_RUN_BUDGET >= 30s" comment to the chain reservation. Tests: 10/10 in market-implications-budget-starve (both new regressions red without their respective fix), plus seed-health (4) and cache-guard (5) green. Claude-Session: https://claude.ai/code/session_01XKZZy7bzUNTZeJDVWgXbjj
koala73
added a commit
that referenced
this pull request
Jul 7, 2026
…et_implications' groq fallback isn't stranded (#4978 follow-up) (#5003) * fix(forecast): reserve run-budget for the full provider chain so market_implications' groq fallback isn't stranded (#4978 follow-up) marketImplications flipped /api/health to WARNING (SEED_ERROR) intermittently. Root cause (confirmed from a live seed-forecasts run log): market_implications is the tail LLM stage under the shared 150s run budget, provider order openrouter→groq. #4978's admission guard reserved budget for only the PRIMARY attempt (max provider timeout + guard = 30s). When deepseek-v4-flash timed out (~25s), the run budget was drained and the groq FALLBACK was stranded ("groq llm budget exhausted") — so a recoverable timeout was misreported as SEED_ERROR. Fix: reserve the ENTIRE provider chain in the admission guard — sum(provider.timeout) + stage guard (25s + 20s + 5s = 50s), computed from FORECAST_LLM_PROVIDERS so it tracks the config. An admitted call can now exhaust the primary AND still run the fallback; below that it skips and preserves last-good (green) rather than attempting a chain it can't finish. Keeps deepseek primary (per the #4944 migration). Genuine both-providers-failed still SEED_ERRORs. Tests: new reproduction (40s budget covers the primary but not the chain → skips, no SEED_ERROR — fails on the old 30s constant, passes on the fix). Existing tests that intended admission were recalibrated from 35-40s to 60s for the new threshold. Claude-Session: https://claude.ai/code/session_01XKZZy7bzUNTZeJDVWgXbjj * fix(forecast): make market_implications budget guard retry- and chain-aware (PR #5003 review) Addresses the review of PR #5003 (both P1s + both P2s): #1 (P1) retries strand the fallback: each provider runs under withRetry(3) = 4 attempts, so openrouter's retries drained the run budget and groq was never reached (reviewer reproduced: OpenRouter called twice, Groq never, SEED_ERROR). A static reservation can't cover the retry chain (4×25 + 4×20 = 180s > the 150s budget). Fix: add a per-call `maxRetries` option to callForecastLLM and pass `maxRetries: 0` for market_implications — one attempt per provider, so a slow primary falls straight through to the fallback. New regression asserts openrouter=1, groq=1 (red without it). #2 (P1) reservation ignored provider-order overrides + runnable keys: replaced the static all-provider sum with getMarketImplicationsMinRunBudgetMs(llmOptions), which sums the RESOLVED, key-filtered chain + stage guard. A single-provider override now reserves 30s (not 50s) so a 40s budget admits it; a no-key chain reserves only the guard so a genuine outage surfaces SEED_ERROR instead of hiding as a starve. New regression covers the single-provider-admitted case. #3 (P2) test hygiene: ENV_KEYS now saves/restores GROQ_API_KEY and the global FORECAST_LLM_PROVIDER_ORDER; default-chain tests clear both order envs. #4 (P2) updated the stale "MIN_RUN_BUDGET >= 30s" comment to the chain reservation. Tests: 10/10 in market-implications-budget-starve (both new regressions red without their respective fix), plus seed-health (4) and cache-guard (5) green. Claude-Session: https://claude.ai/code/session_01XKZZy7bzUNTZeJDVWgXbjj
This was referenced Jul 7, 2026
koala73
added a commit
that referenced
this pull request
Jul 12, 2026
…5243 review) P1 (correctness) — a stale post-deadline sample survived feed recovery and was scored over the fresh quote. samplePendingEntries stamped samples with the CYCLE time, so a stale kept-warm reading got a post-deadline ts; the next fresh cycle then (a) skipped sampling (a post-deadline sample already existed) and (b) had the at-deadline path prefer that stored stale value. Now samples are stamped with the SOURCE observation time (asOf) via a new extractMetricObservation, so a stale reading carries a pre-deadline ts, is never selected, and never blocks the fresh quote. Two-cycle stale→fresh regression added. P2 #2 — a partial refresh that dropped a symbol VOIDed the bet immediately (missing record → gate fell through → no_establishable_metric). Now a present feed with an absent matched record pends through the settlement grace and VOIDs only if it never returns. P2 #3 — the seeder could mint a newly dated bet from a stale kept-warm envelope. A per-feed generation freshness contract (commodities: 5d, tolerates any weekend/holiday) drops a feed whose _seed.fetchedAt is beyond it, so no bet is generated from stale prices. Also: the settlement gate is now scoped to POINT windows (at-deadline) — a within-horizon spec resolves from the asOf-stamped sample timeline, not the current feed record, and must not be gated on it (caught by an existing test). 876/876 forecast+bet green; api tsc exit 0. Refs #5233, #5243 Claude-Session: https://claude.ai/code/session_01StNurp4TGC3JLHbTtKJhbp
This was referenced Jul 13, 2026
koala73
added a commit
that referenced
this pull request
Jul 24, 2026
* feat(activation): pure pro-activation state core — mount decision, step model, fire-once keying (U1) Claude-Session: https://claude.ai/code/session_01VUcnpsWficDVsPmEZEUJP7 * feat(activation): interstitial shell — overlay, step chrome, focus trap, exit summary, en copy (U3) Claude-Session: https://claude.ai/code/session_01VUcnpsWficDVsPmEZEUJP7 * feat(activation): brief/alerts/power step wiring + finish-setup chip (U4-U6) Brief: atomic setNotificationConfig with explicit hour+IANA tz, insights world-brief preview, inline hour select. Alerts: pre-denied blocked state, patch-not-clobber channels, cadence-honest copy. Power: injected deep links + R8 settings pointer. Chip: versioned-key dismissal. Claude-Session: https://claude.ai/code/session_01VUcnpsWficDVsPmEZEUJP7 * feat(activation): checkout-return marker + post-reload boot mount hook (U2) Marker written before clearCheckoutAttempt on the success branch only; mount decision evaluated off the boot critical path with bounded snapshot-retry. Surfaces subscriptionId/currentPeriodStart on getSubscriptionForUser (additive, existing columns) for the fire-once key — accepted plan deviation. Claude-Session: https://claude.ai/code/session_01VUcnpsWficDVsPmEZEUJP7 * fix(activation): re-arm mount retry on subscription snapshot changes too With the real subscription snapshot as the fire-once key input, a boot with live entitlement but a not-yet-loaded subscription snapshot would stall in 'keep' forever watching entitlement only. Claude-Session: https://claude.ai/code/session_01VUcnpsWficDVsPmEZEUJP7 * feat(activation): proActivation locale fan-out — 24 locales, register-calibrated (fa per its file convention) Claude-Session: https://claude.ai/code/session_01VUcnpsWficDVsPmEZEUJP7 * feat(activation): funnel telemetry + end-to-end spec (U7) Typed Umami events with whitelisted minimized payloads (planKey/step/exit counts — never billing identifiers); single entered fire site at mount; 7 Playwright scenarios green against the dev server. Claude-Session: https://claude.ai/code/session_01VUcnpsWficDVsPmEZEUJP7 * refactor(activation): simplify pass — shared focus-trap util, leaf record parsers, type reuse, idle-handle cleanup Claude-Session: https://claude.ai/code/session_01VUcnpsWficDVsPmEZEUJP7 * fix(review): apply findings #1-4, #8-12, #14 — hour-capture ref, seeded alerts fallback, preview tri-state, coverage locks P0 #1: digest hour captured in a closure ref so the in-flight re-render can't wipe the user's pick. P1 #2: alerts catch-path seeds from flow context (+email when brief confirmed) instead of empty — convex channels field is full-replace. P1 #3 + P2 #8-12: failed-state e2e, chip assertion, expired-Pro branch, payload combo, catalog-derived drift guard, focus-trap unit tests. P2 #4 tri-state preview guard. P3 #14 unexported helper. Claude-Session: https://claude.ai/code/session_01VUcnpsWficDVsPmEZEUJP7 * fix(review): apply findings #6, #7, #13 — account-scoped records, cross-tab mount claim Marker/fire-once/chip records carry the Clerk userId; foreign-user markers never mount (and are left for the buyer, TTL-reaped); unscoped markers are bound to the first resolved session that observes them. Multi-tab mounts serialize through a nonce claim with a 10s TTL. 92/92 unit, 8/8 e2e. Claude-Session: https://claude.ai/code/session_01VUcnpsWficDVsPmEZEUJP7 * docs: refresh component/service counts for pro-activation modules (docs-stats gate) Claude-Session: https://claude.ai/code/session_01VUcnpsWficDVsPmEZEUJP7 * fix(activation): address greptile review — preserve state on partial failure + product-id guard - Brief digest cadence: buildBriefDigestPayload no longer forces 'daily' when an enabled weekly/twice_daily rule exists without a verified email channel; it omits cadence fields so the relay's field-guarded write preserves the schedule. - Failed channel reads: readActivationContext now flags channelsKnown; confirm writes omit the channels field on an untrusted (failed) read so the relay preserves existing Telegram/Slack/etc. instead of replacing with an empty set. - Fire-once ordering: persist fire-once + clear the marker only AFTER the interstitial opens; an import/init failure leaves the marker for a later retry (double-mount still prevented by the in-session latch + cross-tab claim). - Product-id guard: derive PRO_PRODUCT_IDS from DODO_PRODUCTS (no raw pdt_ literal); exclude the e2e test dir like tests/. Addresses greptile P1 threads on #5534. Claude-Session: https://claude.ai/code/session_01VUcnpsWficDVsPmEZEUJP7 * fix(activation): keep pro-activation leaf import-free; exclude it from product-id guard Importing DODO_PRODUCTS into the leaf dragged the checkout-only product catalog into the eager dashboard entry chunk (the leaf is statically imported by panel-layout), failing the eager-chunk budget test. Revert to mirrored literals kept in sync by the drift-guard test, and exclude the leaf from the raw-pdt_ guard instead (the drift-guard provides the catalog-sync the guard wants). Claude-Session: https://claude.ai/code/session_01VUcnpsWficDVsPmEZEUJP7 * fix(activation): make onboarding account-safe
8 tasks
koala73
added a commit
that referenced
this pull request
Jul 26, 2026
… mutation-proof Code review of #5605 found the outcome signal wrong in both directions and all three new regression guards passing with the bug restored (proven by execution). Addresses every finding. Classification (#1, #5, #8) — `lastAttemptedProvider === 'none'` conflated causes: - too quiet: canAttemptServerSummarization() denies both an anon/free principal AND an entitled one inside the 403/429 cooldown, so a paying user's live entitlement outage was demoted to console.debug for up to 15 min (24 h via Retry-After). trackLLMFailure is a no-op and no captureConsoleIntegration exists, so that was the only signal — the #5600 shape. summarize-gate now exports isServerSummarizationSuppressed() and the chain records which denial it hit. - too loud: CircuitBreaker.execute returns its default WITHOUT running the callback while on cooldown (breaker defaults maxFailures=2, cooldown=5min), so a chain that contacted nobody still reported "All providers failed". Marking now happens INSIDE the breaker callback, and a short-circuit is recorded as its own cause. - the quiet path no longer claims "using designed fallback"; reaching it means browser T5 never ran and callers render an error/unavailable state. Guards (#2, #3, #6) — each of these was green with the bug restored: - indexOf matched the gate name inside a COMMENT, so moving the mark above the gate passed. Source assertions now strip comments; more importantly the mark lives inside the breaker callback, making the ordering structural. - the locality assertion ran against the whole file, so hoisting the state to module scope passed. Now scoped to generateSummary with a creation-count pin and a broadened module-global check. - the raw-warn regex only matched quoted literals while this file's idiom is a template literal. Now delimiter-agnostic. - slice anchors are asserted to resolve; a renamed anchor made slice(a, -1) silently widen to end-of-file. The decision surface moved into a pure classifier covered by an executed truth table rather than grep. Verified: all three original bypasses now fail (32/32 green, 31/32 with each mutation applied); typecheck and biome clean. Claude-Session: https://claude.ai/code/session_018XDm48Kzuv1GQe4PR1qimE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
history.replaceStatefor debounced URL updates.Copy Link).Description
src/utils/urlState.tswithparseMapUrlStateandbuildMapUrlto serialize/deserializeview,zoom,lat,lon,timeRange, andlayersto query params.MapComponent(src/components/Map.ts) to exposegetState,getCenter,setZoom,setCenter,setLayers, and aonStateChangedcallback, plus minor UI sync helpers for layer/time buttons.src/App.tsusingparseMapUrlState, and add debounced syncing to the address bar viasetupUrlStateSyncwithhistory.replaceState.Copy Linkbutton and clipboard helper plus CSS (src/styles/main.css) to let users copy the current shareable URL with brief feedback.Testing
npm run dev) and confirmed Vite served the app (server started successfully).http://127.0.0.1:4173/and captured a screenshot to verify the header UI including the newCopy Linkbutton, which completed successfully.Codex Task