fix(header): use www.worldmonitor.app instead of bare worldmonitor.app#5
Closed
fuleinist wants to merge 1 commit into
Closed
fix(header): use www.worldmonitor.app instead of bare worldmonitor.app#5fuleinist wants to merge 1 commit into
fuleinist wants to merge 1 commit into
Conversation
… World nav link worldmonitor.app does not resolve DNS (bare domain has no A/AAAA records). All other subdomains (tech., finance., happy., commodity., energy.) resolve fine, but clicking the World nav link from any variant lands on a dead domain. Fix: change the World variant href from https://worldmonitor.app to https://www.worldmonitor.app (which resolves with HTTP 200 and 301-redirects to www). Fixes koala73#3339
fuleinist
pushed a commit
that referenced
this pull request
Apr 24, 2026
…ntryDeepDive row (§R #5 = B) (koala73#3377) * feat(energy-atlas): seed-side countries[] denorm + CountryDeepDive row (§R #5 = B) Per plan §R/#5 decision B: denormalise countries[] at seed time on each disruption event so CountryDeepDivePanel can filter events per country without an asset-registry round trip. Schema join (pipeline/storage → event.assetId) happens once in the weekly cron, not on every panel render. The alternative (client-side join) was rejected because it couples UI logic to asset-registry internals and duplicates the join for every surface that wants a per-country filter. Changes: - `proto/.../list_energy_disruptions.proto`: add `repeated string countries = 15` to EnergyDisruptionEntry with doc comment tying it to the plan decision and the always-non-empty invariant. - `scripts/_energy-disruption-registry.mjs`: • Load pipeline-gas + pipeline-oil + storage-facilities registries once per seed cycle; index by id. • `deriveCountriesForEvent()` resolves assetId to {fromCountry, toCountry, transitCountries} (pipeline) or {country} (storage), deduped + alpha-sorted so byte-diff stability holds. • `buildPayload()` attaches the computed countries[] to every event before writing. • `validateRegistry()` now requires non-empty countries[] of ISO2 codes. Combined with the seeder's `emptyDataIsFailure: true`, this surfaces orphaned assetIds loudly — the next cron tick fails validation and seed-meta stays stale, tripping health alarms. - `scripts/data/energy-disruptions.json`: fix two orphaned assetIds that the new join caught: • `cpc-force-majeure-2022`: `cpc-pipeline` → `cpc` (matches the entry in pipelines-oil.json). • `pdvsa-designation-2019`: `ve-petrol-2026-q1` (non-existent) → `venezuela-anzoategui-puerto-la-cruz`. - `server/.../list-energy-disruptions.ts`: project countries[] into the RPC response via coerceStringArray. Legacy pre-denorm rows surface as empty array (always present on wire, length 0 => old). - `src/components/CountryDeepDivePanel.ts`: add 4th Atlas row — "Energy disruptions in {iso2}" — filtered by `iso2 ∈ countries[]`. Failure is silent; EnergyDisruptionsPanel (upcoming) is the primary disruption surface. - `tests/energy-disruptions-registry.test.mts`: switch to validating the buildPayload output (post-denorm), add §R #5 B invariant tests, plus a raw-JSON invariant ensuring curators don't hand-edit countries[] (it's derived, not declared). Proto regen note: `make generate` currently fails with a duplicate openapi plugin collision in buf.gen.yaml (unrelated bug — 3 plugin entries emit to the same out dir). Worked around by temporarily trimming buf.gen.yaml to just the TS plugins for this regen. Added only the `countries: string[]` wire field to both service_client and service_server; no other generated-file drift in this PR. * chore(proto): regenerate openapi specs for countries[] field Runs `make generate` with the sebuf v0.11.1 plugin now correctly resolved via the PATH fix (cherry-picked from fix/makefile-generate-path-prefix). The new `countries` field on EnergyDisruptionEntry propagates into: - docs/api/SupplyChainService.openapi.yaml (primary per-service spec) - docs/api/SupplyChainService.openapi.json (machine-readable variant) - docs/api/worldmonitor.openapi.yaml (consolidated bundle) No TypeScript drift beyond the already-committed service_client.ts / service_server.ts updates in 80797e7. * fix(energy-atlas): drop highlightEventId emission (review P2) Codex P2: loadDisruptionsForCountry dispatched `highlightEventId` but neither PipelineStatusPanel nor StorageFacilityMapPanel consumes it (the openDetailHandler reads only pipelineId / facilityId). The UI's implicit promise (event-specific highlighting) wasn't delivered — clickthrough was asset-generic, and the extra wire field was a misleading API surface. Fix: emit only {pipelineId, facilityId} in the dispatched detail. Row click opens the asset drawer; user sees the full per-asset disruption timeline and locates the event visually. Symmetric fix for PR koala73#3378's EnergyDisruptionsPanel — both emitters now match the drawer contract exactly. Re-add `highlightEventId` here when the drawer panels ship matching consumer code (openDetailHandler accepts it, loadDetail stores it, renderDisruptionTimeline scrolls + emphasises the matching event). Typecheck clean, test:data 6698/6698 pass. * fix(energy-atlas): collision detection + abort signal + label clamp (review P2) Three Codex P2 findings on PR koala73#3377: 1. `loadAssetRegistries()` spread-merged gas + oil pipelines, silently overwriting entries on id collision. No collision today, but a curator adding a pipeline under the same id to both files would cause `deriveCountriesForEvent` to return wrong-commodity country data with no test flagging it. Fix: explicit merge loop that throws on duplicate id. The next cron tick fails validation, seed-meta stays stale, health alarms fire — same loud-failure pattern the rest of the seeder uses. 2. `loadDisruptionsForCountry` didn't thread `this.signal` through the RPC fetch shim. The stale-closure guard (`currentCode !== iso2`) discarded stale RESULTS, but the in-flight request couldn't be cancelled when the user switched countries or closed the panel. Fix: wrap globalThis.fetch with { signal: this.signal } in the client factory, matching the signal lifecycle the rest of the panel already uses. 3. `shortDescription` values up to 200 chars rendered without ellipsis in the compact Atlas row, overflowing the row layout. Fix: new `truncateDisruptionLabel` helper clamps to 80 chars with ellipsis. Full text still accessible via click-through to the asset drawer. Typecheck clean, test:data 6698/6698 pass.
fuleinist
pushed a commit
that referenced
this pull request
Apr 25, 2026
…lead divergence (koala73#3396) * feat(brief-llm): canonical synthesis prompt + v3 cache key Extends generateDigestProse to be the single source of truth for brief executive-summary synthesis (canonicalises what was previously split between brief-llm's generateDigestProse and seed-digest- notifications.mjs's generateAISummary). Ports Brain B's prompt features into buildDigestPrompt: - ctx={profile, greeting, isPublic} parameter (back-compat: 4-arg callers behave like today) - per-story severity uppercased + short-hash prefix [h:XXXX] so the model can emit rankedStoryHashes for stable re-ranking - profile lines + greeting opener appear only when ctx.isPublic !== true validateDigestProseShape gains optional rankedStoryHashes (≥4-char strings, capped to MAX_STORIES_PER_USER × 2). v2-shaped rows still pass — field defaults to []. hashDigestInput v3: - material includes profile-SHA, greeting bucket, isPublic flag, per-story hash - isPublic=true substitutes literal 'public' for userId in the cache key so all share-URL readers of the same (date, sensitivity, pool) hit ONE cache row (no PII in public cache key) Adds generateDigestProsePublic(stories, sensitivity, deps) wrapper — no userId param by design — for the share-URL surface. Cache prefix bumped brief:llm:digest:v2 → v3. v2 rows expire on TTL. Per the v1→v2 precedent (see hashDigestInput comment), one-tick cost on rollout is acceptable for cache-key correctness. Tests: 72/72 passing in tests/brief-llm.test.mjs (8 new for the v3 behaviors), full data suite 6952/6952. Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md Step 1, Codex-approved (5 rounds). * feat(brief): envelope v3 — adds digest.publicLead for share-URL surface Bumps BRIEF_ENVELOPE_VERSION 2 → 3. Adds optional BriefDigest.publicLead — non-personalised executive lead generated by generateDigestProsePublic (already in this branch from the previous commit) for the public share-URL surface. Personalised `lead` is the canonical synthesis for authenticated channels; publicLead is its profile-stripped sibling so api/brief/public/* never serves user-specific content (watched assets/regions). SUPPORTED_ENVELOPE_VERSIONS = [1, 2, 3] keeps v1 + v2 envelopes in the 7-day TTL window readable through the rollout — the composer only ever writes the current version, but readers must tolerate older shapes that haven't expired yet. Same rollout pattern used at the v1 → v2 bump. Renderer changes (server/_shared/brief-render.js): - ALLOWED_DIGEST_KEYS gains 'publicLead' (closed-key-set still enforced; v2 envelopes pass because publicLead === undefined is the v2 shape). - assertBriefEnvelope: new isNonEmptyString check on publicLead when present. Type contract enforced; absence is OK. Tests (tests/brief-magazine-render.test.mjs): - New describe block "v3 publicLead field": v3 envelope renders; malformed publicLead rejected; v2 envelope still passes; ad-hoc digest keys (e.g. synthesisLevel) still rejected — confirming the closed-key-set defense holds for the cron-local-only fields the orchestrator must NOT persist. - BRIEF_ENVELOPE_VERSION pin updated 2 → 3 with rollout-rationale comment. Test results: 182 brief-related tests pass; full data suite 6956/6956. Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md Step 2, Codex Round-3 Medium #2. * feat(brief): synthesis splice + rankedStoryHashes pre-cap re-order Plumbs the canonical synthesis output (lead, threads, signals, publicLead, rankedStoryHashes from generateDigestProse) through the pure composer so the orchestration layer can hand pre-resolved data into envelope.digest. Composer stays sync / no I/O — Codex Round-2 High #2 honored. Changes: scripts/lib/brief-compose.mjs: - digestStoryToUpstreamTopStory now emits `hash` (the digest story's stable identifier, falls back to titleHash when absent). Without this, rankedStoryHashes from the LLM has nothing to match against. - composeBriefFromDigestStories accepts opts.synthesis = {lead, threads, signals, rankedStoryHashes?, publicLead?}. When passed, splices into envelope.digest after the stub is built. Partial synthesis (e.g. only `lead` populated) keeps stub defaults for the other fields — graceful degradation when L2 fallback fires. shared/brief-filter.js: - filterTopStories accepts optional rankedStoryHashes. New helper applyRankedOrder re-orders stories by short-hash prefix match BEFORE the cap is applied, so the model's editorial judgment of importance survives MAX_STORIES_PER_USER. Stable for ties; stories not in the ranking come after in original order. Empty/missing ranking is a no-op (legacy callers unchanged). shared/brief-filter.d.ts: - filterTopStories signature gains rankedStoryHashes?: string[]. - UpstreamTopStory gains hash?: unknown (carried through from digestStoryToUpstreamTopStory). Tests added (tests/brief-from-digest-stories.test.mjs): - synthesis substitutes lead/threads/signals/publicLead. - legacy 4-arg callers (no synthesis) keep stub lead. - partial synthesis (only lead) keeps stub threads/signals. - rankedStoryHashes re-orders pool before cap. - short-hash prefix match (model emits 8 chars; story carries full). - unranked stories go after in original order. Test results: 33/33 in brief-from-digest-stories; 182/182 across all brief tests; full data suite 6956/6956. Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md Step 3, Codex Round-2 Low + Round-2 High #2. * feat(brief): single canonical synthesis per user; rewire all channels Restructures the digest cron's per-user compose + send loops to produce ONE canonical synthesis per user per issueSlot — the lead text every channel (email HTML, plain-text, Telegram, Slack, Discord, webhook) and the magazine show is byte-identical. This eliminates the "two-brain" divergence that was producing different exec summaries on different surfaces (observed 2026-04-25 0802). Architecture: composeBriefsForRun (orchestration): - Pre-annotates every eligible rule with lastSentAt + isDue once, before the per-user pass. Same getLastSentAt helper the send loop uses so compose + send agree on lastSentAt for every rule. composeAndStoreBriefForUser (per-user): - Two-pass winner walk: try DUE rules first (sortedDue), fall back to ALL eligible rules (sortedAll) for compose-only ticks. Preserves today's dashboard refresh contract for weekly / twice_daily users on non-due ticks (Codex Round-4 High #1). - Within each pass, walk by compareRules priority and pick the FIRST candidate with a non-empty pool — mirrors today's behavior at scripts/seed-digest-notifications.mjs:1044 and prevents the "highest-priority but empty pool" edge case (Codex Round-4 Medium #2). - Three-level synthesis fallback chain: L1: generateDigestProse(fullPool, ctx={profile,greeting,!public}) L2: generateDigestProse(envelope-sized slice, ctx={}) L3: stub from assembleStubbedBriefEnvelope Distinct log lines per fallback level so ops can quantify failure-mode distribution. - Generates publicLead in parallel via generateDigestProsePublic (no userId param; cache-shared across all share-URL readers). - Splices synthesis into envelope via composer's optional `synthesis` arg (Step 3); rankedStoryHashes re-orders the pool BEFORE the cap so editorial importance survives MAX_STORIES. - synthesisLevel stored in the cron-local briefByUser entry — NOT persisted in the envelope (renderer's assertNoExtraKeys would reject; Codex Round-2 Medium #5). Send loop: - Reads lastSentAt via shared getLastSentAt helper (single source of truth with compose flow). - briefLead = brief?.envelope?.data?.digest?.lead — the canonical lead. Passed to buildChannelBodies (text/Telegram/Slack/Discord), injectEmailSummary (HTML email), and sendWebhook (webhook payload's `summary` field). All-channel parity (Codex Round-1 Medium #6). - Subject ternary reads cron-local synthesisLevel: 1 or 2 → "Intelligence Brief", 3 → "Digest" (preserves today's UX for fallback paths; Codex Round-1 Missing #5). Removed: - generateAISummary() — the second LLM call that produced the divergent email lead. ~85 lines. - AI_SUMMARY_CACHE_TTL constant — no longer referenced. The digest:ai-summary:v1:* cache rows expire on their existing 1h TTL (no cleanup pass). Helpers added: - getLastSentAt(rule) — extracted Upstash GET for digest:last-sent so compose + send both call one source of truth. - buildSynthesisCtx(rule, nowMs) — formats profile + greeting for the canonical synthesis call. Preserves all today's prefs-fetch failure-mode behavior. Composer: - compareRules now exported from scripts/lib/brief-compose.mjs so the cron can sort each pass identically to groupEligibleRulesByUser. Test results: full data suite 6962/6962 (was 6956 pre-Step 4; +6 new compose-synthesis tests from Step 3). Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md Steps 4 + 4b. Codex-approved (5 rounds). * fix(brief-render): public-share lead fail-safe — never leak personalised lead Public-share render path (api/brief/public/[hash].ts → renderer publicMode=true) MUST NEVER serve the personalised digest.lead because that string can carry profile context — watched assets, saved-region names, etc. — written by generateDigestProse with ctx.profile populated. Previously: redactForPublic redacted user.name and stories.whyMatters but passed digest.lead through unchanged. Codex Round-2 High (security finding). Now (v3 envelope contract): - redactForPublic substitutes digest.lead = digest.publicLead when the v3 envelope carries one (generated by generateDigestProsePublic with profile=null, cache-shared across all public readers). - When publicLead is absent (v2 envelope still in TTL window OR v3 envelope where publicLead generation failed), redactForPublic sets digest.lead to empty string. - renderDigestGreeting: when lead is empty, OMIT the <blockquote> pull-quote entirely. Page still renders complete (greeting + horizontal rule), just without the italic lead block. - NEVER falls back to the original personalised lead. assertBriefEnvelope still validates publicLead's contract (when present, must be a non-empty string) BEFORE redactForPublic runs, so a malformed publicLead throws before any leak risk. Tests added (tests/brief-magazine-render.test.mjs): - v3 envelope renders publicLead in pull-quote, personalised lead text never appears. - v2 envelope (no publicLead) omits pull-quote; rest of page intact. - empty-string publicLead rejected by validator (defensive). - private render still uses personalised lead. Test results: 68 brief-magazine-render tests pass; full data suite remains green from prior commit. Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md Step 5, Codex Round-2 High (security). * feat(digest): brief lead parity log + extra acceptance tests Adds the parity-contract observability line and supplementary acceptance tests for the canonical synthesis path. Parity log (per send, after successful delivery): [digest] brief lead parity user=<id> rule=<v>:<s>:<lang> synthesis_level=<1|2|3> exec_len=<n> brief_lead_len=<n> channels_equal=<bool> public_lead_len=<n> When channels_equal=false an extra WARN line fires — "PARITY REGRESSION user=… — email lead != envelope lead." Sentry's existing console-breadcrumb hook lifts this without an explicit captureMessage call. Plan acceptance criterion A5. Tests added (tests/brief-llm.test.mjs, +9): - generateDigestProsePublic: two distinct callers with identical (sensitivity, story-pool) hit the SAME cache row (per Codex Round-2 Medium #4 — "no PII in public cache key"). - public + private writes never collide on cache key (defensive). - greeting bucket change re-keys the personalised cache (Brain B parity). - profile change re-keys the personalised cache. - v3 cache prefix used (no v2 writes). Test results: 77/77 in brief-llm; full data suite 6971/6971 (was 6962 pre-Step-7; +9 new public-cache tests). Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md Steps 6 (partial) + 7. Acceptance A5, A6.g, A6.f. * test(digest): backfill A6.h/i/l/m acceptance tests via helper extraction * fix(brief): close two correctness regressions on multi-rule + public surface Two findings from human review of the canonical-synthesis PR: 1. Public-share redaction leaked personalised signals + threads. The new prompt explicitly personalises both `lead` and `signals` ("personalise lead and signals"), but redactForPublic only substituted `lead` — leaving `signals` and `threads` intact. Public renderer's hasSignals gate would emit the signals page whenever `digest.signals.length > 0`, exposing watched-asset / region phrasing to anonymous readers. Same privacy bug class the original PR was meant to close, just on different fields. 2. Multi-rule users got cross-pool lead/storyList mismatch. composeAndStoreBriefForUser picks ONE winning rule for the canonical envelope. The send loop then injected that ONE `briefLead` into every due rule's channel body — even though each rule's storyList came from its own (per-rule) digest pool. Multi-rule users (e.g. `full` + `finance`) ended up with email bodies leading on geopolitics while listing finance stories. Cross-rule editorial mismatch reintroduced after the cross- surface fix. Fix 1 — public signals + threads: - Envelope shape: BriefDigest gains `publicSignals?: string[]` + `publicThreads?: BriefThread[]` (sibling fields to publicLead). Renderer's ALLOWED_DIGEST_KEYS extended; assertBriefEnvelope validates them when present. - generateDigestProsePublic already returned a full prose object (lead + signals + threads) — orchestration now captures all three instead of just `.lead`. Composer splices each into its envelope slot. - redactForPublic substitutes: digest.lead ← publicLead (or empty → omits pull-quote) digest.signals ← publicSignals (or empty → omits signals page) digest.threads ← publicThreads (or category-derived stub via new derivePublicThreadsStub helper — never falls back to the personalised threads) - New tests cover all three substitutions + their fail-safes. Fix 2 — per-rule synthesis in send loop: - Each due rule independently calls runSynthesisWithFallback over ITS OWN pool + ctx. Channel body lead is internally consistent with the storyList (both from the same pool). - Cache absorbs the cost: when this is the winner rule, the synthesis hits the cache row written during the compose pass (same userId/sensitivity/pool/ctx) — no extra LLM call. Only multi-rule users with non-overlapping pools incur additional LLM calls. - magazineUrl still points at the winner's envelope (single brief per user per slot — `(userId, issueSlot)` URL contract). Channel lead vs magazine lead may differ for non-winner rule sends; documented as acceptable trade-off (URL/key shape change to support per-rule magazines is out of scope for this PR). - Parity log refined: adds `winner_match=<bool>` field. The PARITY REGRESSION warning now fires only when winner_match=true AND the channel lead differs from the envelope lead (the actual contract regression). Non-winner sends with legitimately different leads no longer spam the alert. Test results: - tests/brief-magazine-render.test.mjs: 75/75 (+7 new for public signals/threads + validator + private-mode-ignores-public-fields) - Full data suite: 6995/6995 (was 6988; +7 net) - typecheck + typecheck:api: clean Plan: docs/plans/2026-04-25-002-fix-brief-email-two-brain-divergence-plan.md Addresses 2 review findings on PR koala73#3396 not anticipated in the 5-round Codex review. * fix(brief): unify compose+send window, fall through filter-rejection Address two residual risks in PR koala73#3396 (single-canonical-brain refactor): Risk 1 — canonical lead synthesized from a fixed 24h pool while the send loop ships stories from `lastSentAt ?? 24h`. For weekly users that meant a 24h-pool lead bolted onto a 7d email body — the same cross-surface divergence the refactor was meant to eliminate, just in a different shape. Twice-daily users hit a 12h-vs-24h variant. Fix: extract the window formula to `digestWindowStartMs(lastSentAt, nowMs, defaultLookbackMs)` in digest-orchestration-helpers.mjs and call it from BOTH the compose path's digestFor closure AND the send loop. The compose path now derives windowStart per-candidate from `cand.lastSentAt`, identical to what the send loop will use for that rule. Removed the now-unused BRIEF_STORY_WINDOW_MS constant. Side-effect: digestFor now receives the full annotated candidate (`cand`) instead of just the rule, so it can reach `cand.lastSentAt`. Backwards-compatible at the helper level — pickWinningCandidateWithPool forwards `cand` instead of `cand.rule`. Cache memo hit rate drops since lastSentAt varies per-rule, but correctness > a few extra Upstash GETs. Risk 2 — pickWinningCandidateWithPool returned the first candidate with a non-empty raw pool as winner. If composeBriefFromDigestStories then dropped every story (URL/headline/shape filters), the caller bailed without trying lower-priority candidates. Pre-PR behaviour was to keep walking. This regressed multi-rule users whose top-priority rule's pool happens to be entirely filter-rejected. Fix: optional `tryCompose(cand, stories)` callback on pickWinningCandidateWithPool. When provided, the helper calls it after the non-empty pool check; falsy return → log filter-rejected and walk to the next candidate; truthy → returns `{winner, stories, composeResult}` so the caller can reuse the result. Without the callback, legacy semantics preserved (existing tests + callers unaffected). Caller composeAndStoreBriefForUser passes a no-synthesis compose call as tryCompose — cheap pure-JS, no I/O. Synthesis only runs once after the winner is locked in, so the perf cost is one extra compose per filter-rejected candidate, no extra LLM round-trips. Tests: - 10 new cases in tests/digest-orchestration-helpers.test.mjs covering: digestFor receiving full candidate; tryCompose fall-through to lower-priority; all-rejected returns null; composeResult forwarded; legacy semantics without tryCompose; digestWindowStartMs lastSentAt-vs-default branches; weekly + twice-daily window parity assertions; epoch-zero ?? guard. - Updated tests/digest-cache-key-sensitivity.test.mjs static-shape regex to match the new `cand.rule.sensitivity` cache-key shape (intent unchanged: cache key MUST include sensitivity). Stacked on PR koala73#3396 — targets feat/brief-two-brain-divergence.
fuleinist
pushed a commit
that referenced
this pull request
Apr 25, 2026
…2 — flag-gated dark) (koala73#3407) * feat(resilience): financialSystemExposure dim scaffold (Phase 2 Ship 2 — flag-gated dark) Adds the new `financialSystemExposure` resilience dimension introduced in plan 2026-04-25-004 Phase 2, behind the `RESILIENCE_FIN_SYS_EXPOSURE_ENABLED` env flag. The flag defaults OFF so the dim ships dark — the scorer returns the empty-data shape (score=0, coverage=0, imputationClass=null) until the 3 component seeders (seed-bis-lbs, seed-fatf-listing, seed-wb-external-debt) are populating in production. Rollout pattern matches energy v2 (plan 2026-04-24-001). Components (weights total 1.0 inside the dim): short_term_external_debt_pct_gni 0.35 (WB IDS, lowerBetter, 15-0) bis_lbs_xborder_us_eu_uk_pct_gdp 0.30 (BIS LBS by-parent, U-shape) fatf_listing_status 0.20 (FATF, discrete: black=0, gray=30, compliant=100) financial_center_redundancy 0.15 (BIS LBS by-parent count, higherBetter, 1-10) When the flag flips ON, the scorer preflights all 3 required seed-meta envelopes (the BIS LBS seed serves both Component 2 and Component 4 — no separate Component 4 seeder). Missing envelopes throw `ResilienceConfigurationError(message, missingKeys)` (two-arg form per Codex R3 P1 #2) which `scoreAllDimensions` catches and routes to `imputationClass='source-failure'`. Per-country data gaps inside an otherwise-published envelope are distinct: per-component reads return null, the slot drops out of the weighted blend. Cache prefixes bumped in lockstep with the new dim: resilience:score:v13: → v14: resilience:ranking:v13 → v14 resilience:history:v8: → v9: The scaffold also reweights `tradePolicy` 1.0 → 0.5 in RESILIENCE_DIMENSION_WEIGHTS (the new dim shares the economic-domain weight). This shifts the headline overallScore by ~0.46 points in the flag-off baseline because the half-weighted tradePolicy contributes proportionally less to the coverage-weighted economic-domain mean. When the flag flips on with seeders populated, the new dim's actual signal will rebalance the headline score. What this PR ships: - new `scoreFinancialSystemExposure` + `normalizeBandLowerBetter` U-shape helper - 4 new INDICATOR_REGISTRY entries (BIS-derived tagged non-commercial/enrichment per Codex R1 koala73#8) - api/health.js + api/seed-health.js dual-registry entries for the 3 new seed keys - frontend label map + ResilienceWidget "20 dimensions" copy - methodology doc heading "Financial System Exposure" + indicator table - tests/resilience-financial-system-exposure.test.mts: 13 tests pinning the formula contract + fail-closed preflight + flag-off rollout posture - parity test extension for v14/v9 prefixes - 22-dim count update across release-gate + handlers + indicator-registry tests - flag-gated-dark exemption in release-gate's coverage check What this PR does NOT ship (deliberately deferred): - The 3 component seeders themselves (seed-bis-lbs, seed-fatf-listing, seed-wb-external-debt). These need real-API integration with BIS SDMX, FATF HTML scraping, and WB IDS — best landed as a separate PR with fixture-recorded tests. - Methodology doc `docs/methodology/financial-system-exposure.md` with full alternatives + license attribution. - Bundle registration in scripts/seed-bundle-macro.mjs. - Cohort sanity-check anchor test (RU/IR/KP < 20 on financialSystemExposure) — needs real seeder data. Once those land and seeders populate Redis, flip `RESILIENCE_FIN_SYS_EXPOSURE_ENABLED=true` in Vercel + Railway env config to activate the dim. Stacked on PR koala73#3405 (Phase 1 Ship 1). 🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0 Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]> * feat(resilience): financialSystemExposure component seeders + methodology doc Completes Phase 2 of plan 2026-04-25-004 by shipping the 3 component seeders, registering them in the macro bundle, adding 34 new fixture- based tests, and writing the full methodology doc with license attribution. Seeders (per plan §Files (Phase 2)): scripts/seed-wb-external-debt.mjs — WB IDS short-term external debt as % GNI (DT.DOD.DSTC.IR.ZS × DT.DOD.DECT.GN.ZS, mrv=5 + pickLatestPerCountry per memory `feedback_wb_bulk_mrv1_null_coverage_trap`) scripts/seed-bis-lbs.mjs — BIS LBS by-parent SDMX integration (12-dim key with L_POS_TYPE=N per Codex R3 P1 #1; 16 enumerated parent ISO2 codes per Codex R4 P1 #2 — no `4F` aggregate; ISO2 direct, no M49 mapping per CL_BIS_IF_REF_AREA) scripts/seed-fatf-listing.mjs — FATF entry-page parser with dynamic publication-URL follow + sanity-check band gates + monthly cadence + 90d cache TTL fallback Bundle registration: scripts/seed-bundle-macro.mjs — Option A per Codex R1 #5 (less operational overhead than provisioning a new bundle). Tests (34 new): tests/seed-wb-external-debt.test.mjs — combineExternalDebt formula pinning, IMF Article IV 15% GNI anchor, conservative-year selection, validate floor tests/seed-bis-lbs.test.mjs — combineLbsByCounterparty Brazil 2024Q4 ground-truth anchor, 1% GDP threshold for parentCount, BIS aggregate-code allow-list (5J/5A skipped), SDMX-JSON shape parsing, parser-regression guard tests/seed-fatf-listing.test.mjs — findPublicationLink entry-page anchor extraction (case-insensitive), country- name lookup with apostrophe handling (`People's` → `peoples`), pub-date inference from URL slug + header, DPRK-on-call-for-action invariant Methodology doc: docs/methodology/financial-system-exposure.md — full construct definition, per-component formulas + score shapes + coverage matrices, fail-closed preflight, methodology invariants, sanctions-isolated-jurisdiction sanity-check anchors (RU/IR/KP/... < 20), bounded-movement gate (60%+ |Δ|<3pt), data-source licensing (BIS terms of use), alternatives considered (5), future considerations (Phase 3-5) Quality: - npm run typecheck + typecheck:api: clean - npm run test:data: 7149/7149 pass (was 7115; +34 new seeder tests) - npm run lint + lint:md + version:check: clean - Edge function bundle check: clean Activation runbook (when ready to flip the dim live): 1. Merge this PR 2. Run seed-wb-external-debt + seed-bis-lbs + seed-fatf-listing manually via `railway run --service seed-bundle-macro -- node scripts/<seeder>.mjs` 3. Verify all 3 seed-meta envelopes published: redis-cli GET 'seed-meta:economic:wb-external-debt' redis-cli GET 'seed-meta:economic:bis-lbs' redis-cli GET 'seed-meta:economic:fatf-listing' 4. Set RESILIENCE_FIN_SYS_EXPOSURE_ENABLED=true in Vercel + Railway env config 5. Flush v14 caches: bulk DEL resilience:score:v14:* + DEL resilience:ranking:v14 6. Run seed-resilience-scores.mjs to bulk-warm v14 with the new dim's signal 7. Cohort audit: snapshot resilience:score:v14:* for all 222 countries; RU/IR/KP must score < 20 on financialSystemExposure (gate the construct before stable-rollout) 8. Bounded-movement gate: 60% of countries |Δ|<3pt; outliers > 12pt must be in the explicitly-predicted RU/IR/KP/CU/VE/BY/LY/MM set 9. Remove FLAG_GATED_DARK_DIMENSIONS allow-list entry in tests/resilience-release-gate.test.mts in the same commit that flips the flag 🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0 Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]> * chore(resilience): address self-review hardening (P1 + 4×P2 + 4×P3) P1 — BIS LBS sequential-fetch timeout math: Sequential 16 parents × 60s timeout = 960s worst-case, exceeding the bundle's 600s timeoutMs. SIGTERM mid-flight would have leaked the child-lock + produced a covertly-degraded payload (per memory `bundle-runner-sigkill-leaks-child-lock`). Fix: parallelize parent fetches with concurrency=4 via a bounded-concurrency runner. Caps wall time at ~4 × 60s = 240s on the slow path while staying polite to BIS. P2 — BIS LBS parent-success gate: Previous "all 16 failed" short-circuit was too permissive. If 15 of 16 parent fetches failed, a single-parent payload would pass validate (>100 counterparty floor) and skew Component 4 (financialCenterRedundancy) low for every counterparty until the next successful run. Added MIN_SUCCESSFUL_PARENTS=12 gate; below that, throw → seed-meta unchanged → previous valid payload stays alive under cache TTL. P2 — FATF findPublicationLink prefers highest-year anchor: Previous "first anchor matching label" was vulnerable to FATF page layouts where a sidebar links to historical publications using the same wording. Fix: collect all candidates, sort by year (URL slug or anchor text), return highest. Logs all candidates at WARN when more than one matches so ops can spot drift. P2 — FATF unmatched country-name surfacing: Previous parser silently dropped country names not in shared/country-names.json. If FATF introduced a new spelling ("Mauretania", "Türkiye"), the country fell out of the listing and defaulted to "compliant" (score 100) — materially shifting its financialSystemExposure score under a fresh seed-meta. Fix: extractListedCountries now returns { listed, unmatchedCandidates }. Seeder logs unmatched at WARN; throws if > 2 candidates unmatched (indicates parser drift / new spellings the lookup needs to learn). P2 — WB external-debt yearMismatch metadata: Composition can mix vintages of the two source indicators (different WB IDS lag patterns). Previous output silently used min(year) as the conservative anchor. Fix: emit `yearMismatch: boolean` + `shortTermPctOfTotalDebtYear` + `totalDebtPctOfGniYear` on each country record so the dashboard / scorer / audit can flag countries with cross-year composition. Pinning test asserts min(year) selection + correct flag. P3 — BIS LBS upper-bound corruption guard: Added `latestVal > 1e8` skip in extractClaimsByCounterparty. 1e8 millions = $100T (>half of global GDP), well above any plausible bilateral claim. Drops corrupt SDMX values silently rather than letting them inflate totalXborderPctGdp downstream. P3 — BIS LBS validation floor 100 → 150: BIS LBS counterparty coverage is ~200+ jurisdictions. Floor of 100 would have accepted a payload with massive coverage regression. Tightened to 150. P3 — FATF grey-list floor 8 → 12 + band 8-40 → 12-40: Historical FATF grey-list size has been 15+ since 2020. Floor of 8 was too lenient. Tightened to 12 with comment explaining the historical band. P3 — FATF parse-failure WARN logging: All sanity-check throw paths in seed-fatf-listing now emit a console.warn explaining the failure and noting "previous valid payload remains under cache TTL" before throwing. Plan called for "warn loudly"; the implicit fall-back-to-cache pattern is now diagnostic-friendly. Suggestion — BIS LBS droppedForMissingGdp provenance: Counterparties seen in BIS LBS but dropped because no WB GDP record was available are now collected into a `droppedForMissingGdp` array on the seed payload. Surfaces silent coverage gaps for ops triage without polluting the main `countries` map. Suggestion — methodology doc operational footguns + smoke test: New §"Common operational footguns" section in docs/methodology/financial-system-exposure.md surfacing the BIS LBS 4F-aggregate-rejection lesson + ISO 3166-1 (not M49) clarification + pre-flag-flip smoke test commands. Quality gates: - npm run typecheck + typecheck:api: clean - npm run lint + lint:md + version:check: clean - npm run test:data: 7153/7153 pass (was 7149; +4 hardening tests) 🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0 Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]> * fix(resilience): scoreFinancialSystemExposure preflights UNVERSIONED seed-meta keys P1 reviewer catch: the preflight in scoreFinancialSystemExposure was reading `seed-meta:economic:<key>:v1` while runSeed (scripts/_seed-utils.mjs) writes the freshness record at `seed-meta:${dataKey.replace(/:v\d+$/, '')}` — i.e. with the trailing :v\d+ stripped. Once `RESILIENCE_FIN_SYS_EXPOSURE_ENABLED=true` was flipped, every /api/resilience/* request would have hit the missing- seed-meta path indefinitely, throwing ResilienceConfigurationError and stamping every country's financialSystemExposure as imputationClass='source-failure' even with healthy seeders running. The same unversioned shape is already used by api/health.js + api/seed-health.js + every other in-tree scorer that walks seed-meta keys via _dimension-freshness.ts. Fix: route the preflight through `resolveSeedMetaKey` from _dimension-freshness.ts. That helper already strips the trailing :v\d+ AND applies the SOURCE_KEY_META_OVERRIDES table — the canonical in-tree pattern for "given a registry data-key, return the seed-meta key that runSeed actually writes." Inlining the regex would have re-introduced the same writer/reader drift this helper exists to prevent. Tests: - All formula + preflight tests (which previously mocked the incorrect versioned form) updated to the unversioned key shape so they actually exercise the production read path. - New regression-guard test "preflight reads UNVERSIONED seed-meta keys (matches runSeed write-key shape)" pins the exact key shape the scorer probes. Asserts both presence of the unversioned form AND absence of the versioned form. A future refactor that accidentally re-versions the preflight will fail loudly here instead of silently routing every country to source-failure. Quality gates: - npm run typecheck:api: clean - npm run test:data: 7154/7154 pass (was 7153; +1 contract guard) 🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0 Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]> * fix(resilience): address Greptile P1 + P2 catches on PR koala73#3407 (4 issues) P1 — 30-point scoring cliff at 25% boundary in normalizeBandLowerBetter: Original draft had piecewise-linear segments with mismatched endpoints: - At value=25: sweet spot ended at 100, over-exposed started at 70 → 30-point cliff × 0.30 weight ≈ 9-pt headline swing - At value=5: low-int ended at 70, sweet started at 75 → 5-pt jump Cliffs in piecewise-linear scorers cause ranking instability for countries near band edges (24.9% scores ~99.9, 25.1% scores ~70). Re-anchored adjacent segments to share endpoints — function is now piecewise-CONTINUOUS at 5%, 25%, and 60% transitions: 0%-5%: 60 → 75 (slope +3/pct, was +2) 5%-25%: 75 → 100 (unchanged) 25%-60%: 100 → 30 (slope −2/pct, was 70 → 30 / slope −1.14) 60%+: 30 → 0 (unchanged) New regression test pins continuity at all three boundaries (samples values immediately above/below each transition, asserts |Δ| ≤ 1pt). P2 — readFatfStatus defaults empty listings dict to "compliant": An empty `listings: {}` payload that bypassed the seeder's validate() would silently score every country at 100 (compliant default), masking a parser regression. Added defense-in-depth guard: empty dict → null component score → slot drops out of weighted blend → coverage shrinks visibly rather than the dim looking healthy. Seeder validate already enforces ≥1 black + ≥12 grey so this can't reach production through the normal write path; the guard costs nothing and catches malformed payloads that bypass validation. New regression test pins. P2 — bisLbsXborderPctGdp goalposts mismatch U-shape peak: Registry entry had `goalposts: { worst: 60, best: 15 }` implying a linear lowerBetter scale peaking at 15%. The U-shape function actually peaks at 25% (value=15% scores 87.5, not 100). Updated to `{ worst: 60, best: 25 }` (over-exposed branch, peak anchor) with an explicit comment that goalposts here are documentation-only — the actual scorer uses normalizeBandLowerBetter, not a generic linear normalizer. Tooling reading the registry should consult the scorer helper directly. P2 (resolved differently than originally suggested) — api/bootstrap.js missing 3 new data keys: Greptile flagged that AGENTS.md requires bootstrap hydration for new data sources. Verified the bootstrap-hydration-coverage test enforces this via a `getHydratedData` consumer requirement in src/. The 3 new keys (economic:wb-external-debt:v1, economic:bis-lbs:v1, economic:fatf-listing:v1) are SERVER-ONLY — they feed scoreFinancialSystemExposure inside /api/resilience/* handlers; no client panel consumes them directly. Adding them to bootstrap without a consumer would fail the parity tests. Documented in api/bootstrap.js as a deferred entry: "if a future PR adds a client panel that displays raw BIS LBS / FATF / WB external-debt data, register the keys here AND add the corresponding consumer + cache-keys.ts entries in the same PR." Note on Greptile's P1 #1 (preflight `:v1` suffix mismatch): already resolved in commit d904103 (uses `resolveSeedMetaKey` from _dimension-freshness.ts). Greptile reviewed an earlier commit. Quality gates: - npm run typecheck:api: clean - npm run lint + lint:md + version:check: clean - npm run test:data: 7156/7156 pass (was 7154; +2 regression guards) 🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via [Claude Code](https://claude.com/claude-code) + Compound Engineering v3.0.0 Co-Authored-By: Claude Opus 4.7 (1M context, extended thinking) <[email protected]> --------- Co-authored-by: Claude Opus 4.7 (1M context, extended thinking) <[email protected]>
fuleinist
pushed a commit
that referenced
this pull request
May 10, 2026
* feat(convex): add mcpProTokens table + issue/validate/revoke + http routes (U1)
Non-key Pro MCP identity layer for the Pro-tier MCP access plan. Mirrors
apiKeys.ts shape but stores no key material — the row's _id IS the
bearer identifier (referenced from OAuth code/token records as
mcpTokenId).
- mcpProTokens table: {userId, clientId?, name?, createdAt, lastUsedAt?,
revokedAt?} indexed by_userId; userApiKeys untouched.
- Internal mutations: issueProMcpToken (tier ≥ 1 gate, 5-row cap with
silent oldest-revoke rotation, audit-preserving), validateProMcpToken,
internalRevokeProMcpToken (server-side rollback path for U5 — no Clerk
context needed), touchProMcpTokenLastUsed (debounced 5min).
- Public mutations: revokeProMcpToken + listProMcpTokens (Clerk-auth).
- HTTP internal routes: /api/internal-{issue,validate,revoke}-pro-mcp-token
with x-convex-shared-secret guard. Validate route schedules touch via
ctx.scheduler.runAfter, matching apiKeys pattern (http.ts:839).
28 new tests; full convex suite 245/245.
Plan unit U1: docs/plans/2026-05-10-001-feat-pro-mcp-clerk-auth-quota-plan.md
* feat(server): pro-mcp-token edge helper — issue/validate/revoke (U2)
Edge-runtime-safe wrappers around U1's Convex internal HTTP routes. No
positive cache on validate (revoke takes effect on next request);
negative-cache only at pro-mcp-token-neg:<tokenId> (60s TTL) for
known-bad bearers.
- issueProMcpTokenForUser: typed ProMcpIssueFailed (pro-required /
invalid-user-id / config / network), 3s timeout.
- validateProMcpToken: neg-cache short-circuit BEFORE Convex hit; fail-soft
on 5xx/timeout (returns null but does NOT write neg-cache sentinel —
prevents transient-blip poisoning of legitimate tokens). Convex's
validate route schedules touchProMcpTokenLastUsed internally; no
edge-side waitUntil needed.
- revokeProMcpToken: returns result object (no throw) so rollback callers
don't have their original error masked. Sets neg-cache sentinel after
successful revoke.
- invalidateProMcpTokenCache: public sentinel writer for U9.
25 tests including a load-bearing integration test for revoke → next
validate short-circuits without Convex round-trip.
Plan unit U2.
* feat(oauth): apex Clerk grant page + signed-grant mint API (U3)
Bridge between the apex Clerk session and the api-subdomain Pro MCP
consent flow. Apex /mcp-grant SPA reads the OAuth nonce + registered
client metadata so users see the real client_name + redirect host
(anti-phishing). On Authorize the page POSTs to mcp-grant-mint, gets
back a fixed redirect URL with a HMAC-signed grant token, and navigates.
- mcp-grant.html + src/mcp-grant-main.ts: Vite multi-page entry (matches
existing settings.html / live-channels.html convention).
- api/internal/mcp-grant-mint.ts: Clerk-auth POST. Re-checks tier ≥ 1,
re-validates redirect_uri allowlist (defense-in-depth), HMAC-signs
{userId, nonce, exp+5min}, SETEX mcp-grant:<nonce>, returns FIXED
redirect URL (https://api.worldmonitor.app/oauth/authorize-pro).
- api/internal/mcp-grant-context.ts: GET companion for the SPA to fetch
the real client_name + redirect_host. Same Pro-tier gate as mint.
- api/_mcp-grant-hmac.ts: shared sign/verify helper. Signature is over
exact post-base64url-decode bytes (key-order/whitespace independent).
U5 imports verifyGrant from this module.
- vercel.json: /mcp-grant → /mcp-grant.html rewrite + no-store header
rule. Catch-all SPA fallback adjusted to exclude the new page.
- api/oauth/register.js: export isAllowedRedirectUri so DCR allowlist is
reused (no parallel impl).
- 35 tests; full deploy-config + edge-functions + mcp + pro-mcp-token
suites green.
Plan unit U3.
* feat(oauth): consent page Pro-CTA-default + 'Use API key instead' (U4)
R1 of the plan: a logged-in Pro user authorising MCP from claude.ai
never sees the API-key input field by default. The consent page now
leads with a brand-green "Sign in with WorldMonitor Pro" CTA pointing
at the apex /mcp-grant page; the existing API-key form is hidden
behind a "Use API key instead" disclosure for Starter+ holders.
- Default state: form display:none; Pro CTA dominant.
- Error state: existing errorMsg path still works — when ke.textContent
is non-empty (or XHR-retry returns invalid_key), inline script auto-
reveals the form so the user doesn't lose context after a bad key.
- Deep-link: /oauth/authorize?...#api-key shows the form on initial
load, so Starter+ users can bookmark.
- Pro CTA href: https://worldmonitor.app/mcp-grant?nonce=<URL-encoded n>.
Apex page reads the OAuth nonce server-side (no client_name forwarded
via URL — already covered by U3's mcp-grant-context).
- nonce shared with U5: the same oauth:nonce:<n> row that U5's
/oauth/authorize-pro will GETDEL. Handler still mints exactly one
nonce per GET — no double-issue.
- XSS-escape preserved on client_name + redirect_host + nonce + errorMsg.
- 18 new tests; existing handler logic untouched.
Plan unit U4.
* feat(oauth): /oauth/authorize-pro bounce-back endpoint (U5)
Receives the apex grant bounce-back, validates the HMAC-signed grant,
atomically consumes both Redis nonces (mcp-grant + oauth:nonce), issues
a non-key Pro identity row in Convex via U2, and writes the OAuth code
with {kind:'pro', userId, mcpTokenId, ...} for U6 to read at exchange
time.
Security ordering (HMAC-first to avoid burning nonces on forged tokens):
1. verifyGrant (U3's _mcp-grant-hmac, no Redis)
2. URL-nonce vs payload-nonce match
3. GETDEL mcp-grant:<n>
4. strict {userId, exp} tuple match vs grant payload (forge defense)
5. GETDEL oauth:nonce:<n>
6. GET oauth:client:<client_id> + redirect_uri allowlist re-check
7. getEntitlements(userId) tier ≥ 1 re-check (lapse defense)
8. issueProMcpTokenForUser (U2)
9. SETEX oauth:code:<code> (kind:'pro', 600s)
10. 302 redirect_uri?code=<code>[&state=...] with Cache-Control:no-store
oauth:code shape (load-bearing for U6):
{kind:'pro', userId, mcpTokenId, client_id, redirect_uri,
code_challenge, scope:'mcp_pro'}
Best-effort revoke when SETEX fails after issueProMcpTokenForUser
succeeds — calls U2's no-throw revokeProMcpToken; logs orphan-row id
on revoke failure but never masks the original 500.
35 tests; vercel rewrite + api-route-exceptions registered as
external-protocol.
Plan unit U5.
* feat(oauth): McpAuthContext discriminated union + Pro token shape (U6)
Token endpoint and bearer resolver learn about Pro-shape tokens without
breaking Starter+ legacy paths.
api/_oauth-token.js:
- New resolveBearerToContext(token) returns a discriminated union:
{kind:'env_key', apiKey} -- legacy WORLDMONITOR_VALID_KEYS resolution
{kind:'pro', userId, mcpTokenId} -- new Pro identity
- resolveApiKeyFromBearer kept as backward-compat wrapper. Returns
apiKey for env_key kind, null for pro kind (so legacy callers can't
mis-handle a Pro bearer).
api/oauth/token.js → api/oauth/token.ts:
- Converted to TypeScript so it can import validateProMcpToken from
server/_shared/pro-mcp-token cleanly. Vercel routes by basename;
/oauth/token rewrite unaffected.
- Default handler wires injected deps via tokenHandler(req, deps) for
testability (mirrors U3/U5 pattern).
- authorization_code branch: dispatches on codeData.kind. Pro path
validates client_id + redirect_uri bind, PKCE-verifies, mints tokens
via storeProTokens (object shape). Legacy path unchanged.
- refresh_token branch: Pro refreshes call validateProMcpToken
(per-request Convex hit, no positive cache); revoked row → invalid_grant.
Cross-user defensive guard: Convex-returned userId must match bearer's.
family_id, mcpTokenId, scope ('mcp_pro') preserved across rotation.
- client_credentials grant untouched.
oauth:token:<uuid> shapes:
- Legacy (preserved): JSON.stringify("<sha256-hex-64>") or "<fingerprint-16>"
- Pro (new): JSON.stringify({kind:'pro', userId, mcpTokenId})
- oauth:refresh:<uuid> Pro shape includes client_id, scope, family_id
for rotation discipline (matches legacy semantics).
26 new tests; sibling suites (oauth-authorize, oauth-authorize-pro, mcp,
pro-mcp-token — 101 tests) regress green. tsc -p tsconfig.api.json clean.
Plan unit U6.
* feat(mcp): Pro identity, atomic INCR-first quota, internal HMAC fetch (U7)
Behavioral heart of the Pro MCP plan. api/mcp.ts now resolves bearers
as McpAuthContext (env_key | pro), runs Pro-specific pre-checks, and
enforces a hard 50/UTC-day cap via INCR-first reservation with
DECR-rollback on cap-exceed and on tool-dispatch failure.
Pre-dispatch chain for Pro context (synchronous, in order):
1. validateProMcpToken(mcpTokenId) — null = 401 revoked
2. defensive userId match — 401 cross-user
3. getEntitlements(userId) tier ≥ 1, mcpAccess: true, validUntil — fail-closed
4. per-minute slidingWindow(60, 60s) keyed pro-user:<userId> — fail-open
5. for tools/call ONLY: pipeline INCR + EXPIRE 172800
newCount > 50 → DECR rollback + -32029 + 429 + Retry-After
INCR Redis transient → -32603 + 503 + Retry-After:5 (hard-cap correctness)
6. dispatch tool; on throw → DECR rollback + -32603
Internal-HMAC tool fetches (replaces X-WorldMonitor-Key for Pro):
payload = ${ts}:${METHOD}:${pathname}:${queryHash}:${bodyHash}:${userId}
queryHash = SHA-256(canonicalQueryString) sorted keys, URL-encoded
bodyHash = SHA-256(bodyBytes) SHA-256("") for empty
sig = HMAC-SHA-256(MCP_INTERNAL_HMAC_SECRET, payload)
headers = X-WM-MCP-Internal: ${ts}.${b64url(sig)}, X-WM-MCP-User-Id
Replay defense: payload binds method+path+queryHash+bodyHash, so a
captured signature for /api/news/v1/list-feed-digest cannot be reused
on /api/intelligence/v1/deduct-situation. ±30s window.
server/_shared/mcp-internal-hmac.ts: single source of truth for sign
helpers. U8 verify imports the same canonicalisation primitives.
server/_shared/pro-mcp-token.ts: dailyCounterKey(userId), 50/172800s
constants exported for U9 to read the SAME Redis key the enforcement
writes.
_execute signatures changed (params, base, context) — option A from
the plan. Cache-only tools unchanged.
waitUntil unused: Convex internal-validate-pro-mcp-token schedules
touchProMcpTokenLastUsed itself (verified at convex/http.ts:1035-1040
from U1's commit 4530bdf).
22 new tests + 23 Starter+ regression tests in tests/mcp.test.mjs;
sibling chain (oauth-token, pro-mcp-token, oauth-authorize-pro,
mcp-grant-mint, mcp-proxy) all green. tsc + biome clean.
Plan unit U7.
* feat(gateway): HMAC-verify internal-MCP requests + sanitised propagation (U8)
Verifier counterpart to U7's signer. Gateway accepts X-WM-MCP-Internal
HMAC headers in lieu of an API key for Pro internal-MCP traffic, then
hands a freshly-constructed Request with sanitised trusted markers to
the downstream handler. isCallerPremium learns to honour those markers
so Pro framework/systemAppend semantics survive in summarize-article,
get-country-intel-brief, deduct-situation.
Gateway flow at the top of createDomainGateway's handler:
1. STRIP inbound x-wm-mcp-internal-verified + x-user-id (anti-injection)
2. If X-WM-MCP-Internal present:
verifyInternalMcpRequest re-canonicalises and HMAC-verifies the
request shape (method+pathname+queryHash+bodyHash+userId), ±30s
window, timing-safe compare. Failure → 401 invalid_internal_mcp_
signature (no fall-through to validateApiKey — present-but-bad is
a deliberate forge attempt).
getEntitlements(userId): tier ≥ 1 + mcpAccess === true (fail-closed).
Construct NEW Request via new Request(url, {method, body, headers})
with x-wm-mcp-internal-verified=<per-process nonce> +
x-user-id=<verified userId>. Skip validateApiKey + IP rate limit.
isCallerPremium adds a NEW first branch: timing-safe compare of
x-wm-mcp-internal-verified against the per-process nonce + non-empty
x-user-id + defensive getEntitlements re-fetch. Catches direct-edge-
function consumers (api/widget-agent.ts, chat-analyst.ts, me/entitlement.ts,
v2/shipping/webhooks/...) that don't run the gateway strip step.
DEFENSE-IN-DEPTH BEYOND PLAN: trusted marker is a per-process random
16-byte nonce, NOT the literal '1'. Sweep found direct edge functions
calling isCallerPremium without gateway-strip protection — a constant
marker would be spoofable from outside. The nonce is born once at
process startup, only the gateway knows it, comparison is timing-safe.
verifyInternalMcpRequest lives in server/_shared/mcp-internal-hmac.ts
next to U7's sign helpers — single source of truth for canonical form
+ payload + ±30s window. Drift between sign and verify is structurally
impossible.
26 new tests + 100 sibling regression tests (gateway-cdn-origin-policy,
premium-stock-gateway, premium-fetch, pro-mcp-token, mcp). tsc clean.
Plan unit U8.
* feat(catalog): mcpAccess feature flag + env vars + Pro-MCP docs (U10)
Catalog, schema, type, env, and docs scaffolding to make the Pro MCP
flow deployable end-to-end. U9 unblocks on this — settings UI gates on
hasFeature('mcpAccess') (distinct from existing apiAccess paywall).
- convex/config/productCatalog.ts: PlanFeatures.mcpAccess on every tier.
Free=false. Pro_monthly/Pro_annual=true. API_starter / API_starter_annual
/ API_business=true. Enterprise=true. Pro plan marketing copy mentions
"MCP access for Claude Desktop & other AI clients (50 calls/day)".
- convex/schema.ts + convex/payments/cacheActions.ts: mcpAccess optional
field on the entitlements validator; legacy rows pass.
- server/_shared/entitlement-check.ts: CachedEntitlements.features
mcpAccess?: boolean (consumed by U7 + U8).
- src/services/entitlements.ts: EntitlementState.features.mcpAccess?:
boolean — unblocks hasFeature('mcpAccess') for U9's settings tab gate.
- .env.example: new "Pro MCP" section with MCP_PRO_GRANT_HMAC_SECRET
(apex grant bridge, U3/U5) and MCP_INTERNAL_HMAC_SECRET (gateway
service-auth, U7/U8). Both 32-byte base64; both required.
- docs/mcp-server.mdx: "Pro sign-in flow" section + 50/day quota
callout + "Connected MCP clients" management pointer.
Bundled fixups for U7/U8 strict-mode TS errors:
- mcp-internal-hmac.ts: bufferToBase64Url uses for-of (avoids
noUncheckedIndexedAccess error on bytes[i]).
- gateway.ts: drop unused INTERNAL_MCP_USER_ID_HEADER import.
tsc -p tsconfig.api.json + tsc -p tsconfig.json clean.
9 entitlements tests + 71 gateway-internal-mcp + mcp tests pass.
Plan unit U10.
* feat(settings): Connected MCP clients tab + quota endpoint + revoke (U9)
User-facing surface that makes the Pro MCP plan visible. Settings UI
gets a new "Connected MCP clients" tab gated on hasFeature('mcpAccess')
(distinct from the existing apiAccess-gated 'API Keys' tab — Pro users
without apiAccess see only the new tab).
Endpoints:
- GET /api/user/mcp-quota → {used, limit:50, resetsAt}. Reads the
SAME Redis key (dailyCounterKey from U2) that U7 enforcement writes.
Single source of truth.
- POST /api/user/mcp-revoke {tokenId} → forwards Clerk-derived userId
(NOT client-supplied) to Convex internal-revoke-pro-mcp-token, then
calls invalidateProMcpTokenCache so any in-flight bearer with this
tokenId 401s within 60s. 404 on NOT_FOUND, 409 on ALREADY_REVOKED.
Tenancy enforced inside Convex (row.userId === userId).
UI (in src/components/UnifiedSettings.ts, no child component split —
matches existing API Keys tab pattern):
- Tab gated on hasFeature('mcpAccess') so Pro users see it without
needing apiAccess.
- Quota header auto-refreshes every 30s; interval cleared on tab-
switch / close / destroy.
- Each row: name, createdAt, lastUsedAt (relative), revokedAt
(struck-through + badge); active rows show Revoke button with
confirm() dialog (matches existing pattern).
- Empty state: click-to-copy https://api.worldmonitor.app/mcp.
Frontend service in src/services/mcp-clients.ts: listMcpClients (Convex
query), revokeMcpClient (POST /api/user/mcp-revoke), fetchMcpQuota.
11 + 14 = 25 new tests; api-route-exceptions registered as
internal-helper.
Plan unit U9.
---
PLAN COMPLETE — 10/10 units shipped. Follow-up polish noted in U9
report: CSS rules for .mcp-clients-* classes (suggest copy from
.api-keys-* rules); empty-state URL is hardcoded to
api.worldmonitor.app/mcp.
* style(settings): mcp-clients CSS rules mirroring api-keys (U9 polish)
* fix(pro-mcp): apply Tier-2 code-review residuals — security + correctness
Review pass after U10 (3 reviewers: code-reviewer, ce-security-reviewer,
ce-adversarial-reviewer). Findings BLOCKING + HIGH + MEDIUM + LOW
applied per user direction.
BLOCKING (gateway):
- Add validUntil check to gateway's HMAC-verify entitlement re-check.
Defense-in-depth against Convex-fallback paths returning a stale row
past its expiry.
HIGH — adv-001: cross-user OAuth nonce hijack:
- mcp-grant-mint now claims oauth nonces atomically via Upstash SET NX
semantics. If the nonce is already claimed by a DIFFERENT userId,
return 403 NONCE_CLAIMED_BY_OTHER_USER. Same userId multi-tab retry
is idempotent: re-sign the grant using the existing claim's exp so
authorize-pro's strict tuple equality still matches.
- mcp-grant-context performs the same cross-user check so the apex SPA
refuses to render consent context for a hijacked nonce.
MEDIUM — adv-008: refresh-token loss during Convex transient:
- validateProMcpToken returns a discriminated union {ok:'valid',userId}
| {ok:'revoked'} | {ok:'transient'}. Negative-cache writes only on
'revoked'. validateProMcpTokenOrNull wrapper preserves backward
compat for callers that don't need the distinction (per-request MCP
edge stays fail-closed).
- token.ts refresh path: on 'transient', best-effort restore the
consumed refresh token to Redis with the original TTL and return
503 server_error + Retry-After:5. Client retries; if Convex recovers,
refresh succeeds. No more permanent session loss on a Convex blip.
MEDIUM — adv-002: counter overshoot lockout:
- mcp.ts: on cap-exceeded after DECR rollback, if newCount is still
far above PRO_DAILY_QUOTA_LIMIT (overshoot from prior transient),
pipelined INCR+DECR probe + bounded DECR-sweep to converge the
counter back near the limit. Prevents one Redis hiccup from locking
out a paying Pro user for the rest of the UTC day.
MEDIUM — code-reviewer #4: 5-row cap race in Convex:
- mcpProTokens.issueProMcpToken now revokes ALL active rows beyond
MAX-1 (sorted by createdAt) on each issue, so concurrent inserts
that briefly produce 6 active rows converge back to 5 on the next
call.
LOW:
- adv-003: executeTool throws cache_all_null when every cache read
returns null AND there were keys configured — triggers DECR rollback
in dispatchToolsCall instead of silently burning quota on a degenerate
empty result.
- code-reviewer koala73#10: gateway strips X-WM-MCP-Internal + X-WM-MCP-User-Id
from the trusted Request before forwarding to handler (no info leak).
- adv-004: 256 KB body cap (Content-Length + post-buffer count) on
both gateway strip and HMAC-verify paths — bounds memory amplification.
- adv-006: dailyCounterKey now env-prefixes (preview deploys can't
collide with production traffic on the same Upstash instance).
Reader (mcp-quota.ts) and writer (mcp.ts) share byte-identical keys
via the helper.
- adv-007 / code-reviewer #5: signInternalMcpRequest throws on
Blob/FormData/ReadableStream/plain-object bodies instead of silently
JSON.stringify'ing them (which would produce a hash that can't match
the wire bytes).
- code-reviewer koala73#9: timestamp regex tightened to ^[0-9]{1,15}$ so
future ms-precision timestamps don't silently truncate through Number().
- code-reviewer koala73#8: MCP_INTERNAL_HMAC_SECRET preflight in runProPreChecks
surfaces config errors at auth-resolution rather than mid-tool-fetch.
NITPICK:
- code-reviewer #6: rewritten readNegCache comment.
- code-reviewer #7: malformed-body error now reports reason
'malformed_request' instead of misleading 'auth_401'. New
RequestReason value added to usage.ts enum.
30 new tests + ~14 adapted; full convex suite 247/247, edge/server
suites 254/254. tsc clean across both tsconfig.json and tsconfig.api.json.
Plan: docs/plans/2026-05-10-001-feat-pro-mcp-clerk-auth-quota-plan.md
* chore(pro-marketing): regenerate /pro bundle for U10 catalog copy
U10's productCatalog.ts added an mcpAccess feature flag and updated the
Pro plan description copy to "MCP access for Claude Desktop & other AI
clients (50 calls/day)". scripts/generate-product-config.mjs cascades
that copy into pro-test/src/generated/tiers.json, which the /pro
marketing app builds into public/pro/assets/.
Vercel does NOT rebuild pro-test on deploy — public/pro/ ships whatever
is committed. Pre-push hook auto-ran the build and produced a new bundle
hash; this commit lands the regenerated artifacts so deploy picks them up.
Plan unit U10 (follow-on artifact).
* fix(pro-mcp): apply external review round-2 findings (P1+P2+P3)
Round-2 review on PR koala73#3646 surfaced 5 must-fix issues:
P1 — vercel.json `(?:...)` rejected by Vercel CI:
- Replace `mcp-grant(?:\\.html)?` with explicit `mcp-grant\\.html|mcp-grant`
inside the SPA catch-all negative-lookahead (lines 18 + 134).
- Split the headers `/mcp-grant(?:\\.html)?` source rule into two explicit
entries — Vercel's path-to-regexp `source` field doesn't accept `(?:...)`
with `?` quantifier as a standalone source.
- Update tests/deploy-config.test.mjs expectations to match.
P1 — api/api-route-exceptions.json points at deleted file:
- Update the entry path from `api/oauth/token.js` (deleted in U6) to
`api/oauth/token.ts`. `node scripts/enforce-sebuf-api-contract.mjs`
was failing pre-push CI.
P1 — mcpAccess migration gap on existing entitlement rows:
- convex/entitlements.ts now read-time merges `getFeaturesForPlan(planKey)`
defaults under stored features. Pre-U10 rows lacking `mcpAccess` see the
catalog default surfaced immediately, with NO wait for the next Dodo
webhook to rewrite the row. Stored features still win on conflict
(preserves per-user overrides). Mirrored explicitly in
convex/mcpProTokens.ts:55 (issueProMcpToken's direct ctx.db.query path
computes the same merge inline since it bypasses the query handler).
P2 — grant/issue path missing mcpAccess gate (let tier-1-without-mcpAccess
users complete OAuth then fail every tools/call at the gateway):
- mcp-grant-context.ts:95, mcp-grant-mint.ts:236, authorize-pro.ts:356,
convex/mcpProTokens.ts:55 now ALL gate on `tier ≥ 1 && mcpAccess === true`,
matching the downstream MCP-edge runProPreChecks check. Widens the
`getEntitlements` deps return type at all three handlers.
P3 — inline theme script blocked by global CSP:
- mcp-grant.html:34 inline `<script>` is removed (global CSP at vercel.json:92
is hash-allowlist-based and we don't allowlist this page's hashes).
- Theme application moved into the bundled module at src/mcp-grant-main.ts
(top of file, runs on module evaluation). Brief default-theme flash on
light-preference users is acceptable for a transient consent UI.
10 new tests covering: mcpAccess: false at each of the 4 gates,
mcpAccess: undefined (legacy row) at each of the 4 gates, and the
read-time catalog merge in both directions (legacy gets default,
override preserved).
Full gate post-fix:
- Convex 249/249 (+2 new merge tests).
- Edge/server 280/280 (+10 new gate tests).
- tsc both configs clean.
- sebuf contract clean (was failing before P1.2).
- Sentry coverage clean.
- deploy-config tests assert new explicit-source rules.
Plan: docs/plans/2026-05-10-001-feat-pro-mcp-clerk-auth-quota-plan.md
* fix(entitlement-cache): treat pre-U10 cache entries lacking mcpAccess as stale
Reviewer round-2 P2 (cache layer): _getEntitlementsImpl returned hot
Redis cache entries as-is, bypassing the read-time catalog merge that
convex/entitlements.ts:50 applies on the Convex read path. Paying users
with cache entries written before plan 2026-05-10-001 U10 deployed see
mcpAccess !== true at the grant/MCP gates and are blocked for up to
the 15-min cache TTL.
Cache predicate now also requires `typeof features.mcpAccess === 'boolean'`.
Legacy entries (mcpAccess: undefined) fall through to Convex, which
returns the merged shape and the cache is rewritten with the post-U10
layout. Self-healing, bounded to one extra Convex round-trip per
affected user during the migration window.
Targeted choice over alternatives:
- Importing convex/config/productCatalog into server/_shared to apply
the merge at the cache layer would pull non-edge-safe deps. Rejected.
- Treating ALL cached entries as stale would defeat the cache layer.
Rejected.
- typeof check on the new field is the minimum delta that fixes the
migration window without restructuring.
Test fixture in server/__tests__/entitlement-check.test.ts updated:
makeEntitlements now includes mcpAccess (tier ≥ 1 → true). Two new
focused tests:
1. Legacy cache entry without mcpAccess → cache rejected → Convex
round-trip → result returned (verifies fetch called once).
2. Post-U10 cache entry WITH mcpAccess → cache honored → no Convex
round-trip (verifies fetch called zero times).
Convex 251/251 (+2 new cache tests).
Plan: docs/plans/2026-05-10-001-feat-pro-mcp-clerk-auth-quota-plan.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix koala73#3339 — Broken Link
The World variant nav link in the header pointed to https://worldmonitor.app, which does not resolve DNS. All other variant subdomains (tech., finance., happy., commodity., energy.) resolve correctly.
Changed the World variant href from https://worldmonitor.app to https://www.worldmonitor.app (which resolves with HTTP 200 and 301-redirects to www).
Files: src/app/panel-layout.ts (1 URL change)