Skip to content

fix(resilience): repair externalDebtCoverage broken-data-source (mrv=1 + HIC=0)#3427

Merged
koala73 merged 2 commits into
mainfrom
fix/resilience-external-debt-coverage
Apr 26, 2026
Merged

fix(resilience): repair externalDebtCoverage broken-data-source (mrv=1 + HIC=0)#3427
koala73 merged 2 commits into
mainfrom
fix/resilience-external-debt-coverage

Conversation

@koala73

@koala73 koala73 commented Apr 26, 2026

Copy link
Copy Markdown
Owner

Summary

PR #3425's post-merge dry-run surfaced that externalDebtCoverage (0.25-weight recovery dimension) awards debtToReservesRatio: 0score 100 to 72/164 countries (44%) including NO/CH/DK/SE/FI/IS/KW/AE/SG/LU. A 25%-weighted dim scoring nearly half the universe at 100 is not discriminating.

Root cause (three layers)

  1. WB mrv=1 trap (memory feedback_wb_bulk_mrv1_null_coverage_trap) — silently drops late-reporters. Fixed via mrv=5 + per-country pickLatest, matching the pattern already used by seed-wb-external-debt.mjs.
  2. Number(null) === 0 defeats the latest-picker — caught by reviewer post-initial-push (commit b44e74bff). Without an explicit if (record?.value == null) continue BEFORE coercion, a late-arriving null overwrites an older non-null, defeating the entire mrv=5 fix. Now skips nulls upfront.
  3. WB IDS is LMIC-scoped — HICs return value: 0 (not null) for short-term external debt. Old code accepted 0 as legitimate, divided by reserves, scored 100. Fix: drop debt.value <= 0. HICs fall to the existing IMPUTE.recoveryExternalDebt fallback (50/0.3/unmonitored).

Same mrv=1 + null-skip fix also applied to seed-recovery-reserve-adequacy.mjs for the FI.RES.TOTL.MO indicator.

Propagation — manual ops required (do NOT skip)

The recovery seeders ship inside seed-bundle-resilience-recovery.mjs which uses a 30-day freshness gate (scripts/_bundle-runner.mjs:240 skips when elapsed < intervalMs * 0.8 ≈ 24 days). After merge, the new seeder code WILL NOT RUN on Railway for ~24 days because the bundle runner sees the prior canonical envelope as fresh. Meanwhile seed-resilience-scores keeps recomputing scores against stale recovery data — fix appears not to land.

Release checklist (run in order, post-merge, with prod env):

# 1. Force-refresh the two affected canonical keys (bypasses bundle runner gate)
node scripts/post-pr3427-force-refresh.mjs

# 2. Bulk-warm the v15 score cache against the corrected seed data
WORLDMONITOR_API_KEY=<key> node scripts/seed-resilience-scores.mjs

# 3. Re-run the cohort dry-run to capture empirical impact
PRE_RANKING_KEY=resilience:ranking:v14 npm run dryrun:resilience

The 72 false-100 countries should drop to IMPUTE 50 in step 2; rank movements (BS/SE/FI/NO/CH/etc. dropping) should appear in step 3's CSV.

Test plan

  • Run node scripts/post-pr3427-force-refresh.mjs — confirm both seeders run, see "Dropped N countries with debt=0" warning
  • Run WORLDMONITOR_API_KEY=<key> node scripts/seed-resilience-scores.mjs — confirm 222/222 cached
  • Re-run npm run dryrun:resilience — inspect rank deltas for sane reorderings
  • Verify /api/health.resilienceRanking stays OK

Lineage

Plan 2026-04-26-001 cohort dry-run → user audit findings #6+#7+#8 → this PR. Second seeder migrated to the mrv=5 + pickLatest recipe (first was seed-wb-external-debt.mjs in PR #3412).

Reviewer findings addressed in commit b44e74b

  • P1 — Number(null) === 0 defeats the picker — explicit null-skip before coercion in both seeders
  • P2 — bundle freshness gate delays propagation ~24 days — added scripts/post-pr3427-force-refresh.mjs one-shot + release checklist above

…1 trap + HIC=0 sentinel)

PR #3425's post-merge cohort dry-run surfaced that `externalDebtCoverage`
(0.25-weight recovery dimension) is awarding `debtToReservesRatio: 0` →
score 100 to **72/164 countries (44%)** — including NO/CH/DK/SE/FI/IS/
KW/AE/SG/LU. A 25%-weighted dimension that scores nearly half the
universe at 100 is not discriminating; it's structurally elevating
wealthy countries on a metric that should differentiate them.

Two-layered root cause:

1. **WB `mrv=1` coverage trap** (memory `feedback_wb_bulk_mrv1_null_coverage_trap`).
   `mrv=1` returns a SINGLE year across all countries with `value: null`
   for late-reporters; the script silently drops them. The mature
   pattern (already used by `seed-wb-external-debt.mjs` for the
   financialSystemExposure dim) is `mrv=5` + per-country
   pickLatest-non-null. This script was written before the memory was
   captured and never migrated.

2. **WB IDS dataset is LMIC-scoped.** `DT.DOD.DSTC.CD` (short-term
   external debt) returns the literal `0` (not `null`) for high-income
   countries that don't report into the IDS series — they're
   out-of-scope, not data-sparse. Under the prior code, `Number(0) → 0`
   passed `Number.isFinite()`, divided by reserves, and yielded
   `debtToReservesRatio: 0` → score 100. The construct semantically
   does not apply to HICs (which manage external debt through different
   channels), so dropping them is correct — they fall to the dim's
   existing IMPUTE fallback (`recoveryExternalDebt`: 50/0.3/'unmonitored').

**Same fix applied to `seed-recovery-reserve-adequacy.mjs`** which had
the same `mrv=1` trap on `FI.RES.TOTL.MO` (no HIC=0 sentinel issue
there because reserve-months is genuinely measured for HICs).

**Propagation:** No cache-prefix bump. Score cache `resilience:score:v15:`
has 12h TTL; the next bulk-warm tick after this seed lands will refresh
all 222 country scores against the new debt seed, so the 72 false-100
countries naturally drop to IMPUTE 50 within 12h. Manual immediate
warm: run `seed-recovery-external-debt.mjs` then `seed-resilience-scores.mjs`.

**Expected v15 ranking impact:** countries previously inflated by
false-perfect externalDebtCoverage will drop. The cohort dry-run
should be re-run post-deploy to capture the empirical signature; PR
#3425's `npm run dryrun:resilience` script does this.

**Audit lineage:** Plan 2026-04-26-001 cohort dry-run (commit
`ae7229e5e` post-merge) → user audit finding #7+#8 → this PR.
Memory `feedback_wb_bulk_mrv1_null_coverage_trap` is the established
recipe; this is the second seeder migrated to it (first was
`seed-wb-external-debt.mjs` in PR #3412).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@vercel

vercel Bot commented Apr 26, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
worldmonitor Ignored Ignored Preview Apr 26, 2026 10:03am

Request Review

@greptile-apps

greptile-apps Bot commented Apr 26, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes two compounding bugs in externalDebtCoverage seeding: the mrv=1 WB API trap (silently dropping late-reporters) is resolved by switching to mrv=5 + per-country pickLatest, and the IDS-scope sentinel (debt.value === 0 for HICs) is filtered out to prevent 72 countries from receiving a false-perfect score of 100. The same mrv=5 fix is applied to seed-recovery-reserve-adequacy.mjs.

  • The validate threshold of >= 80 in seed-recovery-external-debt.mjs was calibrated when HIC countries were included in the output; after intentionally dropping ~72 of them the expected output is ~92, leaving only a 12-country buffer — a data gap in LMIC coverage could cause validation failure and leave stale data in the cache.

Confidence Score: 3/5

Mergeable with awareness: the logic fixes are correct, but the validate threshold in seed-recovery-external-debt.mjs is now under-calibrated for the intentionally reduced output size, creating a real risk of a validation-triggered cache freeze on next seed run.

The core mrv=5 and HIC-filter fixes are sound and well-reasoned. However the validate threshold (>=80) is now only 12 countries above the expected post-drop output (~92 LMICs). A modest WB data gap could push the count below 80, causing the seeder to refuse to write and leaving the old data in cache.

scripts/seed-recovery-external-debt.mjs — validate threshold needs recalibration after intentional HIC exclusion

Important Files Changed

Filename Overview
scripts/seed-recovery-external-debt.mjs Two-part fix: mrv=1 to mrv=5 with per-country pickLatest, and debt<=0 filter to drop HIC false-100 scores. The mrv=5 change is correct and mirrors seed-wb-external-debt.mjs. The HIC filter is logically sound but the validate threshold (>=80) is now dangerously close to the expected post-drop country count (~92), with only a 12-country buffer.
scripts/seed-recovery-reserve-adequacy.mjs mrv=1 to mrv=5 fix with per-country pickLatest applied correctly. FI.RES.TOTL.MO is not LMIC-scoped so no sentinel-zero issue here; validate threshold (>=100) remains well-calibrated since HIC coverage improves rather than drops with this change.

Sequence Diagram

sequenceDiagram
    participant S as Seeder Script
    participant WB as World Bank API
    participant R as Redis Cache

    S->>WB: GET indicator/DT.DOD.DSTC.CD?mrv=5
    WB-->>S: Up to 5 records/country including HIC value=0
    S->>WB: GET indicator/FI.RES.TOTL.CD?mrv=5
    WB-->>S: Up to 5 records/country reserves
    Note over S: pickLatest per country for each indicator
    S->>S: debt<=0 drop HIC sentinel
    S->>S: reserves<=0 drop
    S->>S: Compute debtToReservesRatio
    S->>S: validate countries.length >= 80
    alt Validation passes
        S->>R: SET resilience:recovery:external-debt:v1
    else Validation fails
        S-->>S: FATAL stale data remains in cache
    end
Loading

Comments Outside Diff (1)

  1. scripts/seed-recovery-external-debt.mjs, line 106 (link)

    P1 Validation threshold now has a thin margin after HIC drop

    The validate function still requires >= 80 countries, but this PR intentionally drops ~72 HIC countries (debt ≤ 0), leaving an expected output of ~92 LMICs. That is only a 12-country buffer above the threshold. If WB data gaps cause a few more LMICs to be missing debt or reserves data, the seeder will fail validation and refuse to update the cache, leaving the stale (now-corrected) data in place. Since the threshold was calibrated when HICs were included in the output, it should be revisited now that they are intentionally excluded.

Reviews (1): Last reviewed commit: "fix(resilience): repair externalDebtCove..." | Re-trigger Greptile

}

if (droppedHicZeroDebt > 0) {
console.warn(`[recovery:external-debt] Dropped ${droppedHicZeroDebt} countries with debt=0 (HIC out-of-IDS-scope; would have falsely scored 100 on debtToReservesRatio).`);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 droppedHicZeroDebt label incorrectly attributes all debt <= 0 drops to HICs

The counter and warning message hard-code the assumption that any debt.value <= 0 entry is an HIC out-of-IDS-scope. An LMIC with genuinely zero short-term external debt would also be silently dropped and mislabeled as HIC in the warning. The log message should reflect the actual filter condition:

Suggested change
console.warn(`[recovery:external-debt] Dropped ${droppedHicZeroDebt} countries with debt=0 (HIC out-of-IDS-scope; would have falsely scored 100 on debtToReservesRatio).`);
console.warn(`[recovery:external-debt] Dropped ${droppedHicZeroDebt} countries with debt<=0 (assumed HIC out-of-IDS-scope or zero-debt; would have falsely scored 100 on debtToReservesRatio).`);

…e-refresh script

Two reviewer findings on PR #3427's initial commit:

**P1: Number(null) === 0 defeats the latest-non-null picker.**
The mrv=5 + pickLatest fix was structurally correct but `Number(null)`
returns `0` (not `NaN`), passes `Number.isFinite()`, and lets a
`value: null` record overwrite an older non-null record in the
year-comparison branch. Net effect: the very late-reporters mrv=5
was supposed to capture (KW/QA/AE) get a final `value: 0` from a
recent null record, then the country-level `debt.value <= 0` HIC
filter drops them entirely — same end-state as before, fix defeated.

Fix: explicit `if (record?.value == null) continue` BEFORE coercion
in both seeders. Same pattern as the standard recipe documented in
memory `feedback_wb_bulk_mrv1_null_coverage_trap`. Reviewer caught
this within minutes of the initial push.

**P2: Bundle-runner freshness gate would delay propagation by ~24d.**
`scripts/_bundle-runner.mjs:240` skips bundle items where
`elapsed < intervalMs * 0.8`. For the 30d recovery bundle, that means
the new seeder code wouldn't RUN on Railway for ~24 days post-merge
even though the code is in the container. Meanwhile
`seed-resilience-scores` would keep recomputing scores against the
old debt seed → fix appears not to land.

Fix: `scripts/post-pr3427-force-refresh.mjs` — one-shot post-merge
script that invokes both seeders directly (bypassing the bundle
runner's freshness gate). Runs the two seeders in series, then
prints the follow-up bulk-warm command. Documented in the script
header and to be referenced in PR #3427's release checklist.

Both fixes are surgical — same files, no logic re-architecture.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@koala73
koala73 merged commit ebbf99b into main Apr 26, 2026
10 checks passed
@koala73
koala73 deleted the fix/resilience-external-debt-coverage branch April 26, 2026 10:15
koala73 added a commit that referenced this pull request Apr 26, 2026
…cher (review fixup on PR #3432)

Reviewer found one more WB seeder I missed in the original PR #3432 sweep:

**P1: `scripts/seed-wb-external-debt.mjs:69` had the exact same compound trap.**
`Number(record?.value)` before any null check, then year-based latest-picker
overwrites. Unlike the SWF imports picker (where `value <= 0` accidentally
catches Number(null)=0 because imports must be positive), this script's
downstream `combineExternalDebt` filter at `:98` only rejects negative
debt — `debt.value < 0`, not `<= 0`. So a `value: null` record from a
late-reporting LMIC (KW/QA/AE publish IDS data 1-2y behind G7) would
coerce to `0`, win the year comparison, and propagate as a false
`debtToReservesRatio: 0` → `0% short-term-debt-to-GNI` for the country.

This feeds `economic:wb-external-debt:v1` consumed by the new
`financialSystemExposure` dim (PR #3412 / #3407), so the false 0% would
silently inflate the dim's score for affected LMICs.

Fix: same recipe as PRs #3427 and #3432 — explicit
`if (record?.value == null) continue;` BEFORE `Number()` coercion.

**Defense-in-depth: `scripts/seed-bis-lbs.mjs:204` (the WB GDP fetcher
inside the BIS LBS seeder) also gets the explicit null-skip.**
This one was technically safe because GDP > 0 is a real-world invariant
and the `value <= 0` filter catches Number(null)=0 by side effect. But
per the lesson now codified in skill `wb-bulk-mrv1-null-coverage-trap`,
the protection is fragile — future indicator-family changes that
relax `value <= 0` would silently lose null protection. Adding the
explicit null-skip makes the picker null-safe regardless of filter
relaxation.

**End state after this commit + PR #3427 + PR #3432 + this fixup:**
ALL WB-bulk seeders in `scripts/seed-*.mjs` either (a) have an explicit
`value == null` skip BEFORE coercion, or (b) use a non-WB endpoint
shape (BIS SDMX, IMF SDMX, etc.) where the trap doesn't apply.
Final sweep recipe documented in the commit; next reviewer can run it
verbatim to audit any new seeder.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
koala73 added a commit that referenced this pull request Apr 26, 2026
…rs (#3432)

* fix(resilience): apply mrv=5 + null-skip recipe to remaining 4 WB seeders

PR #3427 fixed the mrv=1 + Number(null)=0 compound trap in 2 recovery
seeders (external-debt + reserve-adequacy). A subsequent sweep
(`grep -lE "mrv=1[^0-9]" scripts/*.mjs`) found 4 OTHER seeders with
the same trap shape:

- `seed-fossil-electricity-share.mjs` (EG.ELC.FOSL.ZS)
- `seed-low-carbon-generation.mjs` (EG.ELC.{NUCL,RNEW,HYRO}.ZS)
- `seed-power-reliability.mjs` (EG.ELC.LOSS.ZS)
- `seed-sovereign-wealth.mjs` (NE.IMP.GNFS.CD via pickLatestPerCountry)

All four migrated to the established recipe:
1. mrv=1 → mrv=5 + per-country pickLatest
2. per_page=500 → per_page=2000 (5x records per country = need bigger page)
3. Explicit `if (record?.value == null) continue` BEFORE Number() coercion
4. Year sanity check + per-country latest comparison

**Why explicit null-skip matters more for the energy seeders:**
The 3 energy seeders use "% of" indicators (EG.ELC.{FOSL,NUCL,RNEW,
HYRO,LOSS}.ZS) where 0 IS a legitimate value (country has 0% nuclear,
0% fossil, perfect grid reliability, etc.). The accidental
null-protection trick `if (value <= 0) continue` from the SWF picker
WOULD WRONGLY DROP legitimate zeros for these. Must skip null
explicitly so the `value === 0` case can flow through to scoring.

**`seed-sovereign-wealth.mjs` was already mostly correct** (mrv=5 +
pickLatest helper + accidental null protection via `value <= 0` since
imports must be > 0). Added explicit null-skip as defense-in-depth so
a future copy-paste of `pickLatestPerCountry` for a different
indicator family doesn't silently lose null protection.

**Lineage:** Plan 2026-04-26-001 cohort dry-run → user audit Priority
Check #4 ("Replace mrv=1 with mrv=5 + latest non-null in ALL seeders")
→ this PR. Companion to PR #3427.

**Memory `wb-bulk-mrv1-null-coverage-trap`** was updated post-PR-#3427
with the explicit `Number(null) === 0` warning + production case
study — that memory is the recipe these 4 seeders now follow.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* fix(resilience): null-skip in seed-wb-external-debt + bis-lbs GDP fetcher (review fixup on PR #3432)

Reviewer found one more WB seeder I missed in the original PR #3432 sweep:

**P1: `scripts/seed-wb-external-debt.mjs:69` had the exact same compound trap.**
`Number(record?.value)` before any null check, then year-based latest-picker
overwrites. Unlike the SWF imports picker (where `value <= 0` accidentally
catches Number(null)=0 because imports must be positive), this script's
downstream `combineExternalDebt` filter at `:98` only rejects negative
debt — `debt.value < 0`, not `<= 0`. So a `value: null` record from a
late-reporting LMIC (KW/QA/AE publish IDS data 1-2y behind G7) would
coerce to `0`, win the year comparison, and propagate as a false
`debtToReservesRatio: 0` → `0% short-term-debt-to-GNI` for the country.

This feeds `economic:wb-external-debt:v1` consumed by the new
`financialSystemExposure` dim (PR #3412 / #3407), so the false 0% would
silently inflate the dim's score for affected LMICs.

Fix: same recipe as PRs #3427 and #3432 — explicit
`if (record?.value == null) continue;` BEFORE `Number()` coercion.

**Defense-in-depth: `scripts/seed-bis-lbs.mjs:204` (the WB GDP fetcher
inside the BIS LBS seeder) also gets the explicit null-skip.**
This one was technically safe because GDP > 0 is a real-world invariant
and the `value <= 0` filter catches Number(null)=0 by side effect. But
per the lesson now codified in skill `wb-bulk-mrv1-null-coverage-trap`,
the protection is fragile — future indicator-family changes that
relax `value <= 0` would silently lose null protection. Adding the
explicit null-skip makes the picker null-safe regardless of filter
relaxation.

**End state after this commit + PR #3427 + PR #3432 + this fixup:**
ALL WB-bulk seeders in `scripts/seed-*.mjs` either (a) have an explicit
`value == null` skip BEFORE coercion, or (b) use a non-WB endpoint
shape (BIS SDMX, IMF SDMX, etc.) where the trap doesn't apply.
Final sweep recipe documented in the commit; next reviewer can run it
verbatim to audit any new seeder.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* docs: fix stale 'mrv=1' top-comment in seed-low-carbon-generation.mjs

Reviewer follow-up on PR #3432: the top-of-file rationale comment in
`seed-low-carbon-generation.mjs:22` still claimed "We fetch the
most-recent value (mrv=1)" even though the actual fetch was migrated
to mrv=5 + null-skip in commit dd0be80. Non-blocking but worth
cleaning up — this is the same doc-drift pattern memory
`feedback_doc_drift_after_behavior_fix_needs_grep_sweep` warns about.

Updated the comment to describe the current mrv=5 + per-country
pickLatest behavior and reference the relevant skill + this PR.

The other mrv=1 references in PR #3432's touched files are intentional
— they appear inside rationale comments that explain the trap before
documenting why the seeder uses mrv=5 instead. Those should NOT change.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>
koala73 added a commit that referenced this pull request Apr 26, 2026
#3435)

Reviewer P2 finding: get-resilience-ranking.ts returns the cached
resilience:ranking:v15 payload BEFORE calling listScorableCountries(),
and the cache key remains v15 across this PR. So after deploy + static
v8 reseed, normal API requests can still return the pre-PR cached
222-country ranking for up to the 12h ranking TTL — the new
handler-side universe filter at _shared.ts:listScorableCountries
only runs on the recompute path, not on cache hits.

Fix: add the universe filter to the cached-response read path too.
The filter is idempotent — a fresh post-PR-1 cached payload is
already universe-filtered (no-op), a stale pre-PR-1 cached payload
gets filtered at handler-time. Same recipe as the prior 3-layer
defense pattern (write-time + read-time + handler-time).

The rankable-universe contract is now enforced at FOUR layers:
1. seed-resilience-static.mjs:finalizeCountryPayloads (write-time)
2. seed-resilience-scores.mjs (read-time defense)
3. _shared.ts:listScorableCountries (recompute-path defense)
4. get-resilience-ranking.ts cache hit (cached-response defense — new)

This eliminates the operator-toil failure mode entirely. No need for
the post-merge force-refresh checklist that PR #3427 required, and no
need to bump the ranking cache key (which would be wasted churn since
PR 3 of this plan will bump it for the coverage-penalty redesign
anyway).

Test fixup: tests/resilience-ranking.test.mts:382 used 'ZZ' as a
sentinel to verify the auth gate. The new universe filter correctly
drops 'ZZ' as non-rankable, defeating the auth-gate test's intent.
Changed sentinel to 'NR' (Nauru — UN member, real country, but won't
appear in the recompute path's static-index fixture).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
koala73 added a commit that referenced this pull request Apr 26, 2026
…§U1, PR 0) (#3433)

Lands the structural CI gate the universe + coverage rebuild needs.
Passes against current v15 ranking with PERMISSIVE thresholds; PRs 3+5
will tighten the thresholds in lockstep with their corresponding fixes.

**Why this exists.** PR #3425's post-merge cohort dry-run showed that
even after targeted scorer fixes (logisticsSupply default, sovereignFiscalBuffer
Path 3, socialCohesion GPI-only impute) AND the externalDebtCoverage
data-source repair (PR #3427), tiny states still climb in v15 (TV rank
12→6, PW 14→7, NR 58→36). The targeted fixes can't reach the universe
+ coverage handling defects. This fixture pins the cohort-level
invariants so the rebuild PRs (universe filter, coverage penalty,
stable-absence recalibration, per-capita normalization) can each
demonstrate measurable cohort movement at merge time.

**What it asserts (PERMISSIVE baselines passing on current v15).**

| Invariant                                    | v15 measurement | PR 0 threshold |
|----------------------------------------------|-----------------|----------------|
| median(G7) − median(microstate-territories)  | **−5.52pt**     | > −10pt        |
| median(Nordics) − median(GCC)                | **+8.20pt**     | ≥ −30pt        |
| min(G7) − max(Sub-Saharan-LIC)               | **+10.40pt**    | ≥ −20pt        |
| microstate-territories in top 20             | **6** (AD, LI, TV, PW, GL, MO) | report-only |

The G7-vs-microstate inversion (−5.52pt) is the structural problem the
rebuild exists to fix. PR 0 baseline tolerates it up to 10pt; PR 3
(coverage penalty) is expected to flip the sign and tighten to +10pt.

**Files.**

- 5 cohort JSON fixtures at server/worldmonitor/resilience/v1/cohorts/
  (g7, nordics, gcc, sub-saharan-lic, microstate-territories) — committed
  data, not test code, so cohort membership can be reviewed independently
- tests/resilience-cohort-anti-inversion.test.mts — fixture self-tests
  (always run) + live-ranking invariants (skip-on-missing-creds)
- tests/resilience-retired-dimensions-parity.test.mts — extended with
  cohort JSON shape validation (anti-mystery-cohort gate: every cohort
  must carry a non-empty description explaining its purpose)

**Read pattern.** Reads live `resilience:ranking:vN` from production
Upstash via the read-only GET pattern from
scripts/dry-run-resilience-rebalance.mjs (PR #3425). Skips gracefully
when UPSTASH_REDIS_REST_URL is unset — required for CI-without-prod-creds
and dev-without-.env environments. The cache-key import is dynamic
(`await import('../server/.../_shared.ts')`) so the test always reads
the current cache version, never a hardcoded literal — same source-of-truth
pattern PR #3425 established for the dry-run script.

**Coverage anomaly.** microstate-territories cohort has 12/13 present
in v15 (FK Falklands missing — a UK territory not in WB IDS scope, so
expected). The other 4 cohorts are 100% covered. Reported in the
diagnostic `it()` block.

**Verification.**
- 617/617 resilience tests pass (`npx tsx --test tests/resilience-*`)
  — 594 prior + 23 new cohort + parity tests
- Live-invariants pass against current v15 with prod creds
- Skip-on-missing-creds path passes without `.env` loaded

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <[email protected]>
koala73 added a commit that referenced this pull request Apr 26, 2026
…6-04-26-002 PR 1 / U2) (#3435)

* feat(resilience): universe filter — 193 UN members + 3 SARs (Plan 2026-04-26-002 §U2, PR 1)

Lands the rankable-universe whitelist filter at universe-build time.
Earlier behavior: `seed-resilience-static.mjs` admitted every ISO2 from
any source map (~222 entries including AS/GU/GL/IM/GI/FK/PR/etc.).
Post-PR-1: ~196 entries (193 UN members + 3 SARs).

**Decision (per plan KTD Q1)**: Option (b) — UN members + 3 standalone
SARs (HK, MO, TW). The 3-SAR exception preserves their presence in
the dataset; the future `headlineEligible` gate (PR 6) can separate
them from the headline ranking if/when that policy ships.

**Single source of truth.** `server/worldmonitor/resilience/v1/cohorts/sovereign-status.json`
holds the 196-entry whitelist. `scripts/shared/rankable-universe.mjs`
is the helper both seeders consume to ensure their universes match.

**Filter sites (both seeders).**
- `seed-resilience-static.mjs:finalizeCountryPayloads` — drops non-rankable
  entries at universe-build time. Logs `Dropped N non-rankable territory entries`.
- `seed-resilience-scores.mjs` — defense-in-depth filter on the static
  index read, in case the index was seeded by an older version of
  seed-resilience-static during a deploy transient.

**No score-formula changes.** This is a universe-membership change
only. Existing payload shape and per-country score computation
unchanged. The cohort anti-inversion fixture (PR 0 / PR #3433) thresholds
remain PERMISSIVE; PR 3 (coverage penalty) is the next step that
tightens them.

**Testing.**
- `tests/resilience-universe-filter.test.mts` (new, 15 tests):
  - Whitelist shape: 193 UN + 3 SARs = 196, no duplicates
  - SAR cohort = exactly {HK, MO, TW}
  - `isInRankableUniverse` accepts UN members + SARs, rejects territories
    (AS, GU, GL, IM, GI, FK, PR, BM, KY, etc.) and non-UN entities
    (XK Kosovo, PS Palestine, VA Vatican, EH Western Sahara)
  - Case-insensitive iso2 handling
  - Invalid input shapes (empty, ISO3, null, undefined) all return false
- `npm run test:data` — 609/609 resilience tests pass
- `npm run typecheck` clean

**Lineage.** Plan 2026-04-26-002 §U2 (PR 1 of 8). Stacked on PR 0
(#3433) which already shipped the cohort anti-inversion fixture.
This PR's filter directly affects the microstate-territories cohort
in PR 0's fixture (4 of 13 entries — GL, GI, IM, FK — drop out of
the live ranking once this lands), but the PERMISSIVE thresholds
remain valid.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* fix(resilience): bump static-source-version v7→v8 + handler-side universe filter (review fixup on PR #3435)

Two reviewer findings on PR #3435's initial commit:

**P1: static seeder would silently no-op post-merge.**
`shouldSkipSeedYear` (seed-resilience-static.mjs:70) skips when existing
meta has the same `sourceVersion` AND `seedYear`. Prod already has a
successful 2026 v7 static seed, so running the seeder post-merge would
log "already written, skipping" — the new whitelist filter at
`finalizeCountryPayloads` would never run, and `resilience:static:index:v1`
would remain at ~222 entries. The universe filter wouldn't take effect.

Fix: bumped `RESILIENCE_STATIC_SOURCE_VERSION` v7→v8. The next seeder
run after merge will see the new version, treat it as a fresh seed, and
apply the universe filter.

**P2: handler-side defense-in-depth was incomplete.**
`scripts/seed-resilience-scores.mjs` filtered its local `countryCodes`,
but then it calls `/api/resilience/v1/get-resilience-ranking?refresh=1`
which re-reads `listScorableCountries()` from Redis at
`server/worldmonitor/resilience/v1/_shared.ts:661`. That helper was
returning `manifest.countries` unfiltered. So even with the script's
local filter, a stale 222-country manifest could still drive the ranking
endpoint to publish 222 countries.

Fix:
- New `server/worldmonitor/resilience/v1/_rankable-universe.ts` —
  server-side TS mirror of `scripts/shared/rankable-universe.mjs`. Both
  read the SAME canonical JSON at `cohorts/sovereign-status.json`; the
  duplication is the read-path (fs.readFileSync vs ES JSON import), not
  the data.
- `_shared.ts:listScorableCountries` now applies `isInRankableUniverse`
  as a final filter. Idempotent: a fresh manifest from the post-bump
  seeder is already filtered, so the filter is a no-op then; a stale
  pre-PR-1 manifest gets filtered at handler-time.

The rankable-universe contract is now enforced at three points:
1. seed-resilience-static.mjs:finalizeCountryPayloads (write-time)
2. seed-resilience-scores.mjs (read-time defense)
3. _shared.ts:listScorableCountries (handler-time defense)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* fix(resilience): move sovereign-status.json out of cohorts/ to avoid PR #3433 shape gate (review fixup on PR #3435)

Cross-PR finding: PR #3433 added a cohort JSON shape gate that scans
every *.json in `server/worldmonitor/resilience/v1/cohorts/` and asserts
the cohort schema (`{ name, description, iso2: string[] }`). PR #3435
added `sovereign-status.json` to the same directory, but its shape is
a per-country PROPERTY registry (`{ entries: [{iso2, status}] }`), not
a cohort membership list. After both PRs merge, the shape gate fails
on sovereign-status.json.

Fix: directory split. Cohorts (membership lists) stay in `cohorts/`;
per-country property registries move to `registries/`. The shape gate
naturally narrows to homogeneous-shape files.

**Layout post-fix:**
- `server/worldmonitor/resilience/v1/cohorts/` — cohort membership
  lists (PR #3433's g7/nordics/gcc/sub-saharan-lic/microstate-territories)
- `server/worldmonitor/resilience/v1/registries/` — per-country
  property registries (sovereign-status; future Q4 deferred work could
  add landlocked, etc.)

**Updates:**
- Moved `cohorts/sovereign-status.json` → `registries/sovereign-status.json`
- `scripts/shared/rankable-universe.mjs` — path updated + comment
  documenting the directory-split rationale
- `server/worldmonitor/resilience/v1/_rankable-universe.ts` — path +
  comment updated
- `tests/resilience-universe-filter.test.mts` — path updated

**Verified by simulating post-merge state:** copied PR #3433's cohort
JSONs + extended parity test into this branch locally; both PRs'
tests pass (8/8 cohort + 15/15 universe-filter). Reverted the
simulation files before commit — only the PR #3435 intended changes
ship.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* fix(resilience): filter cached ranking response too (review fixup on PR #3435)

Reviewer P2 finding: get-resilience-ranking.ts returns the cached
resilience:ranking:v15 payload BEFORE calling listScorableCountries(),
and the cache key remains v15 across this PR. So after deploy + static
v8 reseed, normal API requests can still return the pre-PR cached
222-country ranking for up to the 12h ranking TTL — the new
handler-side universe filter at _shared.ts:listScorableCountries
only runs on the recompute path, not on cache hits.

Fix: add the universe filter to the cached-response read path too.
The filter is idempotent — a fresh post-PR-1 cached payload is
already universe-filtered (no-op), a stale pre-PR-1 cached payload
gets filtered at handler-time. Same recipe as the prior 3-layer
defense pattern (write-time + read-time + handler-time).

The rankable-universe contract is now enforced at FOUR layers:
1. seed-resilience-static.mjs:finalizeCountryPayloads (write-time)
2. seed-resilience-scores.mjs (read-time defense)
3. _shared.ts:listScorableCountries (recompute-path defense)
4. get-resilience-ranking.ts cache hit (cached-response defense — new)

This eliminates the operator-toil failure mode entirely. No need for
the post-merge force-refresh checklist that PR #3427 required, and no
need to bump the ranking cache key (which would be wasted churn since
PR 3 of this plan will bump it for the coverage-penalty redesign
anyway).

Test fixup: tests/resilience-ranking.test.mts:382 used 'ZZ' as a
sentinel to verify the auth gate. The new universe filter correctly
drops 'ZZ' as non-rankable, defeating the auth-gate test's intent.
Changed sentinel to 'NR' (Nauru — UN member, real country, but won't
appear in the recompute path's static-index fixture).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* fix(resilience): plain JSON import (no `with { type: 'json' }`) for Vercel esbuild compat (review fixup on PR #3435)

Reviewer P1 finding — Vercel deployment failure:

```
vc-file-system:__vc__ns__/12/server/worldmonitor/resilience/v1/_rankable-universe.js:26:65:
ERROR: Expected ";" but found "with"
```

Vercel's esbuild bundler does NOT support the ES import-attribute
syntax (`with { type: 'json' }`) used in the initial _rankable-universe.ts.
Existing server TS JSON imports in this codebase all use the plain
form WITHOUT import attributes — see e.g.
`server/worldmonitor/resilience/v1/_dimension-scorers.ts:1-2`:

```ts
import countryNames from '../../../../shared/country-names.json';
import iso2ToIso3Json from '../../../../shared/iso2-to-iso3.json';
```

Same shape applied here:

```ts
import sovereignStatus from './registries/sovereign-status.json';
```

Verified locally: typecheck clean, 609/609 resilience tests pass,
and `npx esbuild --bundle --platform=neutral --loader:.json=json`
produces a clean bundle (mimics Vercel's bundle path).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant