Skip to content

fix(resilience): targeted construct fixes for small-state bias (A+B+C)#3425

Merged
koala73 merged 5 commits into
mainfrom
fix/resilience-small-state-bias
Apr 26, 2026
Merged

fix(resilience): targeted construct fixes for small-state bias (A+B+C)#3425
koala73 merged 5 commits into
mainfrom
fix/resilience-small-state-bias

Conversation

@koala73

@koala73 koala73 commented Apr 26, 2026

Copy link
Copy Markdown
Owner

Summary

Three score-shape defects were over-promoting tiny states and under-promoting non-SWF advanced economies in the v14 ranking. This PR fixes all three in lockstep with a single cache prefix bump (v14→v15, history v9→v10).

Fix Dim Defect Impact
A logisticsSupply tradeExposure=0.5 default extended global-stress neutralizer to tiny states with no trade-to-GDP data TV/PW/NR cohort gain ~3-5pt
B sovereignFiscalBuffer Path 3 returned score=0, cov=1.0 (substantive absence) for non-SWF countries DE/JP/FR/IT/UK ~50pt swing on 0.5-weight dim
C socialCohesion When displacement+unrest both missing, score collapsed to GPI alone Tiny peaceful states ~93 → ~80

Plan: docs/plans/2026-04-26-001-fix-resilience-small-state-bias-plan.md (5-round Codex-reviewed, APPROVED).

Implementation

  • U1 — Fix A: removed tradeExposure=0.5 default in scoreLogisticsSupply (_dimension-scorers.ts:1387). Closed economies WITH observed tradeToGdp keep the 100*(1-tradeExposure) neutralizer.
  • U2 — Fix C: gated GPI-only impute in scoreSocialCohesion (_dimension-scorers.ts:1623). New IMPUTE.socialCohesionGpiOnlyDisplacement (70/0.6) and socialCohesionGpiOnlyUnrest (70/0.5) fire ONLY in degraded GPI-only mode. Iceland-shape (observed displacement + zero unrest) keeps historical "stable-absence ≈ 85" anchor.
  • U3 — Fix B: scoreSovereignFiscalBuffer Path 3 reframed to score=0, coverage=0, observedWeight=0 (dim-not-applicable). Score stays numeric per ResilienceDimensionScore.score:number contract. New RESILIENCE_NOT_APPLICABLE_WHEN_ZERO_COVERAGE set; computeLowConfidence + computeOverallCoverage apply the filter so non-SWF advanced economies don't get falsely flagged as low-confidence.
  • U4 — Cache bump: v14→v15 score+ranking, v9→v10 history. 11 hardcoded literal sites updated (tests/, scripts/, api/health.js).
  • U5 — Cohort regression test: tests/resilience-cohort-bias-anchors.test.mts (9 tests) pins TV/PW/NR/DE/JP contracts so future construct edits can't silently re-introduce the bias.
  • U6 — Methodology docs: v15 change-log entries for all three dims in docs/methodology/country-resilience-index.mdx.
  • U7 — Dry-run script: scripts/dry-run-resilience-rebalance.mjs (npm run dryrun:resilience). Read-only Upstash, CI-gated, never logs Redis URL/token, writes CSV under tmp/. Population from economic:imf:labor:v1 (ImfLaborEntry.populationMillions), NOT static-country payload.

Test plan

  • npx tsx --test tests/resilience-*.test.m{ts,js}591/591 pass
  • npm run typecheck — clean (root + api configs)
  • npm run lint:md — clean
  • U5 cohort anchor tests pass with all 3 fixes; FAIL if any single fix is reverted (verified)
  • Existing tests updated for new contract:
    • resilience-construct-invariants.test.mts — Path 3 assertion flipped to coverage=0
    • resilience-release-gate.test.mts — extended allow-list with NA_FOR_SOME_COUNTRIES_DIMENSIONS
    • resilience-dimension-scorers.test.mts — Path 3 + scoreLogisticsSupply existing test updated for new contract
  • Post-merge: bulk-warm v15 via seed-resilience-scores.mjs cron (or manual)
  • Post-merge: PRE_RANKING_KEY=resilience:ranking:v14 npm run dryrun:resilience to validate cohort movement before v14 TTL expires (12h window)
  • Post-merge: verify /api/health.resilienceRanking flips to OK (was STALE_SEED from v14 key being stale)

Cohort anchors (success criteria)

Country v14 dim v15 expected
TV logisticsSupply ~95, socialCohesion ~93 logisticsSupply ≤80, socialCohesion ≤83
PW, NR same as TV same direction
DE, JP sovereignFiscalBuffer score=0, cov=1.0 sovereignFiscalBuffer score=0, cov=0
Top 20 12 of 20 are pop<11M ≤9 of 20 (validated by U7 dry-run post-merge)

🤖 Generated with Claude Code

Co-Authored-By: Claude [email protected]

…, plan 2026-04-26-001)

Three score-shape defects were over-promoting tiny states and
under-promoting non-SWF advanced economies in the v14 ranking. This
PR fixes all three in lockstep with a single cache prefix bump.

**Fix A — `scoreLogisticsSupply`** (`_dimension-scorers.ts:1387`)
The `100 × (1 - tradeExposure)` neutralizer correctly suppresses
global-stress penalties for closed economies, but the prior 0.5
default for missing tradeToGdp extended that suppression to tiny
states with no trade-to-GDP data at all (TV, PW, NR). Now: missing
tradeToGdp drops the exposure-weighted components (cov derate to
0.5) instead of imputing them at "average openness". Closed
economies WITH observed tradeToGdp keep the neutralizer.

**Fix C — `scoreSocialCohesion`** (`_dimension-scorers.ts:1623`)
Tiny peaceful states with no displacement registry entry were
collapsing to GPI alone (cov 0.55, score ~93). New gated impute:
when displacement is absent, impute at 70/0.6; when ALSO unrest is
absent (GPI-only mode), impute unrest at 70/0.5; otherwise (observed
displacement + zero unrest), keep historical "stable-absence ≈ 85"
anchor matching `IMPUTE.unhcrDisplacement` so Iceland/Norway don't
regress. Per-row imputation flags do NOT bubble up: dim-level
`imputationClass` stays null when GPI is observed.

**Fix B — `scoreSovereignFiscalBuffer`** (`_dimension-scorers.ts:1923`)
Non-SWF advanced economies (DE, JP, FR, IT, UK, US, NL, AT, BE, ES,
PT) were scoring 0/cov=1.0 — a deliberate "substantive absence"
penalty that empirically over-fired for countries holding reserves
through Treasury / central-bank channels. Reframed Path 3 to
"dim-not-applicable" (`score=0, coverage=0, observedWeight=0`):
score stays numeric per the `ResilienceDimensionScore.score:number`
contract; coverage=0 removes the dim from the recovery-domain
coverage-weighted mean. New `RESILIENCE_NOT_APPLICABLE_WHEN_ZERO_COVERAGE`
set in `_dimension-scorers.ts` mirrors the existing
`RESILIENCE_RETIRED_DIMENSIONS` pattern but gates exclusion at the
country level (zero-coverage = not-applicable for THIS country)
rather than the construct level (retired for ALL countries).
`computeLowConfidence` and `computeOverallCoverage` in `_shared.ts`
both apply the new filter so non-SWF advanced economies don't get
falsely flagged as low-confidence.

**U4 — Cache prefix bump v14→v15 + history v9→v10**
Source-of-truth in `_shared.ts`. 11 hardcoded literal sites updated
across `tests/`, `scripts/`, `api/health.js`. History bump in
lockstep prevents pre/post-fix points from mixing inside the 30-day
rolling trend window (memory: cache-prefix-bump-propagation-scope).

**U5 — Cohort anchor regression test**
`tests/resilience-cohort-bias-anchors.test.mts` (9 tests) pins the
v15 contract for TV/PW/NR (logisticsSupply ≤80, socialCohesion ≤83)
and DE/JP (sovereignFiscalBuffer score=0/cov=0/obsWeight=0). Future
construct edits cannot silently re-introduce the bias.

**U6 — Methodology docs** (`docs/methodology/country-resilience-index.mdx`)
v15 change-log entries for all three dims. The `sovereignFiscalBuffer`
section explicitly documents the construct reframing from
"substantive absence" to "dim-not-applicable" with rationale.

**U7 — Full-country dry-run script** (`scripts/dry-run-resilience-rebalance.mjs`)
One-shot validation runner (`npm run dryrun:resilience`). Reads
PRE/POST ranking snapshots from production Upstash (read-only),
emits CSV under `tmp/`, validates top-20 small-state composition
(target: drop from ~12 to ≤9). CI-gated, defense-in-depth path
validation, never logs Redis URL/token. Population sourced from
`economic:imf:labor:v1` (`ImfLaborEntry.populationMillions`), NOT
the static-country payload.

**Verification**
- 591/591 resilience tests pass (`npx tsx --test tests/resilience-*`).
- `npm run typecheck` clean (root + api configs).
- `npm run lint:md` clean.
- Plan reviewed across 5 Codex rounds before implementation
  (`docs/plans/2026-04-26-001-fix-resilience-small-state-bias-plan.md`).

**Post-merge**
Bulk-warm v15 via `seed-resilience-scores.mjs` cron (or manual).
Run `PRE_RANKING_KEY=resilience:ranking:v14 npm run dryrun:resilience`
to confirm cohort movement before the v14 TTL expires.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@mintlify

mintlify Bot commented Apr 26, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
WorldMonitor 🟢 Ready View Preview Apr 26, 2026, 7:37 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@vercel

vercel Bot commented Apr 26, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
worldmonitor Ignored Ignored Preview Apr 26, 2026 8:48am

Request Review

@greptile-apps

greptile-apps Bot commented Apr 26, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes three score-shape defects in the v14 resilience ranking that over-promoted small states and under-promoted non-SWF advanced economies, with coordinated cache prefix bumps (score v14→v15, history v9→v10) and a new cohort regression test suite. The fixes for A (logisticsSupply tradeExposure default) and B (sovereignFiscalBuffer coverage=1.0coverage=0) are correctly implemented and well-tested.

  • Fix C (scoreSocialCohesion) has a P1 defect: the unrest-impute mode selector at line 1717 checks displacementMetric == null, but that is true both when the country is genuinely absent from the displacement registry and when the displacement seed is temporarily unavailable. During a displacement seed outage, peaceful countries that are in the registry (Iceland, Norway) receive the lower 70 GPI-only impute instead of the stable-absence 85, regressing their score. The fix is a one-character change: displacementRaw != null && displacementMetric == null. The seed-outage test doesn't cover the zero-unrest + outage path, so this was not caught by the U5 test suite.

Confidence Score: 3/5

Not safe to merge as-is — Fix C has a P1 defect that causes Iceland/Norway to regress during displacement seed outages due to a conflated null-check.

One P1 bug in a core scoring path (scoreSocialCohesion) where the GPI-only mode guard is too broad, affecting all peaceful countries with observed displacement whenever the displacement seed is temporarily unavailable. The test suite explicitly doesn't cover this path.

server/worldmonitor/resilience/v1/_dimension-scorers.ts (line 1717) and tests/resilience-dimension-scorers.test.mts (seed-outage test needs a zero-unrest variant).

Important Files Changed

Filename Overview
server/worldmonitor/resilience/v1/_dimension-scorers.ts Core scorer changes for Fixes A/B/C; Fix C contains a P1 bug where displacementMetric == null in the unrest impute block fires during seed outages as well as GPI-only mode, incorrectly regressing Iceland/Norway.
server/worldmonitor/resilience/v1/_shared.ts Cache prefix bumps (v14→v15, v9→v10), RESILIENCE_NOT_APPLICABLE_WHEN_ZERO_COVERAGE set added, and both computeLowConfidence + computeOverallCoverage updated to filter the SWF dim when coverage=0 — all correct.
tests/resilience-cohort-bias-anchors.test.mts New U5 cohort regression tests cover the three fix cohorts well; does not include a zero-unrest + displacement-seed-outage scenario for Iceland-shape countries, leaving the P1 path untested.
tests/resilience-dimension-scorers.test.mts Updated seed-outage test for socialCohesion uses present-events unrest, so zero-unrest + outage edge case is not covered; all other updated assertions are correct.
scripts/dry-run-resilience-rebalance.mjs New read-only dry-run validation script; CI guard, credential non-logging, and tmp/ path-confinement checks are all present and correct.
tests/resilience-construct-invariants.test.mts Path 3 assertion correctly flipped from coverage=1.0 to coverage=0 for Fix B.
tests/resilience-release-gate.test.mts Correctly extends allow-list with NA_FOR_SOME_COUNTRIES_DIMENSIONS to handle sovereignFiscalBuffer coverage=0 without manufacturing source-failure warnings.
tests/resilience-ranking.test.mts Mechanical v14→v15 cache key update across all test fixtures; no logic changes.
api/health.js Single-line bump of resilienceRanking key from v14 to v15.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[scoreSocialCohesion] --> B{displacementRaw == null?}
    B -- Yes --> C[Seed outage\ndisplacementRow: score=null, weight=0.25]
    B -- No --> D{displacementMetric == null?}
    D -- Yes --> E[GPI-only mode\ndisplacementRow: score=70, cov=0.6, imputed]
    D -- No --> F[Happy path\ndisplacementRow: norm log10 metric]

    C --> G{unrestRaw == null?}
    E --> G
    F --> G

    G -- Yes --> H[Seed outage\nunrestRow: score=null, weight=0.2]
    G -- No --> I{unrestCount==0 AND fatalities==0?}
    I -- Yes --> J{displacementMetric == null?}
    I -- No --> K[Observed events\nunrestRow: norm metric]

    J -- "Yes (BUG: true during outage too)" --> L[GPI-only impute\nscore=70, cov=0.5]
    J -- No --> M[Stable-absence impute\nscore=85, cov unhcrDisplacement]

    style L fill:#f96,color:#000
    style J fill:#f96,color:#000
Loading

Reviews (1): Last reviewed commit: "fix(resilience): targeted construct fixe..." | Re-trigger Greptile

// (b) Happy path (displacement observed) → impute at the historical
// "stable-absence ≈ 85" value matching unhcrDisplacement.score
// so peaceful + fully-monitored countries (Iceland, Norway) don't regress.
if (displacementMetric == null) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Seed-outage conflated with GPI-only mode in unrest impute

When displacementRaw == null (seed outage), getCountryDisplacement(null, …) returns null, so displacementMetric is also null. This means the displacementMetric == null guard here is true for both scenarios — genuine GPI-only mode (country not in registry) and a transient seed outage — and both receive the lower 70 impute.

That contradicts the stated design: the comment at line 1672 says the 70 impute is "gated to GPI-only mode" only. During an outage Iceland/Norway, which are in the registry, would silently regress from ~87 to ~80 purely because the displacement seed was temporarily unavailable.

The existing seed-outage test (seed outage (displacementRaw absent)) uses unrestRaw: 'present-events' and therefore never exercises the zero-unrest + outage path.

Change to displacementRaw != null && displacementMetric == null to restrict the 70 impute to genuine GPI-only mode:

Suggested change
if (displacementMetric == null) {
if (displacementRaw != null && displacementMetric == null) {

koala73 and others added 2 commits April 26, 2026 12:06
…s fixes)

Self-review (ce-code-review) + user findings surfaced 5 substantive
issues that are addressed below in one fix-up commit. All 594 resilience
tests pass; typecheck clean (root + api).

**P1 — U2 outage gating (correctness)**
`scoreSocialCohesion`: when `displacementRaw == null` (UNHCR seeder
outage) AND `unrestCount === 0`, the unrest row was imputing at 70
(`socialCohesionGpiOnlyUnrest`) — conflating seed-outage with
country-absent-from-registry. Tiny peaceful states would still get
the GPI-only de-rate during transient seeder failures. Fix: gate the
GPI-only impute branch on `displacementRaw != null && displacementMetric == null`,
so outage collapses to the 85-anchor `unhcrDisplacement.score`
(matching the happy path). Regression test pins the discriminator.

**P1 — Path-3 filter discriminator (correctness)**
`isExcludedFromConfidenceMean` was using `coverage === 0` alone to
decide "not-applicable". A real Path-2 SWF country with
`completeness === 0` (full scrape outage) would also produce
`coverage = 0` while keeping `observedWeight = 1` — and would be
silently filtered out of `computeLowConfidence` /
`computeOverallCoverage`, hiding a data outage from the operator.
Fix: discriminate via the unique Path-3 fingerprint
`coverage === 0 && observedWeight === 0 && imputedWeight === 0`.
Same tightening mirrored in the client widget. Regression test in
`construct-invariants.test.mts` exercises both Path 2 (completeness=0)
and Path 3 against the filter directly.

**P1 — Stale doc claim contradicts new behavior**
Block comment above `scoreSovereignFiscalBuffer` still asserted "score
0 with FULL coverage (substantive absence)" — directly contradicting
the new Path 3. Fix: rewrote block comment + grep-swept the repo for
the stale phrasing ("substantive absence", "FULL coverage", "no-SWF
countries score 0") in `_dimension-scorers.ts:160`,
`_indicator-registry.ts:1048`, and the Path 3 block comment itself.
The prior 5-round Codex review missed this because the contradiction
was in prose far from the changed code. Memory entry
`feedback_doc_drift_after_behavior_fix_needs_grep_sweep` already
covers this; this fix-up is the recurrence.

**P1 — Cross-surface drift (UI vs agent)**
The server now filters non-SWF `sovereignFiscalBuffer` from
`computeOverallCoverage`, but the client widget's
`formatResilienceConfidence` only filtered `RESILIENCE_RETIRED_DIMENSION_IDS`,
NOT the new not-applicable set. Result: non-SWF advanced economies
(DE/JP/FR/IT) showed lower Coverage% in the widget than the server's
`overallCoverage` field — same data, different number, agent vs UI
disagreement. Fix:
  - Mirror the server set client-side via
    `RESILIENCE_NOT_APPLICABLE_WHEN_ZERO_COVERAGE_IDS` in
    `resilience-widget-utils.ts`.
  - Apply the same triple-zero discriminator client-side.
  - Extend `RESILIENCE_RETIRED_DIMENSION_IDS` parity test to also
    parity-check the new set (catches drift automatically going forward).
  - Emit `imputationClass: 'not-applicable'` from Path 3 (the proto's
    documented 4-class taxonomy member, previously never used).
    Updated proto comment to document the structurally-not-applicable
    sentinel and its `coverage:0 + observed_weight:0` pairing.
  - Extend `formatDimensionConfidence` so per-dim row rendering
    uses `status: 'not-applicable'` (with the existing tooltip "Not
    applicable: structurally N/A for this country" and "—" symbol)
    instead of the misleading `status: 'absent'` ("Rare, indicates a
    data-collection bug").

**P1 — Dry-run script can exit 0 on broken payloads**
`scripts/dry-run-resilience-rebalance.mjs`: `indexRanking()` silently
accepted unknown shapes as `[]` and `NaN` scores; the cohort gate at
the end could then fall through to `process.exit(0)` on a fully
broken ranking. Since this is the post-merge validation gate for the
v15 rebalance, silent success is the wrong default. Fix: added
`assertRankingHealth(name, byIso2)` requiring >=20 indexed countries
and >=20 with finite scores; added `MIN_TOP20_POP_KNOWN` guard
requiring 15/20 of the top-20 to have known population before the
cohort gate fires. Three new exit codes (7, 8) for these classes.

**P2 — Extracted shared filter helper**
`isExcludedFromConfidenceMean` lives in `_dimension-scorers.ts` next
to the two sets. `computeLowConfidence` and `computeOverallCoverage`
in `_shared.ts` both consume it. Single source of truth — future
construct decisions update one site, not two.

**P3 — `unwrapEnvelope` guard tightened**
Tightened `if (raw && typeof raw === 'object' && 'data' in raw) return raw.data`
(always unwrap when wrapper shape detected) so a `{ data: null, ... }`
wrapper triggers the downstream FATAL guard instead of silently
returning the wrapper object as the payload.

**Verification**
- 594/594 resilience tests pass (`npx tsx --test tests/resilience-*`)
- `npm run typecheck` clean (root + api configs)
- New parity test: `RESILIENCE_NOT_APPLICABLE_WHEN_ZERO_COVERAGE` ↔
  `RESILIENCE_NOT_APPLICABLE_WHEN_ZERO_COVERAGE_IDS`
- New regression: outage-vs-absence in `scoreSocialCohesion`
- New regression: Path-2 completeness=0 NOT excluded by filter

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
…oc update

Routine generated-code refresh after the proto comment update for
imputation_class in the previous commit. No semantic changes — just
a comment in the proto descriptor regenerated into the TS client/server
+ OpenAPI files.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
koala73 and others added 2 commits April 26, 2026 12:38
…emit path

Doc-drift fixup caught by user review: the imputation-class taxonomy
table in country-resilience-index.mdx still claimed `not-applicable`
was "reserved for future dimensions" with "no current call site" —
contradicting both the §U3 changelog entry in the same doc AND the
runtime semantics introduced by this PR.

My earlier grep sweep (commit 575eae8) hit "substantive absence",
"FULL coverage", and "no-SWF countries score 0" but missed this
phrasing because it described the FUTURE state of the taxonomy
member rather than the prior behavior of the SWF dim. Same family
as memory `feedback_doc_drift_after_behavior_fix_needs_grep_sweep` —
the lesson is: when activating a previously-unused taxonomy member,
also grep for "reserved for future" / "not yet" / "no current call"
phrasing on that member.

Updates:
- Taxonomy table row for `not-applicable`: describes the runtime emit
  shape (`score=0, coverage=0, observedWeight=0, imputedWeight=0`),
  the discriminator vs Path 2 with `completeness=0` (must NOT be
  filtered as N/A), and the canonical example (sovereignFiscalBuffer
  for non-SWF countries).
- Closing paragraph: replaces the "reserved for future dimensions"
  claim with a description of the actual emit path, the
  `RESILIENCE_NOT_APPLICABLE_WHEN_ZERO_COVERAGE` opt-in set, and the
  `isExcludedFromConfidenceMean` single-source helper.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
…tlify deploy fix)

Mintlify deploy failed with `Unexpected character 5 before name` after
PR #3425's prior commits. Mintlify's MDX parser treats bare `<` followed
by a digit/letter as the opener of a JSX tag, breaking on prose like
`isolation (<5%)` or `<date>` placeholders.

Sweep found 5 unsafe sites in this single file (user reported 3, the
two `{date}` placeholders were caught in the same sweep):

- Line 34: `coverage < 137 countries` → `coverage under 137 countries`
- Line 129: `isolation (<5%)` and `over-exposure (>60%)`
  → `isolation (under 5%)` and `over-exposure (above 60%)`
  (the > variant is actually MDX-safe but mirroring the prose for readability)
- Line 728: `*-pr1-<date>.json`/`*-acceptance-<date>.json`
  → `*-pr1-{date}.json`/`*-acceptance-{date}.json`
- Line 736: `ratios < 0.5` → `ratios under 0.5`
- Line 738: `*-pr3-<date>.json` → `*-pr3-{date}.json`

`>` followed by digit/letter is fine in MDX prose — only `<` triggers
the JSX-tag-opener path. The `{date}` form is also unambiguous to MDX
(no JSX expression parser in plain prose), preserves the placeholder
intent, and avoids needing HTML entities in a methodology doc.

Verified: `grep -nE "<[0-9A-Za-z]" docs/methodology/country-resilience-index.mdx`
returns no JSX-opener-shaped matches after this commit.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@koala73
koala73 merged commit 1e34925 into main Apr 26, 2026
11 checks passed
@koala73
koala73 deleted the fix/resilience-small-state-bias branch April 26, 2026 08:55
koala73 added a commit that referenced this pull request Apr 26, 2026
…ld (#3426)

* docs(internal): clarify v1.0 review snapshot pre-dates the 6-domain rebuild

`docs/internal/upgrading-country-resilience.md` was an undated review
snapshot using current-tense claims like "the model scores 13 dimensions
across 5 weighted domains" — accurate when written, misleading now that
the Recovery domain has landed (6 domains × 22 dimensions in production).

Adds a brief parenthetical making the v1.0 vintage explicit and pointing
to the live methodology page. The other "5 domains" hits surfaced by
grep are LEFT UNCHANGED:
- `country-resilience-index.mdx:613` is already labeled
  `### v1.0 (April 2026) — Baseline (pre-Recovery domain)` — correct
  version-history, not drift.
- `docs/internal/country-resilience-upgrade-plan.md` (4 sites) is the
  historical narrative that DESCRIBES the journey from 5→6 domains;
  rewriting "5" as "6" would falsify the document's own arc.

Companion to PR #3425's small-state-bias work (plan 2026-04-26-001).
This is Track C of the post-merge audit.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* docs(internal): correct dimension count to match methodology (19, not 22)

Reviewer caught: the doc-drift fix introduced new doc-drift.

The "22 dimensions" I wrote is the SCHEMA count (20 active + 2
retired stubs from `RESILIENCE_DIMENSION_ORDER` per the release-gate
test) — but the user-facing methodology page consistently uses
"19 dimensions" (lines 3, 13, 50, 603, 711 of
docs/methodology/country-resilience-index.mdx). The two counts
diverge because the methodology counts contributing dims while the
release-gate counts the structural registry including retired stubs
(fuelStockDays, reserveAdequacy).

The right number for an internal doc that LINKS to the methodology
page is the methodology count. Changed 22 → 19.

(There's a separate possible drift between methodology "19" and
release-gate "20 active + 2 retired" — the difference may be
financialSystemExposure being flag-gated dark by default, not yet
counted in methodology. That's out of scope for this PR; flagged
for separate audit.)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>
koala73 added a commit that referenced this pull request Apr 26, 2026
…1 + HIC=0) (#3427)

* fix(resilience): repair externalDebtCoverage broken-data-source (mrv=1 trap + HIC=0 sentinel)

PR #3425's post-merge cohort dry-run surfaced that `externalDebtCoverage`
(0.25-weight recovery dimension) is awarding `debtToReservesRatio: 0` →
score 100 to **72/164 countries (44%)** — including NO/CH/DK/SE/FI/IS/
KW/AE/SG/LU. A 25%-weighted dimension that scores nearly half the
universe at 100 is not discriminating; it's structurally elevating
wealthy countries on a metric that should differentiate them.

Two-layered root cause:

1. **WB `mrv=1` coverage trap** (memory `feedback_wb_bulk_mrv1_null_coverage_trap`).
   `mrv=1` returns a SINGLE year across all countries with `value: null`
   for late-reporters; the script silently drops them. The mature
   pattern (already used by `seed-wb-external-debt.mjs` for the
   financialSystemExposure dim) is `mrv=5` + per-country
   pickLatest-non-null. This script was written before the memory was
   captured and never migrated.

2. **WB IDS dataset is LMIC-scoped.** `DT.DOD.DSTC.CD` (short-term
   external debt) returns the literal `0` (not `null`) for high-income
   countries that don't report into the IDS series — they're
   out-of-scope, not data-sparse. Under the prior code, `Number(0) → 0`
   passed `Number.isFinite()`, divided by reserves, and yielded
   `debtToReservesRatio: 0` → score 100. The construct semantically
   does not apply to HICs (which manage external debt through different
   channels), so dropping them is correct — they fall to the dim's
   existing IMPUTE fallback (`recoveryExternalDebt`: 50/0.3/'unmonitored').

**Same fix applied to `seed-recovery-reserve-adequacy.mjs`** which had
the same `mrv=1` trap on `FI.RES.TOTL.MO` (no HIC=0 sentinel issue
there because reserve-months is genuinely measured for HICs).

**Propagation:** No cache-prefix bump. Score cache `resilience:score:v15:`
has 12h TTL; the next bulk-warm tick after this seed lands will refresh
all 222 country scores against the new debt seed, so the 72 false-100
countries naturally drop to IMPUTE 50 within 12h. Manual immediate
warm: run `seed-recovery-external-debt.mjs` then `seed-resilience-scores.mjs`.

**Expected v15 ranking impact:** countries previously inflated by
false-perfect externalDebtCoverage will drop. The cohort dry-run
should be re-run post-deploy to capture the empirical signature; PR
#3425's `npm run dryrun:resilience` script does this.

**Audit lineage:** Plan 2026-04-26-001 cohort dry-run (commit
`ae7229e5e` post-merge) → user audit finding #7+#8 → this PR.
Memory `feedback_wb_bulk_mrv1_null_coverage_trap` is the established
recipe; this is the second seeder migrated to it (first was
`seed-wb-external-debt.mjs` in PR #3412).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* fix(resilience): null-skip before Number() coercion + post-merge force-refresh script

Two reviewer findings on PR #3427's initial commit:

**P1: Number(null) === 0 defeats the latest-non-null picker.**
The mrv=5 + pickLatest fix was structurally correct but `Number(null)`
returns `0` (not `NaN`), passes `Number.isFinite()`, and lets a
`value: null` record overwrite an older non-null record in the
year-comparison branch. Net effect: the very late-reporters mrv=5
was supposed to capture (KW/QA/AE) get a final `value: 0` from a
recent null record, then the country-level `debt.value <= 0` HIC
filter drops them entirely — same end-state as before, fix defeated.

Fix: explicit `if (record?.value == null) continue` BEFORE coercion
in both seeders. Same pattern as the standard recipe documented in
memory `feedback_wb_bulk_mrv1_null_coverage_trap`. Reviewer caught
this within minutes of the initial push.

**P2: Bundle-runner freshness gate would delay propagation by ~24d.**
`scripts/_bundle-runner.mjs:240` skips bundle items where
`elapsed < intervalMs * 0.8`. For the 30d recovery bundle, that means
the new seeder code wouldn't RUN on Railway for ~24 days post-merge
even though the code is in the container. Meanwhile
`seed-resilience-scores` would keep recomputing scores against the
old debt seed → fix appears not to land.

Fix: `scripts/post-pr3427-force-refresh.mjs` — one-shot post-merge
script that invokes both seeders directly (bypassing the bundle
runner's freshness gate). Runs the two seeders in series, then
prints the follow-up bulk-warm command. Documented in the script
header and to be referenced in PR #3427's release checklist.

Both fixes are surgical — same files, no logic re-architecture.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>
koala73 added a commit that referenced this pull request Apr 26, 2026
…§U1, PR 0) (#3433)

Lands the structural CI gate the universe + coverage rebuild needs.
Passes against current v15 ranking with PERMISSIVE thresholds; PRs 3+5
will tighten the thresholds in lockstep with their corresponding fixes.

**Why this exists.** PR #3425's post-merge cohort dry-run showed that
even after targeted scorer fixes (logisticsSupply default, sovereignFiscalBuffer
Path 3, socialCohesion GPI-only impute) AND the externalDebtCoverage
data-source repair (PR #3427), tiny states still climb in v15 (TV rank
12→6, PW 14→7, NR 58→36). The targeted fixes can't reach the universe
+ coverage handling defects. This fixture pins the cohort-level
invariants so the rebuild PRs (universe filter, coverage penalty,
stable-absence recalibration, per-capita normalization) can each
demonstrate measurable cohort movement at merge time.

**What it asserts (PERMISSIVE baselines passing on current v15).**

| Invariant                                    | v15 measurement | PR 0 threshold |
|----------------------------------------------|-----------------|----------------|
| median(G7) − median(microstate-territories)  | **−5.52pt**     | > −10pt        |
| median(Nordics) − median(GCC)                | **+8.20pt**     | ≥ −30pt        |
| min(G7) − max(Sub-Saharan-LIC)               | **+10.40pt**    | ≥ −20pt        |
| microstate-territories in top 20             | **6** (AD, LI, TV, PW, GL, MO) | report-only |

The G7-vs-microstate inversion (−5.52pt) is the structural problem the
rebuild exists to fix. PR 0 baseline tolerates it up to 10pt; PR 3
(coverage penalty) is expected to flip the sign and tighten to +10pt.

**Files.**

- 5 cohort JSON fixtures at server/worldmonitor/resilience/v1/cohorts/
  (g7, nordics, gcc, sub-saharan-lic, microstate-territories) — committed
  data, not test code, so cohort membership can be reviewed independently
- tests/resilience-cohort-anti-inversion.test.mts — fixture self-tests
  (always run) + live-ranking invariants (skip-on-missing-creds)
- tests/resilience-retired-dimensions-parity.test.mts — extended with
  cohort JSON shape validation (anti-mystery-cohort gate: every cohort
  must carry a non-empty description explaining its purpose)

**Read pattern.** Reads live `resilience:ranking:vN` from production
Upstash via the read-only GET pattern from
scripts/dry-run-resilience-rebalance.mjs (PR #3425). Skips gracefully
when UPSTASH_REDIS_REST_URL is unset — required for CI-without-prod-creds
and dev-without-.env environments. The cache-key import is dynamic
(`await import('../server/.../_shared.ts')`) so the test always reads
the current cache version, never a hardcoded literal — same source-of-truth
pattern PR #3425 established for the dry-run script.

**Coverage anomaly.** microstate-territories cohort has 12/13 present
in v15 (FK Falklands missing — a UK territory not in WB IDS scope, so
expected). The other 4 cohorts are 100% covered. Reported in the
diagnostic `it()` block.

**Verification.**
- 617/617 resilience tests pass (`npx tsx --test tests/resilience-*`)
  — 594 prior + 23 new cohort + parity tests
- Live-invariants pass against current v15 with prod creds
- Skip-on-missing-creds path passes without `.env` loaded

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant