docs(resilience): reference-grade upgrade plan + origin review#2938
Merged
Conversation
Why this PR? Captures the 90-day plan to upgrade Country Resilience to a reference-grade composite index, decomposed into three phases: transparency + calibration, structural rebuild with three pillars, explanatory product. Colocates the plan with the internal origin review that prompted it, so every carried-forward decision has a working reference to its source. Locks the construct memo (definition, horizon, audience, polarity, partly non-compensatory aggregation via penalized weighted mean with tunable alpha) before any code begins, so the index cannot be re-argued dimension by dimension later. Splits the product into an Annual Reference Edition + Live Monitor, per INFORM, ND-GAIN, WorldRiskIndex, and FSI precedent. Prerequisite PRs verified merged before landing: - #2821 (baseline / stress engine) - #2847 (formula revert + RSF direction fix) - #2858 (seed direct scoring) Docs only. docs/internal/ is gitignored; both files are force-added, matching the existing youtube-desktop.md precedent in the same directory. No runtime impact. Generated with Claude Opus 4.6 (1M context) via Claude Code + Compound Engineering v2.49.0 Co-Authored-By: Claude Opus 4.6 <[email protected]>
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Contributor
Greptile SummaryAdds two
Confidence Score: 4/5Safe to merge after resolving two internal inconsistencies that could mislead implementers — no code changes, no runtime impact. Two P1 findings in the upgrade plan document could cause implementing engineers to build the wrong behavior: the interaction graph's 'computed lazily' language directly contradicts the fully-offline design, and the single Spearman ≥0.70 acceptance criterion contradicts the per-pillar benchmark gating approach described elsewhere. The coverage threshold discrepancy (≥180 vs ≥170) is P2. These are docs-only changes with no runtime impact, but fixing these before implementation begins avoids rework on concrete tasks T3.1 and T2.4. docs/internal/country-resilience-upgrade-plan.md — lines 798 (interaction graph), 876–878 (acceptance criteria coverage threshold and Spearman gate) Important Files Changed
Sequence DiagramsequenceDiagram
participant C as Client
participant H as GetResilienceScore Handler
participant B as buildResilienceScore()
participant D as Dimension Scorers (×13)
participant R as Redis
participant Batch as Interval Batch Job (Railway cron / 6h)
C->>H: GetResilienceScore(countryCode)
H->>B: buildResilienceScore(cc)
B->>D: score all dimensions (parallel)
D->>R: read memoized global keys (UCDP, WGI, RSF…)
R-->>D: signal values + imputationClass
D-->>B: per-dimension scores + freshness
B->>B: aggregate domains → pillars → overall (penalized weighted mean)
B->>R: read resilience:intervals:v2:<cc>
alt intervals present and fresh (<48h)
R-->>B: pillarIntervals, overallInterval, rankBand
else missing or stale
R-->>B: null / staleIntervals:true
Note over B: intervals OMITTED from response (never computed inline — T3.1)
end
B->>R: write history sorted set (if first entry today)
B-->>H: ResilienceScore v2.0 shape
H-->>C: response (with or without intervals)
Note over Batch: Runs every 6h (offline only)
Batch->>R: read all scored countries
Batch->>Batch: Bootstrap N=500 + Monte Carlo (weight ±20%, α variants)
Batch->>R: write resilience:intervals:v2:<cc> for each country
Batch->>R: write resilience:benchmark:external:v1 (weekly)
Reviews (1): Last reviewed commit: "docs(resilience): reference-grade upgrad..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| - `src/components/ResilienceWidget.ts` (primary) | ||
| - `src/components/CountryDeepDivePanel`, resilience card integration | ||
| - `src/components/CountryBrief`, brief sections use resilience score | ||
| - Resilience choropleth map layer (DeckGL + Globe) |
Contributor
There was a problem hiding this comment.
The interaction graph says intervals are "computed lazily (T3.1)" on a cache miss, but T3.1 explicitly prohibits exactly this: "Zero lazy computation on the read path — missing key means intervals are omitted from the response; the read path never blocks on bootstrap or MC work." The (T3.1) reference here will mislead the implementing engineer into thinking T3.1 introduces lazy computation, when it is the task that removes it.
Suggested change
| - Resilience choropleth map layer (DeckGL + Globe) | |
| History sorted set updated if current day not yet written. Interval cache read if present, otherwise **omitted** (T3.1 — never computed inline; intervals arrive on the next batch tick). |
Comment on lines
+877
to
+878
| - [ ] Waterfall chart + peer comparison + change attribution in widget. | ||
| - [ ] Pillar toggle (structural / live-shock / recovery / overall) in widget. |
Contributor
There was a problem hiding this comment.
The benchmark design section explicitly states: "The benchmark does not publish a single pass/fail number. Each of the five hypothesis rows above is its own gate." But this acceptance criterion collapses everything back to a single ≥0.70 threshold against INFORM and ND-GAIN.
This also creates a logical trap: the expected Spearman band for Live Shock Exposure vs INFORM is 0.55–0.75, so a result of 0.80 would be a flag under the per-pillar design (too similar, measuring the same thing) yet would pass the ≥0.70 criterion here. The single-threshold criterion effectively invalidates the "above-band is a flag, not a pass" principle.
Replace with a criterion that mirrors the per-pillar gating table, e.g.: "Cross-index benchmark hypotheses within expected Spearman bands for all 5 rows, or annotated with signed outlier commentary." (This is already captured in Phase 2 acceptance — the global AC should match.)
| - [ ] Outcome backtest AUC ≥0.75 on held-out 2024–2025 shock events. | ||
| - [ ] Decomposed uncertainty in widget: per-dimension coverage, imputation | ||
| class, freshness badge, bootstrapped pillar intervals. | ||
| - [ ] Rank stability bands shown in widget and ranking API. |
Contributor
There was a problem hiding this comment.
Phase 2 acceptance sets the bar at ≥180 countries: "Recovery capacity pillar has real Core-tier data coverage for ≥180 countries." The global acceptance criteria here says ≥170 countries. The two thresholds should agree; the Phase 2 value (≥180) is the more conservative and consistent with the Core signal admission rule ("≥180 countries covered") so it should be the canonical bar.
…y reads Fixes three internal inconsistencies caught in review after the plan landed on its PR branch: 1. Interaction graph contradicted the offline-only interval section. Read path was described as "computed lazily" (line 792) while the decomposed-uncertainty section (line 709) commits to zero lazy computation. Rewrote the interaction graph so the read path is cache-only: present payload attached, stale payload flagged, missing payload omitted. All bootstrap and Monte Carlo work lives in the Railway cron batch and rewrites the key on the next tick. 2. Global functional acceptance collapsed the benchmark into a single Spearman gate against INFORM and ND-GAIN, contradicting the benchmark design section which says there is no single pass/fail number and each of the five comparator rows is its own gate (including the two negative-correlation rows against FSI and INFORM). Rewrote the acceptance to mirror all five hypothesis rows and the seven per-event-family backtest gates. 3. Recovery capacity coverage threshold drifted: Phase 2 acceptance said Core-tier coverage for >=180 countries, global acceptance said >=170 scored countries. Unified on >=180 Core-tier covered countries with an explicit link to the signal tiering registry definition. Same-class fixes while in the area: - External expert review (T3.8b) is no longer listed as a blocking shipping criterion; it runs in parallel and gates only the public "reference-grade" promotion. The internal methodology gate (T3.8a) is the actual ship gate. - T1.1 ceiling-bug acceptance no longer pre-asserts a fix; it now accepts either a landed fix if the bug reproduces or an origin-doc changelog update if the symptom is misattributed. - Scorecard re-rating clarified as internal self-assessment, pre-external-review. Docs-only, no code changes. Follow-up to PR #2938 on the same branch. Generated with Claude Opus 4.6 (1M context) via Claude Code + Compound Engineering v2.49.0 Co-Authored-By: Claude Opus 4.6 <[email protected]>
koala73
added a commit
that referenced
this pull request
Apr 11, 2026
Why this PR? Ships the Phase 1 T1.1 deliverable from the country-resilience reference-grade upgrade plan: reproduce the origin-doc "Norway and US both hit 100" ceiling bug with a failing test before any fix lands. The plan explicitly committed to reproducing-before-fixing and to updating the origin doc's changelog if the symptom turned out to be misattributed. This PR documents the measured outcome and pins the current correct behavior. Investigation outcome: the claim does NOT reproduce. Measured scores under the current release-gate fixtures and the post-PR-#2847 domain-weighted-average formula: NO (elite tier) overallScore = 86.58, baseline 86.85, stress 84.36 US (strong tier) overallScore = 72.80, baseline 73.15, stress 70.58 Delta NO - US = 13.78 points Nothing pins at 100. The ordering elite > strong > stressed > fragile is preserved, and the domain-weighted average cannot reach 100 unless every dimension saturates (which does not happen for any fixture tier). The origin claim is misattributed or stale, probably predating PR #2847 which reverted the multiplicative baseline*(1-stressFactor) formula that had over-penalized all countries. What this PR commits: - A new regression test in tests/resilience-release-gate.test.mts asserting that NO and US are not pinned at 100, that NO > US, and that the delta is at least 3 points. The threshold leaves room for fixture tuning without over-fitting to the measured 13.78 delta. - A detailed comment block inside the test capturing the measured numbers, the conclusion, and the rationale so future regressions to a real ceiling bug are caught immediately and so the origin-doc changelog update (tracked separately) has a cite-able evidence base. The origin-doc changelog update is deliberately deferred to a trailing commit after PR #2938 (the reference-grade plan, which contains the origin doc) merges, to avoid a cross-branch conflict. Side finding, out of scope for this PR, tracked as a follow-up: the release-gate fixtures use qualityFor(profile) to derive all inputs from a single quality value per tier (elite/strong/stressed/ fragile), so every country within a tier produces identical scores. This means the release-gate suite cannot detect within-tier ordering issues. Not a scorer bug, a fixture-design limitation. Prerequisite PRs verified merged: - #2821 (baseline / stress engine) - #2847 (formula revert + RSF direction fix) - #2858 (seed direct scoring) Test suite: 172/172 resilience tests pass locally. Typecheck clean. Generated with Claude Opus 4.6 (1M context) via Claude Code + Compound Engineering v2.49.0 Co-Authored-By: Claude Opus 4.6 <[email protected]>
This was referenced Apr 11, 2026
The docs site parses docs/internal/*.md through the MDX pipeline
(the literal .md extension does not downgrade it to CommonMark).
MDX treats `<identstart` as the beginning of a JSX element and
`{expression}` as a JavaScript expression, so several patterns that
were benign in pure Markdown became parse errors:
Failed to parse page content at path
internal/country-resilience-upgrade-plan.md: Unexpected character
`8` (U+0038) before name, expected a character that can start a
name, such as a letter, `$`, or `_`
Root cause was `<80 countries` on the ICU surge capacity row of the
signal tiering table (`<80` was read as a JSX tag opener, and `8`
is not a valid name-start character). Once that fired, the next
hazards in the file would have fired on each subsequent retry, so
this PR fixes them all in a single pass.
Fixes:
- Rewrote `<80 countries` to `fewer than 80 countries` in the
signal tiering table (the immediate cause of the parse error).
- Rewrote three `< 0.03 AUC` prose strings to `under 0.03 AUC` in
the Phase 2 acceptance, the global functional acceptance, and the
success metrics. Space after `<` protects these in lenient MDX
parsers but not in strict ones; rewriting is safer than escaping.
- Wrapped three bare `{0, 0.25, 0.5, 0.75, 1}` prose expressions
(the alpha sensitivity axis set) in backticks so MDX treats them
as inline code instead of JavaScript expressions. The pre-existing
fenced YAML schema block is untouched because fenced code blocks
already shield curly braces from parsing.
No content semantics change; this is a pure escaping fix. The
origin review document in the same PR was verified clean of the
same class of hazards and is untouched.
Docs-only, no code, no runtime impact. Third commit on the PR.
Generated with Claude Opus 4.6 (1M context) via Claude Code
+ Compound Engineering v2.49.0
Co-Authored-By: Claude Opus 4.6 <[email protected]>
koala73
added a commit
that referenced
this pull request
Apr 11, 2026
Addresses two review comments on PR #2945: 1. **P2 Medium: broken reference in the v1.0 scope note at line 8.** The intro linked to `../internal/country-resilience-upgrade-plan.md`, which resolves to `docs/internal/country-resilience-upgrade-plan.md`. That file lives on PR #2938 and is not in this PR's tree or on origin/main, so until #2938 merges the link would render as a 404 on the docs site. Dropped the markdown link and kept the text reference to "a separate reference-grade upgrade plan" so readers still know context exists elsewhere. The existing prose reference to the same file at line 365 is already inside backticks and is therefore a plain-text citation, not a link, so it stays. 2. **Additional suggestion: roads series shared between two dimensions.** `roadsPavedLogistics` (Logistics & Supply, weight 0.50) and `roadsPavedInfra` (Infrastructure, weight 0.35) both read from World Bank `IS.ROD.PAVE.ZS`. Added an explicit note under the Infrastructure dimension table clarifying that this is deliberate source reuse (Logistics uses it as a transit-viability proxy, Infrastructure uses it as a baseline-public-capital proxy) and pointing forward to the v2.0 plan's consolidation of shared upstream signals into a single indicator registry. No change to the indicator tables themselves; this is a clarifying paragraph. No content semantics change beyond the clarifying paragraph and the dead-link removal. Docs-only, no code. Generated with Claude Opus 4.6 (1M context) via Claude Code + Compound Engineering v2.49.0 Co-Authored-By: Claude Opus 4.6 <[email protected]>
Appends an editor's note to the origin review preserving the original text as written and recording the Phase 1 T1.1 investigation outcome. The review states that "Norway and the US both hit 100 under current fixtures, which broke the intended ordering and exposed a ceiling effect at the top end of the ranking." The T1.1 regression test shipped on PR #2941 investigated this claim and did NOT reproduce it. Measured scores under the current release-gate fixtures and the post-PR-#2847 domain-weighted-average formula: Norway (elite tier): overallScore = 86.58, baseline 86.85, stress 84.36 US (strong tier): overallScore = 72.80, baseline 73.15, stress 70.58 Delta: NO minus US = 13.78 points Neither country approaches 100, the ordering is preserved, and the scorer cannot produce a hard 100 ceiling under any fixture tier. The specific Norway=US=100 illustration is retracted; the scorecard judgment and the six prescribed improvements remain valid. Also records the side finding from the investigation that the release-gate fixtures use one quality value per tier (elite, strong, stressed, fragile), so every country within a tier produces byte- identical scores. This is a fixture-design limitation, not a scorer bug, and is tracked as Phase 2 follow-up work. The original review text is preserved unchanged; the changelog is appended as an "Editor's Note" section at the end of the file so the historical record of what was originally filed stays auditable. The T1.1 regression test itself stays in the release-gate suite so a real ceiling bug, if ever introduced, is caught immediately by CI. Closes the one deliberate deferral noted in PR #2941's description (the "origin-doc changelog update" trailing commit that was waiting to land alongside the plan once the cross-branch conflict concern was resolved). Lands as a fourth commit on this PR so both the plan and the corrected origin doc ship together. Docs only, no code, no runtime impact. Generated with Claude Opus 4.6 (1M context) via Claude Code + Compound Engineering v2.49.0 Co-Authored-By: Claude Opus 4.6 <[email protected]>
koala73
added a commit
that referenced
this pull request
Apr 11, 2026
…2945) * docs(resilience): promote methodology to .mdx at CII parity (T1.3) Why this PR? Ships Phase 1 T1.3 of the country-resilience reference-grade upgrade plan: promote the v1.0 methodology document from a scratch markdown file to a published MDX page at parity with the Country Instability Index (docs/country-instability-index.mdx). Before this PR there was no publicly documented resilience methodology page, only an internal markdown draft. The reference-grade upgrade plan's origin review explicitly called out methodological opacity as the biggest gap keeping the product below OECD/JRC standards, and committed to documenting every dimension, formula, goalpost, cadence, weight rationale, and imputation rule before any Phase 2 schema work begins. This PR is that documentation. What this PR commits: - git mv docs/methodology/resilience-index.md to docs/methodology/country-resilience-index.mdx so git history tracks the rename; edits land as modifications on the new path. - Adds MDX frontmatter (title + description) matching CII shape so the page renders correctly in the docs site and shows up in search. - Prepends a short v1.0 scope note that distinguishes the currently shipping behavior (this doc) from the planned v2.0 three-pillar rebuild (tracked in the reference-grade upgrade plan). Readers who land on the page know exactly what is live and what is coming. - Fixes an orphan table row (`| Food & Water | Mixed |`) that had drifted into the Overall Score section from an earlier edit and was breaking rendering around the score-formula block. - Rewrites the Imputation Taxonomy section to use the formal four-class naming from T1.7 (stable-absence, unmonitored, source-failure, not-applicable) and adds a mapping table from each concrete IMPUTATION / IMPUTE entry to its class. This is the public-facing surface of the T1.7 foundation PR. - Adds a Reproducibility Appendix listing every Redis key used by the scorer (score cache, ranking cache, history sorted set, intervals, seed-meta, static record, static index), the meaning of the `dataVersion` field, and a step-by-step procedure for reproducing any published country score by hand from a Redis snapshot. - Adds a Changelog section with a v1.0 entry that references the prerequisite PRs (#2821, #2847, #2858) and the Phase 1 work landing so far (T1.1 regression test, T1.4 dataVersion widget wire, T1.7 imputation taxonomy foundation), plus a v2.0 placeholder that summarizes the reference-grade upgrade plan for readers. - Adds editorial notes pointing at the Phase 1 T1.8 methodology doc linter that will enforce parity between this document and the indicator registry in _dimension-scorers.ts. What is deliberately NOT in this PR: - No v2.0 content presented as shipping. The three-pillar rebuild, recovery capacity pillar, penalized weighted mean aggregation, cross-index benchmark, and annual Reference Edition are all tracked in the reference-grade upgrade plan but not yet implemented. The changelog v2.0 section names them as planned work, not current behavior. - No methodology doc linter. That is T1.8 and ships separately so it can be reviewed as a tooling change. - No deletion of the old .md path. git mv preserves history; the old location renders a 404 which is expected. Prerequisite PRs verified merged: - #2821 (baseline / stress engine) - #2847 (formula revert + RSF direction fix) - #2858 (seed direct scoring) Testing: - Markdown-only change, no code. - npm run typecheck: clean - Pre-push hook (typecheck + build:full + version:check) passes. Generated with Claude Opus 4.6 (1M context) via Claude Code + Compound Engineering v2.49.0 Co-Authored-By: Claude Opus 4.6 <[email protected]> * docs(resilience): add v1.0 content to methodology mdx (T1.3 followup) Companion commit to 4e7f64a (which was a rename-only because git add after git mv re-staged the original file content). This commit adds the actual v1.0 content the T1.3 task requires: - MDX frontmatter (title + description) matching the Country Instability Index shape so the page renders in the docs site. - Short v1.0 scope note that distinguishes the currently shipping behavior (this doc) from the planned v2.0 three-pillar rebuild (tracked in the reference-grade upgrade plan). - Removes an orphan table row (`| Food & Water | Mixed |`) from the Overall Score section that had drifted from an earlier edit. - Rewrites the Imputation Taxonomy section to use the formal four-class naming from T1.7 (stable-absence, unmonitored, source-failure, not-applicable) with a mapping table from each concrete IMPUTATION / IMPUTE entry to its class. - Reproducibility Appendix listing every Redis key used by the scorer, the dataVersion semantics, and a step-by-step reproduction procedure. - Changelog section with v1.0 (prerequisite PRs + T1.1 / T1.4 / T1.7 landing) and v2.0 placeholder summarizing the reference-grade upgrade plan. - Editorial notes pointing at the T1.8 methodology doc linter that will enforce parity between this document and the indicator registry. Net change: 89 insertions, 13 deletions, file grows from 300 to 376 lines. Docs-only, no code, no runtime impact. Generated with Claude Opus 4.6 (1M context) via Claude Code + Compound Engineering v2.49.0 Co-Authored-By: Claude Opus 4.6 <[email protected]> * docs(resilience): address PR #2945 review (dead link + roads reuse note) Addresses two review comments on PR #2945: 1. **P2 Medium: broken reference in the v1.0 scope note at line 8.** The intro linked to `../internal/country-resilience-upgrade-plan.md`, which resolves to `docs/internal/country-resilience-upgrade-plan.md`. That file lives on PR #2938 and is not in this PR's tree or on origin/main, so until #2938 merges the link would render as a 404 on the docs site. Dropped the markdown link and kept the text reference to "a separate reference-grade upgrade plan" so readers still know context exists elsewhere. The existing prose reference to the same file at line 365 is already inside backticks and is therefore a plain-text citation, not a link, so it stays. 2. **Additional suggestion: roads series shared between two dimensions.** `roadsPavedLogistics` (Logistics & Supply, weight 0.50) and `roadsPavedInfra` (Infrastructure, weight 0.35) both read from World Bank `IS.ROD.PAVE.ZS`. Added an explicit note under the Infrastructure dimension table clarifying that this is deliberate source reuse (Logistics uses it as a transit-viability proxy, Infrastructure uses it as a baseline-public-capital proxy) and pointing forward to the v2.0 plan's consolidation of shared upstream signals into a single indicator registry. No change to the indicator tables themselves; this is a clarifying paragraph. No content semantics change beyond the clarifying paragraph and the dead-link removal. Docs-only, no code. Generated with Claude Opus 4.6 (1M context) via Claude Code + Compound Engineering v2.49.0 Co-Authored-By: Claude Opus 4.6 <[email protected]> --------- Co-authored-by: Claude Opus 4.6 <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why this PR?
Captures the 90-day plan to upgrade Country Resilience to a reference-grade composite index, decomposed into three phases (transparency + calibration, structural rebuild with three pillars, explanatory product). Colocates the plan with the internal origin review that prompted it, so every carried-forward decision has a working reference to its source.
Locks the construct memo (definition, horizon, audience, polarity, partly non-compensatory aggregation via a penalized weighted mean with tunable alpha) before any code begins, so the index cannot be re-argued dimension by dimension later. Splits the product into an Annual Reference Edition + Live Monitor, per INFORM, ND-GAIN, WorldRiskIndex, and FSI precedent.
Summary
docs/internal/upgrading-country-resilience.md, the origin review: 7.4/10 scorecard, six prescribed improvements, 90-day plan.docs/internal/country-resilience-upgrade-plan.md, the 1143-line reference-grade upgrade plan: construct memo, three phases, per-unit tasks, per-pillar benchmark hypotheses, per-event-family release gates, offline-only interval computation, licensing workstream, controllable success metrics.What is in this PR
No code changes. Markdown documentation only.
docs/internal/is gitignored; both files are force-added, matching the existingyoutube-desktop.mdprecedent in the same directory.Scope boundaries
This PR ships only the decision artifact. Implementation begins in follow-up PRs, starting with T1.1 (reproduce the origin-doc "ceiling bug" with a failing regression test before any fix lands).
Prerequisite PRs (verified merged)
Testing
No automated tests added or required for a docs-only change. Pre-push hook (typecheck + build:full + version:check) passes unchanged.
Post-Deploy Monitoring & Validation
No additional operational monitoring required: markdown documentation only, no runtime impact.
Generated with Claude Opus 4.6 (1M context) via Claude Code