docs(resilience): PR 5.3 — foodWater scorer audit (construct-deterministic GCC identity)#3374
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Greptile SummaryPR 5.3 ships a read-only docs extension and 8 regression-guard tests that pin the Two consistency issues worth a look before merge: the GCC anchor test uses Confidence Score: 5/5Safe to merge — pure docs and tests with zero scorer behaviour changes; remaining findings are P2 documentation/test-input consistency issues. All findings are P2: the anchor test input shape inconsistency and the comment arithmetic error don't cause assertion failures today, and no production code is touched by this PR. tests/resilience-foodwater-field-mapping.test.mts (anchor test input shape) Important Files Changed
Reviews (1): Last reviewed commit: "docs(resilience): PR 5.3 — foodWater sco..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+88
to
+120
| fao: { peopleInCrisis: 0, phase: null }, | ||
| }); | ||
| const [scarceScore, abundantScore] = await Promise.all([ | ||
| scoreFoodWater(TEST_ISO2, scarce), | ||
| scoreFoodWater(TEST_ISO2, abundant), | ||
| ]); | ||
| assert.ok(abundantScore.score > scarceScore.score, | ||
| `more renewable water = higher score; got scarce=${scarceScore.score}, abundant=${abundantScore.score}`); | ||
| }); | ||
| }); | ||
|
|
||
| describe('scoreFoodWater — IPC absence path (stable-absence imputation)', () => { | ||
| it('country not in IPC/HDX (fao=null) imputes ipcFood=88 when static record present', async () => { | ||
| const reader = makeStaticReader({ | ||
| aquastat: { value: 20, indicator: 'water stress' }, | ||
| fao: null, // crisis_monitoring_absent — food-secure country, not a monitored crisis | ||
| }); | ||
| const result = await scoreFoodWater(TEST_ISO2, reader); | ||
| // Blend: {score:88, weight:0.6, cov:0.7, imputed} + {score:80, weight:0.4, cov:1.0, observed} | ||
| // weightedScore = (88*0.6 + 80*0.4) / (0.6+0.4) = 85.6 → 86 | ||
| // Pinning the blended range to catch a formula regression. | ||
| assert.ok(result.score >= 80 && result.score <= 90, | ||
| `IPC-absent + moderate aquastat must blend to 80-90; got ${result.score}`); | ||
| // Per weightedBlend's T1.7 rule (line 601 of _dimension-scorers.ts): | ||
| // `imputationClass` surfaces ONLY when observedWeight === 0. Here | ||
| // AQUASTAT is observed data (score=80), so it "wins" and the final | ||
| // imputationClass is null. The IPC-impute is still reflected in | ||
| // `imputedWeight` and the lower coverage (70% for IPC * 0.6 + 100% | ||
| // for AQUASTAT * 0.4 = 82% weighted). | ||
| assert.equal(result.imputationClass, null, | ||
| 'mixed observed+imputed → imputationClass=null (observed wins); IPC impute reflected in imputedWeight'); | ||
| assert.ok(result.imputedWeight > 0, 'IPC slot must be counted as imputed'); | ||
| assert.ok(result.observedWeight > 0, 'AQUASTAT slot must be counted as observed'); |
Contributor
There was a problem hiding this comment.
The test supplies fao: { peopleInCrisis: 0, phase: null }, which takes the fao != null else-branch (weights 0.45 / 0.15 / 0.40, denominator 0.85). However, the methodology doc this test accompanies explicitly states GCC countries have fao: null seeded, which takes the fao == null imputation branch (weights 0.60 / 0.40, denominator 1.0, impute score 88 not 100).
Coincidentally both paths round to 53:
- Else-branch:
(100×0.45 + 0×0.40) / 0.85 = 52.94 → 53 - Impute-branch:
(88×0.60 + 0×0.40) / 1.00 = 52.80 → 53
Because the assertion only checks Math.round(result.score) === 53, it passes regardless. But the anchor test is intended to pin the production GCC path — and a future change to IMPUTE.ipcFood.score (e.g. from 88 to 85) would silently leave this test green while the actual GCC score shifts. Using fao: null would make the test sensitive to that class of change.
Comment on lines
+136
to
+139
| // Single imputed slot {score:88, weight:0.6, cov:0.7}: | ||
| // weightedScore = 88 * 0.6 / 0.6 = 88 (only this slot has a score) | ||
| // weightedCertainty = 0.7 * 0.6 / (0.6+0.4) = 0.42 total weighted / total | ||
| assert.ok(result.score >= 85 && result.score <= 92, |
Contributor
There was a problem hiding this comment.
The comment states (88×0.6 + 80×0.4) / (0.6+0.4) = 85.6 → 86, but 88×0.6 = 52.8 and 80×0.4 = 32.0, giving 84.8 / 1.0 = 84.8 → 85. The assertion >= 80 && <= 90 correctly accepts 85, so this is a comment-only error, but it will mislead anyone reproducing the math by hand.
Comment on lines
+209
to
+213
| - GCC extreme-withdrawal anchor: AQUASTAT value=2000 + | ||
| peopleInCrisis=0 blends to exactly 53. | ||
| - IPC-absent with static record present: imputes | ||
| `ipcFood=88`; observed AQUASTAT wins → | ||
| `imputationClass=null` per weightedBlend's T1.7 rule. |
Contributor
There was a problem hiding this comment.
Findings #1–2 establish that GCC countries have fao: null seeded, which routes to the fao == null imputation branch (weights 0.60 IPC + 0.40 AQUASTAT, denominator 1.0, impute score = 88). Finding #4 then presents the formula (100×0.45 + 0×0.40) / (0.45 + 0.40) = 45/0.85 ≈ 53, which is the else-branch formula.
The correct formula for the fao: null path actually described in the narrative is:
(88 × 0.60 + 0 × 0.40) / (0.60 + 0.40) = 52.8 / 1.00 ≈ 53
Both round to 53, so the headline conclusion is correct, but a reader tracing the formula through the scorer will find the stated weights (0.45, 0.40) don't match the code path the narrative claims to be describing.
This was referenced Apr 24, 2026
Merged
Merged
koala73
added a commit
that referenced
this pull request
Apr 24, 2026
…x comment math Addresses 3 P2 Greptile findings on #3374 — all variations of the same root cause: the test fixture + doc described two different code paths that coincidentally both produce ~53 for GCC inputs. Changes 1. GCC anchor test now drives the IMPUTE branch (`fao: null`), matching what the static seeder emits for GCC in production. The else branch (`fao: { peopleInCrisis: 0 }`) happens to converge on ~52.94 by coincidence but is NOT the live code path for GCC. 2. Doc finding #4 updated to show the IMPUTE-branch math `(88×0.6 + 0×0.4) / 1.0 = 52.8 → 53` and explicitly notes the else-branch convergence as a coincidence — not the construct's intent. 3. Comment math off-by-one fix at line 107: (88×0.6 + 80×0.4) / (0.6+0.4) = 52.8 + 32.0 = 84.8 → 85 (was incorrectly stated as 85.6 → 86) Test assertion `>= 80 && <= 90` still accepts 85 so behaviour is unchanged; this was a comment-only error that would have misled anyone reproducing the math by hand. Verified - `npx tsx --test tests/resilience-foodwater-field-mapping.test.mts` — 8 pass / 0 fail (IMPUTE-branch anchor test produces 53 as expected) - `npm run lint:md` — clean Also rebased onto updated #3373 (which landed a backtick-escape fix).
koala73
force-pushed
the
fix/resilience-pr53-foodwater-audit
branch
from
April 24, 2026 13:58
7de0630 to
83d47f9
Compare
koala73
added a commit
that referenced
this pull request
Apr 24, 2026
…y domicile question) PR 5.1 of cohort-audit plan 2026-04-24-002. Stacked on PR 5.3 (#3374) so the known-limitations.md section append is additive. Read-only static audit of scoreTradeSanctions + the sanctions:country-counts:v1 seed — framed around the Codex-reformulated construct question: should designated-party domicile count penalize resilience? Findings 1. The count is "OFAC-designated-party domicile locations," NOT "sanctions against this country." Seeder (`scripts/seed-sanctions- pressure.mjs:85-93`) parses OFAC Advanced XML SDN + Consolidated, extracts each designated party's Locations, and increments `map[countryCode]` by 1 for every location country on that party. 2. The count conflates three semantically distinct categories a resilience construct might treat differently: (a) Country-level sanction target (NK SDN listings) — correct penalty (b) Domiciled sanctioned entity (RU bank in Moscow, post-2022) — debatable, country hosts the actor (c) Transit / shell entity (UAE trading co listed under SDGT for Iran evasion; CY SPV for a Russian oligarch) — country is NOT the target, but takes the penalty 3. Observed GCC cohort impact: AE scores 54 vs KW/QA 82. The −28 gap is almost entirely driven by category (c) listings — AE is a financial hub where sanctioned parties incorporate shells. 4. Three options documented for the construct decision (NOT decided in this PR): - Option 1: Keep flat count (status quo, defensible via secondary- sanctions / FATF argument) - Option 2: Program-weighted count — weight DPRK/IRAN/SYRIA/etc. at 1.0, SDGT/SDNTK/CYBER/etc. at 0.3-0.5. Recommended; seeder already captures `programs` per entry — data is there, scorer just doesn't read it. - Option 3: Transit-hub exclusion list (AE, SG, HK, CY, VG, KY) — brittle + normative, not recommended 5. Recommendation documented: Option 2. Implementation deferred to a separate methodology-decision PR (outside auto-mode authority). Shipped - `docs/methodology/known-limitations.md` — new section extending the file: "tradeSanctions — designated-party domicile construct question." Covers what the count represents, the three categories with examples, observed GCC impact, three options w/ trade-offs, recommendation, follow-up audit list (entity-sample gated on API-key access), and file references. - `tests/resilience-sanctions-field-mapping.test.mts` (new) — 10 regression-guard tests pinning CURRENT behavior: 1-6. normalizeSanctionCount piecewise anchors: count=0→100, 1→90, 10→75, 50→50, 200→25, 500→≤1 7. Monotonicity: strictly decreasing across the ramp 8. Country absent from map defaults to count=0 → score 100 (intentional "no designated parties here" semantics) 9. Seed outage (raw=null) → null score slot, NOT imputed (protects against silent data-outage scoring) 10. Construct anchor: count=1 is exactly 10 points below count=0 (pins the "first listing drops 10" design choice) Verified - `npx tsx --test tests/resilience-sanctions-field-mapping.test.mts` — 10 pass / 0 fail - `npm run test:data` — 6721 pass / 0 fail - `npm run typecheck` / `typecheck:api` — green - `npm run lint` / `lint:md` — clean
…istic GCC identity) PR 5.3 of cohort-audit plan 2026-04-24-002. Stacked on PR 5.2 (#3373) so the known-limitations.md section append is additive. Read-only static audit of scoreFoodWater. Findings 1. The observed GCC-all-score-53 is CONSTRUCT-DETERMINISTIC, not a regional-default leak. Pinned mathematically: - IPC/HDX doesn't publish active food-crisis data for food-secure states → scorer's fao-null branch imputes IMPUTE.ipcFood=88 (class='stable-absence', cov=0.7) at combined weight 0.6 - WB indicator ER.H2O.FWST.ZS (labelled 'water stress') for GCC is EXTREME (KW ~3200%, BH ~3400%, UAE ~2080%, QA ~770%) — all clamp to sub-score 0 under the scorer's lower-better 0..100 normaliser at weight 0.4 - Blended with peopleInCrisis=0 (fao block present with zero): (100 * 0.45 + 0 * 0.4) / (0.45 + 0.4) = 45 / 0.85 ≈ 53 Every GCC country has the same inputs → same outputs. That's construct math, not a regional lookup. 2. Indicator-keyword routing is code-correct. `'water stress'`, `'withdrawal'`, `'dependency'` route to lower-better; `'availability'`, `'renewable'`, `'access'` route to higher-better; unrecognized indicators fall through to a value-range heuristic with a WARN log. 3. No bug or methodology decision required. The 53-all-GCC output is a correct summary statement: "non-crisis food security + severe water-withdrawal stress." A future construct decision might split foodWater into separate food and water dims so one saturated sub-signal doesn't dominate the combined dim for desert economies — but that's a construct redesign, not a bug. Shipped - `docs/methodology/known-limitations.md` — extended with a new section documenting the foodWater audit findings, the exact blend math that yields ~53 for GCC, cohort-determinism vs regional-default, and a follow-up data-side spot-check list gated on API-key access. - `tests/resilience-foodwater-field-mapping.test.mts` — 8 new regression-guard tests: 1. indicator='water stress' routes to lower-better 2. GCC extreme-withdrawal anchor (value=2000 → blended score 53) 3. indicator='renewable water availability' routes to higher-better 4. fao=null with static record → imputes 88; imputationClass=null because observed AQUASTAT wins (weightedBlend T1.7 rule) 5. fully-imputed (fao=null + aquastat=null) surfaces imputationClass='stable-absence' 6. static-record absent entirely → coverage=0, NOT impute 7. Cohort determinism — identical inputs → identical scores 8. Different water-profile inputs → different scores (rules out regional-default hypothesis) Verified - `npx tsx --test tests/resilience-foodwater-field-mapping.test.mts` — 8 pass / 0 fail - `npm run test:data` — 6711 pass / 0 fail (PR 5.2's 9 + PR 5.3's 8 = 17 new stacked) - `npm run typecheck` / `typecheck:api` — green - `npm run lint` / `lint:md` — clean
…x comment math Addresses 3 P2 Greptile findings on #3374 — all variations of the same root cause: the test fixture + doc described two different code paths that coincidentally both produce ~53 for GCC inputs. Changes 1. GCC anchor test now drives the IMPUTE branch (`fao: null`), matching what the static seeder emits for GCC in production. The else branch (`fao: { peopleInCrisis: 0 }`) happens to converge on ~52.94 by coincidence but is NOT the live code path for GCC. 2. Doc finding #4 updated to show the IMPUTE-branch math `(88×0.6 + 0×0.4) / 1.0 = 52.8 → 53` and explicitly notes the else-branch convergence as a coincidence — not the construct's intent. 3. Comment math off-by-one fix at line 107: (88×0.6 + 80×0.4) / (0.6+0.4) = 52.8 + 32.0 = 84.8 → 85 (was incorrectly stated as 85.6 → 86) Test assertion `>= 80 && <= 90` still accepts 85 so behaviour is unchanged; this was a comment-only error that would have misled anyone reproducing the math by hand. Verified - `npx tsx --test tests/resilience-foodwater-field-mapping.test.mts` — 8 pass / 0 fail (IMPUTE-branch anchor test produces 53 as expected) - `npm run lint:md` — clean Also rebased onto updated #3373 (which landed a backtick-escape fix).
koala73
force-pushed
the
fix/resilience-pr53-foodwater-audit
branch
from
April 24, 2026 14:17
83d47f9 to
fa08998
Compare
koala73
added a commit
that referenced
this pull request
Apr 24, 2026
…y domicile question) PR 5.1 of cohort-audit plan 2026-04-24-002. Stacked on PR 5.3 (#3374) so the known-limitations.md section append is additive. Read-only static audit of scoreTradeSanctions + the sanctions:country-counts:v1 seed — framed around the Codex-reformulated construct question: should designated-party domicile count penalize resilience? Findings 1. The count is "OFAC-designated-party domicile locations," NOT "sanctions against this country." Seeder (`scripts/seed-sanctions- pressure.mjs:85-93`) parses OFAC Advanced XML SDN + Consolidated, extracts each designated party's Locations, and increments `map[countryCode]` by 1 for every location country on that party. 2. The count conflates three semantically distinct categories a resilience construct might treat differently: (a) Country-level sanction target (NK SDN listings) — correct penalty (b) Domiciled sanctioned entity (RU bank in Moscow, post-2022) — debatable, country hosts the actor (c) Transit / shell entity (UAE trading co listed under SDGT for Iran evasion; CY SPV for a Russian oligarch) — country is NOT the target, but takes the penalty 3. Observed GCC cohort impact: AE scores 54 vs KW/QA 82. The −28 gap is almost entirely driven by category (c) listings — AE is a financial hub where sanctioned parties incorporate shells. 4. Three options documented for the construct decision (NOT decided in this PR): - Option 1: Keep flat count (status quo, defensible via secondary- sanctions / FATF argument) - Option 2: Program-weighted count — weight DPRK/IRAN/SYRIA/etc. at 1.0, SDGT/SDNTK/CYBER/etc. at 0.3-0.5. Recommended; seeder already captures `programs` per entry — data is there, scorer just doesn't read it. - Option 3: Transit-hub exclusion list (AE, SG, HK, CY, VG, KY) — brittle + normative, not recommended 5. Recommendation documented: Option 2. Implementation deferred to a separate methodology-decision PR (outside auto-mode authority). Shipped - `docs/methodology/known-limitations.md` — new section extending the file: "tradeSanctions — designated-party domicile construct question." Covers what the count represents, the three categories with examples, observed GCC impact, three options w/ trade-offs, recommendation, follow-up audit list (entity-sample gated on API-key access), and file references. - `tests/resilience-sanctions-field-mapping.test.mts` (new) — 10 regression-guard tests pinning CURRENT behavior: 1-6. normalizeSanctionCount piecewise anchors: count=0→100, 1→90, 10→75, 50→50, 200→25, 500→≤1 7. Monotonicity: strictly decreasing across the ramp 8. Country absent from map defaults to count=0 → score 100 (intentional "no designated parties here" semantics) 9. Seed outage (raw=null) → null score slot, NOT imputed (protects against silent data-outage scoring) 10. Construct anchor: count=1 is exactly 10 points below count=0 (pins the "first listing drops 10" design choice) Verified - `npx tsx --test tests/resilience-sanctions-field-mapping.test.mts` — 10 pass / 0 fail - `npm run test:data` — 6721 pass / 0 fail - `npm run typecheck` / `typecheck:api` — green - `npm run lint` / `lint:md` — clean
koala73
added a commit
that referenced
this pull request
Apr 24, 2026
…y domicile question) PR 5.1 of cohort-audit plan 2026-04-24-002. Stacked on PR 5.3 (#3374) so the known-limitations.md section append is additive. Read-only static audit of scoreTradeSanctions + the sanctions:country-counts:v1 seed — framed around the Codex-reformulated construct question: should designated-party domicile count penalize resilience? Findings 1. The count is "OFAC-designated-party domicile locations," NOT "sanctions against this country." Seeder (`scripts/seed-sanctions- pressure.mjs:85-93`) parses OFAC Advanced XML SDN + Consolidated, extracts each designated party's Locations, and increments `map[countryCode]` by 1 for every location country on that party. 2. The count conflates three semantically distinct categories a resilience construct might treat differently: (a) Country-level sanction target (NK SDN listings) — correct penalty (b) Domiciled sanctioned entity (RU bank in Moscow, post-2022) — debatable, country hosts the actor (c) Transit / shell entity (UAE trading co listed under SDGT for Iran evasion; CY SPV for a Russian oligarch) — country is NOT the target, but takes the penalty 3. Observed GCC cohort impact: AE scores 54 vs KW/QA 82. The −28 gap is almost entirely driven by category (c) listings — AE is a financial hub where sanctioned parties incorporate shells. 4. Three options documented for the construct decision (NOT decided in this PR): - Option 1: Keep flat count (status quo, defensible via secondary- sanctions / FATF argument) - Option 2: Program-weighted count — weight DPRK/IRAN/SYRIA/etc. at 1.0, SDGT/SDNTK/CYBER/etc. at 0.3-0.5. Recommended; seeder already captures `programs` per entry — data is there, scorer just doesn't read it. - Option 3: Transit-hub exclusion list (AE, SG, HK, CY, VG, KY) — brittle + normative, not recommended 5. Recommendation documented: Option 2. Implementation deferred to a separate methodology-decision PR (outside auto-mode authority). Shipped - `docs/methodology/known-limitations.md` — new section extending the file: "tradeSanctions — designated-party domicile construct question." Covers what the count represents, the three categories with examples, observed GCC impact, three options w/ trade-offs, recommendation, follow-up audit list (entity-sample gated on API-key access), and file references. - `tests/resilience-sanctions-field-mapping.test.mts` (new) — 10 regression-guard tests pinning CURRENT behavior: 1-6. normalizeSanctionCount piecewise anchors: count=0→100, 1→90, 10→75, 50→50, 200→25, 500→≤1 7. Monotonicity: strictly decreasing across the ramp 8. Country absent from map defaults to count=0 → score 100 (intentional "no designated parties here" semantics) 9. Seed outage (raw=null) → null score slot, NOT imputed (protects against silent data-outage scoring) 10. Construct anchor: count=1 is exactly 10 points below count=0 (pins the "first listing drops 10" design choice) Verified - `npx tsx --test tests/resilience-sanctions-field-mapping.test.mts` — 10 pass / 0 fail - `npm run test:data` — 6721 pass / 0 fail - `npm run typecheck` / `typecheck:api` — green - `npm run lint` / `lint:md` — clean
koala73
added a commit
that referenced
this pull request
Apr 24, 2026
…y domicile question) (#3375) * docs(resilience): PR 5.1 — sanctions construct audit (designated-party domicile question) PR 5.1 of cohort-audit plan 2026-04-24-002. Stacked on PR 5.3 (#3374) so the known-limitations.md section append is additive. Read-only static audit of scoreTradeSanctions + the sanctions:country-counts:v1 seed — framed around the Codex-reformulated construct question: should designated-party domicile count penalize resilience? Findings 1. The count is "OFAC-designated-party domicile locations," NOT "sanctions against this country." Seeder (`scripts/seed-sanctions- pressure.mjs:85-93`) parses OFAC Advanced XML SDN + Consolidated, extracts each designated party's Locations, and increments `map[countryCode]` by 1 for every location country on that party. 2. The count conflates three semantically distinct categories a resilience construct might treat differently: (a) Country-level sanction target (NK SDN listings) — correct penalty (b) Domiciled sanctioned entity (RU bank in Moscow, post-2022) — debatable, country hosts the actor (c) Transit / shell entity (UAE trading co listed under SDGT for Iran evasion; CY SPV for a Russian oligarch) — country is NOT the target, but takes the penalty 3. Observed GCC cohort impact: AE scores 54 vs KW/QA 82. The −28 gap is almost entirely driven by category (c) listings — AE is a financial hub where sanctioned parties incorporate shells. 4. Three options documented for the construct decision (NOT decided in this PR): - Option 1: Keep flat count (status quo, defensible via secondary- sanctions / FATF argument) - Option 2: Program-weighted count — weight DPRK/IRAN/SYRIA/etc. at 1.0, SDGT/SDNTK/CYBER/etc. at 0.3-0.5. Recommended; seeder already captures `programs` per entry — data is there, scorer just doesn't read it. - Option 3: Transit-hub exclusion list (AE, SG, HK, CY, VG, KY) — brittle + normative, not recommended 5. Recommendation documented: Option 2. Implementation deferred to a separate methodology-decision PR (outside auto-mode authority). Shipped - `docs/methodology/known-limitations.md` — new section extending the file: "tradeSanctions — designated-party domicile construct question." Covers what the count represents, the three categories with examples, observed GCC impact, three options w/ trade-offs, recommendation, follow-up audit list (entity-sample gated on API-key access), and file references. - `tests/resilience-sanctions-field-mapping.test.mts` (new) — 10 regression-guard tests pinning CURRENT behavior: 1-6. normalizeSanctionCount piecewise anchors: count=0→100, 1→90, 10→75, 50→50, 200→25, 500→≤1 7. Monotonicity: strictly decreasing across the ramp 8. Country absent from map defaults to count=0 → score 100 (intentional "no designated parties here" semantics) 9. Seed outage (raw=null) → null score slot, NOT imputed (protects against silent data-outage scoring) 10. Construct anchor: count=1 is exactly 10 points below count=0 (pins the "first listing drops 10" design choice) Verified - `npx tsx --test tests/resilience-sanctions-field-mapping.test.mts` — 10 pass / 0 fail - `npm run test:data` — 6721 pass / 0 fail - `npm run typecheck` / `typecheck:api` — green - `npm run lint` / `lint:md` — clean * fix(resilience): PR 5.1 review — tighten count=500 assertion; clarify weightedBlend weights Addresses 2 P2 Greptile findings on #3375: 1. Tighten count=500 assertion. Was `<= 1` with a comment stating the exact value is 0. That loose bound silently tolerates roundScore / boundary drift that would be the very signal this regression guard exists to catch. Changed to strict equality `=== 0`. 2. Clarify the "zero weight" comment on the sanctions-only harness. The other slots DO contribute their declared weights (0.15 + 0.15 + 0.25 = 0.55) to weightedBlend's `totalWeight` denominator — only `availableWeight` (the score-computation denominator) drops to 0.45 because their score is null. The previous comment elided this distinction and could mislead a reader into thinking the null slots contributed nothing at all. Expanded to state exactly how `coverage` and `score` each behave. Verified - `npx tsx --test tests/resilience-sanctions-field-mapping.test.mts` — 10 pass / 0 fail (count=500 now pins the exact 0 floor)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR 5.3 of the cohort-audit plan (
docs/plans/2026-04-24-002-fix-resilience-cohort-ranking-structural-audit-plan.md). Read-only static audit ofscoreFoodWater— the dimension where all GCC countries observed score ~53.⚠ Stacked on #3373 (PR 5.2). The base is
fix/resilience-pr52-displacement-audit, so theknown-limitations.mdappend is additive. Merge PR 5.2 first, then this PR rebases onto main cleanly.Ships one methodology-doc extension + 8 regression-guard tests. No scorer behaviour changes.
Finding
The 53-all-GCC observation is construct-deterministic, not a regional-default leak. Pinned mathematically:
fao == nullbranch imputesIMPUTE.ipcFood = { score: 88, certaintyCoverage: 0.7, imputationClass: 'stable-absence' }at combined weight 0.6.ER.H2O.FWST.ZS(labelled'water stress') for GCC is EXTREME — Kuwait ~3200%, Bahrain ~3400%, UAE ~2080%, Qatar ~770% (freshwater withdrawal exceeds available renewable resources due to desalination). Values > 100 clamp the AQUASTAT sub-score to 0 under the scorer's lower-better 0..100 normaliser.Every GCC country has the same three inputs → the same output. That's construct math, not a hardcoded regional lookup. Pinned as an anchor test (value=2000 → score=53).
What shipped
docs/methodology/known-limitations.md— new section extending the file from PR 5.2: "foodWater scorer — construct-deterministic cohort identity". Covers source-to-scorer mapping table, the exact blend math, why identical inputs give identical outputs, what a follow-up data-side spot-check would verify.tests/resilience-foodwater-field-mapping.test.mts(new) — 8 regression-guard tests:'water stress'→ lower-better;'renewable water availability'→ higher-betterfao=nullwith static record present → imputes 88; observed AQUASTAT wins →imputationClass=null(weightedBlend T1.7 rule)imputationClass='stable-absence'Testing
npx tsx --test tests/resilience-foodwater-field-mapping.test.mts— 8 pass / 0 failnpm run test:data— 6711 pass / 0 fail (PR 5.2's 9 + PR 5.3's 8 stacked on PR 5.2's base)npm run typecheck/typecheck:api— greennpm run lint/lint:md— cleanImplication — no fix required
The scorer is producing the construct it's specified to produce. The observed GCC cohort identity is a correct summary: "non-crisis food security + severe water-withdrawal stress." A future construct decision might split
foodWaterinto separate food and water sub-dims so one saturated sub-signal doesn't dominate the combined dim for desert economies — but that's a construct redesign, not a bug.Post-Deploy Monitoring & Validation
No additional operational monitoring required: pure docs + tests. Zero behaviour change, no endpoints/crons/flags/Redis writes touched.Follow-ups (not in this PR)
ER.H2O.FWST.ZSvalues for GCC + IL + JO live, compare against seeded values. Requires API key. Listed inknown-limitations.md.foodWaterbe split into independentfoodandwaterdims so the water-stress signal doesn't saturate for desert economies? Separate methodology PR if wanted.Related
docs/plans/2026-04-24-002-fix-resilience-cohort-ranking-structural-audit-plan.md§PR 5.3🤖 Generated with Claude Opus 4.7 (1M context) via Claude Code + Compound Engineering v2.49.0
Co-Authored-By: Claude Opus 4.7 (1M context) [email protected]