feat(disease-outbreaks): Sprint 2 — content-age pilot (synthetic tagging + 9d budget)#3597
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryImplements Sprint 2 of the health-readiness content-age plan: adds Confidence Score: 4/5Safe to merge; no data-correctness or ordering bugs found — only test quality P2s. All P0/P1 surfaces checked: contentMeta runs before publishTransform (confirmed in _seed-utils.mjs), helpers are stripped correctly and never reach canonical key or API responses, null/undefined/Infinity edge cases in contentMeta are handled, and the maxContentAgeMin integer contract is satisfied. Two P2 findings (replicated test logic, timing sensitivity in one test) do not affect runtime correctness. tests/disease-outbreaks-seed.test.mjs — test suite validates local replicas rather than actual seeder functions, creating a silent drift risk on future seeder changes. Important Files Changed
Sequence DiagramsequenceDiagram
participant F as fetchDiseaseOutbreaks
participant P as Parsers (WHO/RSS/TGH)
participant MI as mapItem
participant CM as contentMeta
participant PT as publishTransform
participant RD as Redis (canonical key)
participant HLT as /api/health
F->>P: fetch raw items
P-->>F: items with _publishedAtIsSynthetic + _originalPublishedMs
F->>MI: map each item
MI-->>F: normalized items (helpers forwarded)
Note over F: data = { outbreaks, fetchedAt }
F->>CM: contentMeta(data) [RAW - helpers present]
CM-->>F: { newestItemAt, oldestItemAt } or null
Note over CM: Skips synthetic items<br/>Skips ts > skewLimit (1h)<br/>Returns null → STALE_CONTENT
F->>PT: publishTransform(data)
PT-->>F: publishData (helpers stripped)
F->>RD: atomicPublish(publishData) + envelope with newestItemAt/maxContentAgeMin
RD-->>F: ACK
HLT->>RD: read seed-meta
RD-->>HLT: newestItemAt, maxContentAgeMin=12960
HLT-->>HLT: ageMin > 12960 → STALE_CONTENT
Reviews (1): Last reviewed commit: "feat(disease-outbreaks): Sprint 2 — cont..." | Re-trigger Greptile |
…est drift and timing flake Greptile P2s on PR #3597: 1. tests/disease-outbreaks-seed.test.mjs replicated parser/mapper/contentMeta logic locally — a drift in fetchWhoDonApi or contentMeta would not have failed any of the 16 tests because they asserted against their own copy of the logic, not the seeder's. 2. The "near-future ≤1h accepted" test relied on Date.now() being stable between test setup and the call into contentMeta. On a loaded CI runner the gap could exceed the (1h - 30min) margin and flake. Fixes both at once: - New scripts/_disease-outbreaks-helpers.mjs exports the pure functions (whoNormalizeItem, rssNormalizeItem, tghNormalizeItem, mapItem, diseaseContentMeta, diseasePublishTransform, DISEASE_MAX_CONTENT_AGE_MIN). diseaseContentMeta accepts an optional nowMs for deterministic skew tests. - Seeder imports those helpers instead of inlining them. ~150 lines removed; behavior unchanged (verified by node -c + smoke test). - Test file imports the real helpers (no replicas). All skew-limit tests inject FIXED_NOW=1700000000000 — no wall-clock dependence. - Tightens the "within 1h tolerance" test from +30min to +5min ahead of injected NOW, well clear of the 1h boundary regardless of the timing fix. Net: -265 lines across the two existing files; +200 in the new helpers module. 17/17 disease tests pass; 49/49 across the full Sprint 1+2 stack.
…ing + STALE_CONTENT @ 9d) Implements Sprint 2 of the 2026-05-04 health-readiness plan (docs/plans/2026-05-04-001-feat-health-readiness-probe-content-age-plan.md). Stacked on Sprint 1 (#3596 — content-age probe infra). Migrates disease-outbreaks as the proof-of-concept content-age consumer. Pilot maxContentAgeMin=9 days chosen so the 2026-05-04 11d-old incident would have correctly tripped STALE_CONTENT. == Source-parser changes (3 sources, uniform shape) == scripts/seed-disease-outbreaks.mjs: WHO DON parser (line ~117): tag synthetic timestamps when the upstream omits PublicationDateAndTime. Carry _originalPublishedMs (parsed ms or null) and _publishedAtIsSynthetic (boolean) alongside the existing publishedMs (which keeps its Date.now() fallback for UI consumer compat). RSS parser (line ~150, both CDC and Outbreak News Today): same pattern when pubDate is missing/unparseable. TGH parser (line ~211): always carries non-synthetic since the line-198 filter rejects undated items earlier. Migration is additive — every TGH item gets _publishedAtIsSynthetic: false and _originalPublishedMs: publishedMs so contentMeta + publishTransform apply uniformly. mapItem (line ~244): carries _publishedAtIsSynthetic and _originalPublishedMs through to the output shape so contentMeta can read them at runSeed time. == runSeed opts (Sprint 2 contract) == contentMeta: excludes _publishedAtIsSynthetic items + 1h clock-skew tolerance + null when validCount === 0 (matches list-feed-digest's FUTURE_DATE_TOLERANCE_MS pattern). maxContentAgeMin: 9 * 24 * 60 = 12960 minutes (9 days) — chosen deliberately so the production incident's 11d-old cache would have flagged STALE_CONTENT. Tighter would page on normal WHO/CDC quiet weeks; looser would have missed the incident. publishTransform: strips _publishedAtIsSynthetic + _originalPublishedMs from every item BEFORE atomicPublish so the helpers never reach: - the Redis canonical key (health:disease-outbreaks:v1) - /api/bootstrap response (data.diseaseOutbreaks) - list-disease-outbreaks RPC response - the DiseaseOutbreakItem proto-generated type The Sprint 1 ordering contract (contentMeta runs BEFORE publishTransform) guarantees contentMeta sees the helpers that publishTransform then strips. == Anti-regression tests == tests/disease-outbreaks-seed.test.mjs (NEW) — 16 cases split by layer: Pre-publish (in-memory) layer (5): - WHO without PublicationDateAndTime → tagged synthetic - WHO with valid PublicationDateAndTime → non-synthetic - RSS without pubDate → tagged synthetic - RSS with valid pubDate → non-synthetic - TGH always non-synthetic contentMeta behavior (5): - All-synthetic → null (→ STALE_CONTENT) - Mixed: synthetic with newer publishedAt does NOT win newest - Picks newest+oldest from non-synthetic set - Future-dated items beyond 1h tolerance excluded - NEAR_FUTURE within 1h tolerance accepted publishTransform strip (3): - Both helper fields stripped from every item - publishedAt remains non-null (UI/RPC consumer contract) - Empty + missing outbreaks handled safely End-to-end (1): - contentMeta runs on raw data WITH helpers, publishTransform strips, canonical-shape JSON contains NEITHER _publishedAtIsSynthetic NOR _originalPublishedMs (combined-regex assertion per Codex round 4 P2) Pilot threshold sanity (2): - 11d-old items DO trip the 9d budget (anti-drift on the pilot threshold — any future change to 9d must update this test) - 5d-old items DO NOT trip (no false positive on normal upstream rhythm) Test totals: 95/95 pass across the seed-envelope, seed-contract, seed-utils, content-age, health-content-age, and disease-outbreaks-seed suites. == Verification (post-deploy) == After Railway bundle redeploy: 1. /api/health.diseaseOutbreaks shows contentAgeMin and maxContentAgeMin. 2. Redis canonical health:disease-outbreaks:v1 contains NEITHER _publishedAtIsSynthetic NOR _originalPublishedMs (combined-regex grep returns 0). 3. /api/bootstrap?keys=diseaseOutbreaks response payload helper-free. 4. With current 11d-old WHO/CDC items + bug-pattern data, STALE_CONTENT surfaces in /api/health and ops can act on it.
…est drift and timing flake Greptile P2s on PR #3597: 1. tests/disease-outbreaks-seed.test.mjs replicated parser/mapper/contentMeta logic locally — a drift in fetchWhoDonApi or contentMeta would not have failed any of the 16 tests because they asserted against their own copy of the logic, not the seeder's. 2. The "near-future ≤1h accepted" test relied on Date.now() being stable between test setup and the call into contentMeta. On a loaded CI runner the gap could exceed the (1h - 30min) margin and flake. Fixes both at once: - New scripts/_disease-outbreaks-helpers.mjs exports the pure functions (whoNormalizeItem, rssNormalizeItem, tghNormalizeItem, mapItem, diseaseContentMeta, diseasePublishTransform, DISEASE_MAX_CONTENT_AGE_MIN). diseaseContentMeta accepts an optional nowMs for deterministic skew tests. - Seeder imports those helpers instead of inlining them. ~150 lines removed; behavior unchanged (verified by node -c + smoke test). - Test file imports the real helpers (no replicas). All skew-limit tests inject FIXED_NOW=1700000000000 — no wall-clock dependence. - Tightens the "within 1h tolerance" test from +30min to +5min ahead of injected NOW, well clear of the 1h boundary regardless of the timing fix. Net: -265 lines across the two existing files; +200 in the new helpers module. 17/17 disease tests pass; 49/49 across the full Sprint 1+2 stack.
Unix timestamp 1700000000000 ms is 2023-11-14T22:13:20Z, not 2025-11-14. Test correctness unaffected (FIXED_NOW is just an injected stable epoch), but a reader reasoning about the skew-limit arithmetic would get the mental date math wrong. Greptile P2 on PR #3598 (which copied the same wrong comment from this file when Sprint 3a was branched off).
1bcf7c5 to
287e3e6
Compare
Sparse seeders sub-PR b/c of the 2026-05-04 health-readiness plan. Branched off Sprint 1 (#3596) as a parallel sibling to Sprint 2 (#3597) and Sprint 3a (#3598) per the plan's "Each PR is independently shippable" note (line 498). ## Why this matters IEA monthly oil stocks publish on an M+2 cadence — August data ships in late October/early November. Without a content-age probe, a stalled publication month is invisible to /api/health: the seeder runs fine on its 6h cron, fetchedAt stays fresh, but data.dataMonth never advances. A 45-day budget trips STALE_CONTENT exactly when a month has been missed (e.g. cache shows "2024-08" past Dec 1 when "2024-10" should have landed). ## Shape contract — different from Sprint 2/3a IEA is a SINGLE-SNAPSHOT seeder: every member shares one `dataMonth` ("YYYY-MM" string at the top level), there is no per-item published-at. The new helper parses dataMonth → end-of-month UTC ms (the latest possible observation date in the named period) and returns it as both `newestItemAt` and `oldestItemAt`. Defensive: contentMeta returns null when dataMonth is missing, malformed ("2024-13", "2024-8" single-digit), or future-dated beyond 1h clock-skew tolerance (guards against upstream yearMonth garbage producing e.g. a 2099-12 dataMonth). ## Pattern parity with Sprint 2/3a Following the established pattern: pure helpers in `scripts/_iea-oil-stocks-helpers.mjs` (`dataMonthToEndOfMonthMs`, `ieaOilStocksContentMeta`, `IEA_OIL_STOCKS_MAX_CONTENT_AGE_MIN`). Seeder imports them; tests import them. No replicas. `seed-iea-oil-stocks.mjs` is NOT in Dockerfile.relay (verified via `grep`), so no COPY-line update needed (unlike Sprint 3a's seed-climate-news which IS relay-COPY'd). ## Verification - 15/15 iea content-age tests pass (incl. leap-year, month-rollover, invalid-shape rejection, M+2 lag realism, future-clock-skew defense) - 78/78 across iea seed + Sprint 1 + Sprint 3b stack - typecheck:api clean; lint clean (pre-existing warnings only) - Dockerfile.relay closure test passes (no relay impact)
* feat(iea-oil-stocks): Sprint 3b — content-age probe (45d budget) Sparse seeders sub-PR b/c of the 2026-05-04 health-readiness plan. Branched off Sprint 1 (#3596) as a parallel sibling to Sprint 2 (#3597) and Sprint 3a (#3598) per the plan's "Each PR is independently shippable" note (line 498). ## Why this matters IEA monthly oil stocks publish on an M+2 cadence — August data ships in late October/early November. Without a content-age probe, a stalled publication month is invisible to /api/health: the seeder runs fine on its 6h cron, fetchedAt stays fresh, but data.dataMonth never advances. A 45-day budget trips STALE_CONTENT exactly when a month has been missed (e.g. cache shows "2024-08" past Dec 1 when "2024-10" should have landed). ## Shape contract — different from Sprint 2/3a IEA is a SINGLE-SNAPSHOT seeder: every member shares one `dataMonth` ("YYYY-MM" string at the top level), there is no per-item published-at. The new helper parses dataMonth → end-of-month UTC ms (the latest possible observation date in the named period) and returns it as both `newestItemAt` and `oldestItemAt`. Defensive: contentMeta returns null when dataMonth is missing, malformed ("2024-13", "2024-8" single-digit), or future-dated beyond 1h clock-skew tolerance (guards against upstream yearMonth garbage producing e.g. a 2099-12 dataMonth). ## Pattern parity with Sprint 2/3a Following the established pattern: pure helpers in `scripts/_iea-oil-stocks-helpers.mjs` (`dataMonthToEndOfMonthMs`, `ieaOilStocksContentMeta`, `IEA_OIL_STOCKS_MAX_CONTENT_AGE_MIN`). Seeder imports them; tests import them. No replicas. `seed-iea-oil-stocks.mjs` is NOT in Dockerfile.relay (verified via `grep`), so no COPY-line update needed (unlike Sprint 3a's seed-climate-news which IS relay-COPY'd). ## Verification - 15/15 iea content-age tests pass (incl. leap-year, month-rollover, invalid-shape rejection, M+2 lag realism, future-clock-skew defense) - 78/78 across iea seed + Sprint 1 + Sprint 3b stack - typecheck:api clean; lint clean (pre-existing warnings only) - Dockerfile.relay closure test passes (no relay impact) * fix(iea-oil-stocks): bump budget 45d→90d to cover M+2 natural lag Greptile P1 on PR #3599: a 45-day budget contradicts the helper's own M+2 cadence claim. End-of-observation-month (Aug 31) is ~60-65 days BEFORE publication (~late Oct/early Nov), so fresh-arrival data is already past the 45d threshold at the moment a successful seed run writes it. STALE_CONTENT would have fired on every cron tick. Corrected math: 90d = ~60d natural M+2 lag + ~30d missed-publication slack. Trips only when a month is missed entirely (cache stuck at "2024-08" past mid-Jan when "2024-10" should have landed). Also addresses 3 P2 review nits in the same edit: - Test "60 days old" → "fresh-arrival regression guard: ~60d-old fresh M+2 data does NOT trip" (the math was right, name was wrong; rewrote the test to actually pin the failure mode the P1 cited). - Test "~30 days old" → "~14 days old" (the fixture was "2023-10" = ~14d before FIXED_NOW, not 30). - M+2 lag scenario comment "Sept data published ~Oct 25" → "~late Nov (M+2 cadence)" — Oct 25 is M+1, not M+2. Added: dedicated fresh-arrival regression guard test that asserts a ~75d-old fresh M+2 dataMonth is within budget. Without it, a future budget tightening could re-introduce the immediate-page bug invisibly. Verification: 16/16 iea content-age (was 15/15 — added regression guard); 79/79 across iea seed + Sprint 1 + Sprint 3b stack; typecheck:api clean.
…ing + 9d budget) (koala73#3597) * feat(disease-outbreaks): Sprint 2 — content-age pilot (synthetic tagging + STALE_CONTENT @ 9d) Implements Sprint 2 of the 2026-05-04 health-readiness plan (docs/plans/2026-05-04-001-feat-health-readiness-probe-content-age-plan.md). Stacked on Sprint 1 (koala73#3596 — content-age probe infra). Migrates disease-outbreaks as the proof-of-concept content-age consumer. Pilot maxContentAgeMin=9 days chosen so the 2026-05-04 11d-old incident would have correctly tripped STALE_CONTENT. == Source-parser changes (3 sources, uniform shape) == scripts/seed-disease-outbreaks.mjs: WHO DON parser (line ~117): tag synthetic timestamps when the upstream omits PublicationDateAndTime. Carry _originalPublishedMs (parsed ms or null) and _publishedAtIsSynthetic (boolean) alongside the existing publishedMs (which keeps its Date.now() fallback for UI consumer compat). RSS parser (line ~150, both CDC and Outbreak News Today): same pattern when pubDate is missing/unparseable. TGH parser (line ~211): always carries non-synthetic since the line-198 filter rejects undated items earlier. Migration is additive — every TGH item gets _publishedAtIsSynthetic: false and _originalPublishedMs: publishedMs so contentMeta + publishTransform apply uniformly. mapItem (line ~244): carries _publishedAtIsSynthetic and _originalPublishedMs through to the output shape so contentMeta can read them at runSeed time. == runSeed opts (Sprint 2 contract) == contentMeta: excludes _publishedAtIsSynthetic items + 1h clock-skew tolerance + null when validCount === 0 (matches list-feed-digest's FUTURE_DATE_TOLERANCE_MS pattern). maxContentAgeMin: 9 * 24 * 60 = 12960 minutes (9 days) — chosen deliberately so the production incident's 11d-old cache would have flagged STALE_CONTENT. Tighter would page on normal WHO/CDC quiet weeks; looser would have missed the incident. publishTransform: strips _publishedAtIsSynthetic + _originalPublishedMs from every item BEFORE atomicPublish so the helpers never reach: - the Redis canonical key (health:disease-outbreaks:v1) - /api/bootstrap response (data.diseaseOutbreaks) - list-disease-outbreaks RPC response - the DiseaseOutbreakItem proto-generated type The Sprint 1 ordering contract (contentMeta runs BEFORE publishTransform) guarantees contentMeta sees the helpers that publishTransform then strips. == Anti-regression tests == tests/disease-outbreaks-seed.test.mjs (NEW) — 16 cases split by layer: Pre-publish (in-memory) layer (5): - WHO without PublicationDateAndTime → tagged synthetic - WHO with valid PublicationDateAndTime → non-synthetic - RSS without pubDate → tagged synthetic - RSS with valid pubDate → non-synthetic - TGH always non-synthetic contentMeta behavior (5): - All-synthetic → null (→ STALE_CONTENT) - Mixed: synthetic with newer publishedAt does NOT win newest - Picks newest+oldest from non-synthetic set - Future-dated items beyond 1h tolerance excluded - NEAR_FUTURE within 1h tolerance accepted publishTransform strip (3): - Both helper fields stripped from every item - publishedAt remains non-null (UI/RPC consumer contract) - Empty + missing outbreaks handled safely End-to-end (1): - contentMeta runs on raw data WITH helpers, publishTransform strips, canonical-shape JSON contains NEITHER _publishedAtIsSynthetic NOR _originalPublishedMs (combined-regex assertion per Codex round 4 P2) Pilot threshold sanity (2): - 11d-old items DO trip the 9d budget (anti-drift on the pilot threshold — any future change to 9d must update this test) - 5d-old items DO NOT trip (no false positive on normal upstream rhythm) Test totals: 95/95 pass across the seed-envelope, seed-contract, seed-utils, content-age, health-content-age, and disease-outbreaks-seed suites. == Verification (post-deploy) == After Railway bundle redeploy: 1. /api/health.diseaseOutbreaks shows contentAgeMin and maxContentAgeMin. 2. Redis canonical health:disease-outbreaks:v1 contains NEITHER _publishedAtIsSynthetic NOR _originalPublishedMs (combined-regex grep returns 0). 3. /api/bootstrap?keys=diseaseOutbreaks response payload helper-free. 4. With current 11d-old WHO/CDC items + bug-pattern data, STALE_CONTENT surfaces in /api/health and ops can act on it. * refactor(disease-outbreaks): extract helpers + inject nowMs to kill test drift and timing flake Greptile P2s on PR koala73#3597: 1. tests/disease-outbreaks-seed.test.mjs replicated parser/mapper/contentMeta logic locally — a drift in fetchWhoDonApi or contentMeta would not have failed any of the 16 tests because they asserted against their own copy of the logic, not the seeder's. 2. The "near-future ≤1h accepted" test relied on Date.now() being stable between test setup and the call into contentMeta. On a loaded CI runner the gap could exceed the (1h - 30min) margin and flake. Fixes both at once: - New scripts/_disease-outbreaks-helpers.mjs exports the pure functions (whoNormalizeItem, rssNormalizeItem, tghNormalizeItem, mapItem, diseaseContentMeta, diseasePublishTransform, DISEASE_MAX_CONTENT_AGE_MIN). diseaseContentMeta accepts an optional nowMs for deterministic skew tests. - Seeder imports those helpers instead of inlining them. ~150 lines removed; behavior unchanged (verified by node -c + smoke test). - Test file imports the real helpers (no replicas). All skew-limit tests inject FIXED_NOW=1700000000000 — no wall-clock dependence. - Tightens the "within 1h tolerance" test from +30min to +5min ahead of injected NOW, well clear of the 1h boundary regardless of the timing fix. Net: -265 lines across the two existing files; +200 in the new helpers module. 17/17 disease tests pass; 49/49 across the full Sprint 1+2 stack. * fix(test): correct FIXED_NOW comment year (2025→2023) Unix timestamp 1700000000000 ms is 2023-11-14T22:13:20Z, not 2025-11-14. Test correctness unaffected (FIXED_NOW is just an injected stable epoch), but a reader reasoning about the skew-limit arithmetic would get the mental date math wrong. Greptile P2 on PR koala73#3598 (which copied the same wrong comment from this file when Sprint 3a was branched off).
…la73#3599) * feat(iea-oil-stocks): Sprint 3b — content-age probe (45d budget) Sparse seeders sub-PR b/c of the 2026-05-04 health-readiness plan. Branched off Sprint 1 (koala73#3596) as a parallel sibling to Sprint 2 (koala73#3597) and Sprint 3a (koala73#3598) per the plan's "Each PR is independently shippable" note (line 498). ## Why this matters IEA monthly oil stocks publish on an M+2 cadence — August data ships in late October/early November. Without a content-age probe, a stalled publication month is invisible to /api/health: the seeder runs fine on its 6h cron, fetchedAt stays fresh, but data.dataMonth never advances. A 45-day budget trips STALE_CONTENT exactly when a month has been missed (e.g. cache shows "2024-08" past Dec 1 when "2024-10" should have landed). ## Shape contract — different from Sprint 2/3a IEA is a SINGLE-SNAPSHOT seeder: every member shares one `dataMonth` ("YYYY-MM" string at the top level), there is no per-item published-at. The new helper parses dataMonth → end-of-month UTC ms (the latest possible observation date in the named period) and returns it as both `newestItemAt` and `oldestItemAt`. Defensive: contentMeta returns null when dataMonth is missing, malformed ("2024-13", "2024-8" single-digit), or future-dated beyond 1h clock-skew tolerance (guards against upstream yearMonth garbage producing e.g. a 2099-12 dataMonth). ## Pattern parity with Sprint 2/3a Following the established pattern: pure helpers in `scripts/_iea-oil-stocks-helpers.mjs` (`dataMonthToEndOfMonthMs`, `ieaOilStocksContentMeta`, `IEA_OIL_STOCKS_MAX_CONTENT_AGE_MIN`). Seeder imports them; tests import them. No replicas. `seed-iea-oil-stocks.mjs` is NOT in Dockerfile.relay (verified via `grep`), so no COPY-line update needed (unlike Sprint 3a's seed-climate-news which IS relay-COPY'd). ## Verification - 15/15 iea content-age tests pass (incl. leap-year, month-rollover, invalid-shape rejection, M+2 lag realism, future-clock-skew defense) - 78/78 across iea seed + Sprint 1 + Sprint 3b stack - typecheck:api clean; lint clean (pre-existing warnings only) - Dockerfile.relay closure test passes (no relay impact) * fix(iea-oil-stocks): bump budget 45d→90d to cover M+2 natural lag Greptile P1 on PR koala73#3599: a 45-day budget contradicts the helper's own M+2 cadence claim. End-of-observation-month (Aug 31) is ~60-65 days BEFORE publication (~late Oct/early Nov), so fresh-arrival data is already past the 45d threshold at the moment a successful seed run writes it. STALE_CONTENT would have fired on every cron tick. Corrected math: 90d = ~60d natural M+2 lag + ~30d missed-publication slack. Trips only when a month is missed entirely (cache stuck at "2024-08" past mid-Jan when "2024-10" should have landed). Also addresses 3 P2 review nits in the same edit: - Test "60 days old" → "fresh-arrival regression guard: ~60d-old fresh M+2 data does NOT trip" (the math was right, name was wrong; rewrote the test to actually pin the failure mode the P1 cited). - Test "~30 days old" → "~14 days old" (the fixture was "2023-10" = ~14d before FIXED_NOW, not 30). - M+2 lag scenario comment "Sept data published ~Oct 25" → "~late Nov (M+2 cadence)" — Oct 25 is M+1, not M+2. Added: dedicated fresh-arrival regression guard test that asserts a ~75d-old fresh M+2 dataMonth is within budget. Without it, a future budget tightening could re-introduce the immediate-page bug invisibly. Verification: 16/16 iea content-age (was 15/15 — added regression guard); 79/79 across iea seed + Sprint 1 + Sprint 3b stack; typecheck:api clean.
Stacked on PR #3596 (Sprint 1 — content-age probe infra). Merge #3596 first; this PR rebases onto main automatically once that lands.
Implements Sprint 2 of the 2026-05-04 health-readiness plan. Migrates disease-outbreaks as the proof-of-concept content-age consumer. Pilot
maxContentAgeMin = 9 dayschosen so the 2026-05-04 incident (cache held 50 outbreaks all 11+ days old, /api/health reported OK, map went empty) would have correctly trippedSTALE_CONTENT.Changes (
scripts/seed-disease-outbreaks.mjs)1. Synthetic-timestamp tagging across all 3 sources
fetchWhoDonApi(line ~117)Date.now()whenPublicationDateAndTimemissing_originalPublishedMs(parsed ms or null) +_publishedAtIsSynthetic(boolean).publishedMskeepsDate.now()fallback for UI consumer compat.fetchRssItems(line ~150, both CDC and Outbreak News Today)Date.now()whenpubDatemissingfetchThinkGlobalHealth(line ~211)_publishedAtIsSynthetic: false,_originalPublishedMs: publishedMs(uniform shape across sources)mapItem(line ~244) carries the helpers through to the output shape socontentMetacan read them at runSeed time.2. runSeed contract opts
contentMeta: excludes_publishedAtIsSyntheticitems + 1h clock-skew tolerance + null whenvalidCount === 0. Synthetic-timestamp items would otherwise falsely report content as fresh and mask real upstream silence.maxContentAgeMin: 9 * 24 * 60 = 12960(9 days). Tighter would page on normal WHO/CDC quiet weeks; looser would have missed the 11d incident.publishTransform: strips_publishedAtIsSynthetic+_originalPublishedMsfrom every item BEFORE atomicPublish. Per Sprint 1's ordering contract, contentMeta has already run on the raw data by this point — safe to drop. The strip ensures helpers never reach the canonical key, /api/bootstrap response, list-disease-outbreaks RPC response, or the proto-generated type.Tests (
tests/disease-outbreaks-seed.test.mjs— NEW)16 cases split by layer per Codex round 4-5 review (pre-publish in-memory CARRIES helpers; post-strip canonical is helper-free):
contentMetabehaviorpublishTransformstrippublishedAtremains non-null; empty/missing handledTest totals: 95/95 pass across seed-envelope, seed-contract, seed-utils, content-age, health-content-age, and disease-outbreaks-seed suites.
Net diff
Verification (post-deploy)
After Railway redeploy of
seed-bundle-health:/api/health.diseaseOutbreaksreportscontentAgeMinandmaxContentAgeMin: 12960/api/health.diseaseOutbreaks.status === 'STALE_CONTENT'/api/bootstrap?keys=diseaseOutbreaksresponse body returns 0newestItemAtshifts to within 9 daysWhat's next
Sprint 3 — sparse seeders (climate news, IEA OPEC, news-digest, economic stress).
Sprint 4 — annual-data seeders (WB resilience indicators).