docs(plans): health-readiness probe — content-age tracking#3594
Conversation
Captures the architectural follow-up identified during the 2026-05-04 disease-outbreaks incident: /api/health currently reports seeder-run freshness, not content freshness. For sparse upstream sources (WHO Disease Outbreak News, IEA OPEC reports, central-bank releases, WB annual indicators) these diverge — seeder runs fine, seed-meta fetchedAt stays fresh, but the freshest item the user sees is days or weeks old. Health says OK; UI renders nothing. Plan opts seeders into a parallel content-age contract: - runSeed accepts itemTimestamp / itemsPath / maxContentAgeMin - seed-meta carries newestItemAt/oldestItemAt/maxContentAgeMin when set - api/health reports new STALE_CONTENT status when content is older than the seeder's content-age budget Backwards compatible — legacy seeders without itemTimestamp keep current behavior. Pilot on disease-outbreaks (today's incident's origin), then migrate sparse + annual seeders over Sprints 3-4. Companion to PRs #3582 (canonical-envelope-mirror), #3593 (disease- outbreaks TGH + time-filter fixes), and the broader 'fetched-recently is not the same as fresh-content' insight.
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR adds a plan document for a new Confidence Score: 4/5Safe to merge as a plan doc; the P1 is a spec-level gap that must be caught in the Sprint 1 implementation PR, not a runtime defect today. Docs-only PR with no production code changes. The P1 (readSeedMeta not extended, STALE_CONTENT would silently never fire) is a real implementation trap if Sprint 1 follows the pseudocode literally, but it affects only the future implementation PR, not this merge. P2s are minor naming and omission issues. Score is 4 rather than 5 because the P1 spec gap in a plan doc should be corrected before Sprint 1 begins. docs/plans/2026-05-04-001-feat-health-readiness-probe-content-age-plan.md — the classifyHealth pseudocode section and the STATUS_COUNTS omission need correction before Sprint 1 kicks off. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[classifyKey called] --> B{Redis error or\nmeta read failed?}
B -- Yes --> C[REDIS_PARTIAL]
B -- No --> D{hasData?}
D -- No --> E{cascadeCovered?}
E -- Yes --> F[OK_CASCADE]
E -- No --> G{EMPTY_DATA_OK?}
G -- Yes --> H{seedStale?}
H -- Yes --> I[STALE_SEED]
H -- No --> J[OK]
G -- No --> K{isOnDemand?}
K -- Yes --> L[EMPTY_ON_DEMAND]
K -- No --> M[EMPTY]
D -- Yes --> N{records == 0?}
N -- Yes --> O{EMPTY_DATA_OK / onDemand?}
O --> P[EMPTY_DATA / EMPTY_ON_DEMAND]
N -- No --> Q{seedError?}
Q -- Yes --> R[SEED_ERROR]
Q -- No --> S{seedStale?}
S -- Yes --> I
S -- No --> T{minRecordCount\nviolated?}
T -- Yes --> U[COVERAGE_PARTIAL]
T -- No --> V{NEW: newestItemAt\nand maxContentAgeMin\nin seed-meta?}
V -- No --> J
V -- Yes --> W{contentAgeMin >\nmaxContentAgeMin?}
W -- Yes --> X[STALE_CONTENT]
W -- No --> J
Reviews (1): Last reviewed commit: "docs(plans): health-readiness probe — co..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
|
|
||
| Add the `STALE_CONTENT` status grade. Today `api/health.js:checkSeedKey` reads `seedMeta` and reports `OK` / `STALE_SEED` / `EMPTY_DATA` based on `seedAgeMin` and `recordCount`. The new logic: | ||
|
|
||
| ```js | ||
| function classifyHealth(meta, freshnessConfig) { | ||
| const now = Date.now(); | ||
| const seedAgeMin = (now - meta.fetchedAt) / 60000; | ||
| if (meta.recordCount === 0 && !freshnessConfig.allowEmpty) return 'EMPTY_DATA'; | ||
| if (seedAgeMin > freshnessConfig.maxStaleMin) return 'STALE_SEED'; | ||
|
|
||
| // NEW — only checked when the seeder opted in | ||
| if (typeof meta.newestItemAt === 'number' && typeof meta.maxContentAgeMin === 'number') { | ||
| const contentAgeMin = (now - meta.newestItemAt) / 60000; | ||
| if (contentAgeMin > meta.maxContentAgeMin) return 'STALE_CONTENT'; | ||
| } |
There was a problem hiding this comment.
readSeedMeta not extended — STALE_CONTENT would silently never fire
The classifyHealth pseudocode reads meta.newestItemAt and meta.maxContentAgeMin directly, but in api/health.js the variable meta inside classifyKey is the return value of readSeedMeta() — an object shaped { seedAge, seedStale, seedError, metaReadFailed, metaCount }. Accessing .newestItemAt on that return object yields undefined, so the guard typeof meta.newestItemAt === 'number' is always false and STALE_CONTENT never fires.
Sprint 1 must explicitly extend readSeedMeta to also pull newestItemAt and maxContentAgeMin from the parsed Redis payload (the raw object is at unwrapEnvelope(parseRedisValue(keyMetaValues.get(seedCfg.key))).data) and return them alongside the existing fields, before the check in classifyKey can work.
| `STALE_CONTENT` is structurally distinct from `STALE_SEED`. Operator playbooks differ: | ||
|
|
||
| - `STALE_SEED` → seeder is broken; check Railway logs. | ||
| - `STALE_CONTENT` → upstream isn't publishing (or our parser is dropping recent items); check the source. | ||
|
|
||
| Health response surfaces both ages when the seeder opted in: |
There was a problem hiding this comment.
STALE_CONTENT missing from STATUS_COUNTS
The plan does not include adding STALE_CONTENT to the STATUS_COUNTS map in api/health.js (line 651). If omitted, the ?? 'warn' fallthrough on line 738 silently handles it — but it also means the status doesn't show up in the ok/warn/crit bucket docs, is invisible to anyone reading the map, and any future change to that fallback would silently alter alert severity. Sprint 1 should explicitly add STALE_CONTENT: 'warn' (or whatever tier is decided in Open Question 1) to STATUS_COUNTS.
| } | ||
| if (newest !== -Infinity) { | ||
| contentMeta = { | ||
| newestItemAt: newest, |
There was a problem hiding this comment.
classifyKey, not checkSeedKey
The Sprint 1 architecture section refers to api/health.js:checkSeedKey as the entry point for the new logic. The actual function in the current file is classifyKey (line 597). The counterpart that reads seed-meta is readSeedMeta (line 556). Naming the right functions in the plan reduces the risk of Sprint 1 touching the wrong call sites.
…revisions (#3595) Five rounds of Codex review against plan #3594: - Round 1 (8 findings): pilot threshold won't catch incident; canonical-mirror loses content fields; synthetic timestamps mask staleness; all-undated falls through to OK; wrong health.js target symbols; soft-disabled budget; brittle autodetect; missing tests. Adopted contentMeta(data) -> {newestItemAt, oldestItemAt} API. - Round 2 (3 findings): envelope-writer chain incomplete (need _seed-envelope-source.mjs + parity mirrors + _seed-contract.mjs); classifier precedence wrong; disease snippet broke isNaN filters and mapItem. - Round 3 (3 findings): TGH source missed migration; stale classifier code block contradicted Sprint 1; helpers leaked via list-disease-outbreaks + bootstrap. - Round 4 (3 findings): replacement classifier still used bare 'return' but real code uses status='X' assignment; mapItem section contradicted strip contract; grep missed _originalPublishedMs. - Round 5 (1 finding): test descriptions still said cached items 'have helper fields'; rewrote to separate pre-publish in-memory layer from post-strip published-canonical layer. Net: ~280 prod LOC + ~250 test LOC for Sprint 1; explicit envelope-writer chain coverage; Sprint 2 disease pilot covers all 3 sources (WHO/RSS/TGH) + helper-field strip via publishTransform + anti-regression tests. Companion: PR #3593 (immediate disease-outbreaks fixes), #3582 (canonical-envelope-mirror).
…a73#3594) Captures the architectural follow-up identified during the 2026-05-04 disease-outbreaks incident: /api/health currently reports seeder-run freshness, not content freshness. For sparse upstream sources (WHO Disease Outbreak News, IEA OPEC reports, central-bank releases, WB annual indicators) these diverge — seeder runs fine, seed-meta fetchedAt stays fresh, but the freshest item the user sees is days or weeks old. Health says OK; UI renders nothing. Plan opts seeders into a parallel content-age contract: - runSeed accepts itemTimestamp / itemsPath / maxContentAgeMin - seed-meta carries newestItemAt/oldestItemAt/maxContentAgeMin when set - api/health reports new STALE_CONTENT status when content is older than the seeder's content-age budget Backwards compatible — legacy seeders without itemTimestamp keep current behavior. Pilot on disease-outbreaks (today's incident's origin), then migrate sparse + annual seeders over Sprints 3-4. Companion to PRs koala73#3582 (canonical-envelope-mirror), koala73#3593 (disease- outbreaks TGH + time-filter fixes), and the broader 'fetched-recently is not the same as fresh-content' insight.
…revisions (koala73#3595) Five rounds of Codex review against plan koala73#3594: - Round 1 (8 findings): pilot threshold won't catch incident; canonical-mirror loses content fields; synthetic timestamps mask staleness; all-undated falls through to OK; wrong health.js target symbols; soft-disabled budget; brittle autodetect; missing tests. Adopted contentMeta(data) -> {newestItemAt, oldestItemAt} API. - Round 2 (3 findings): envelope-writer chain incomplete (need _seed-envelope-source.mjs + parity mirrors + _seed-contract.mjs); classifier precedence wrong; disease snippet broke isNaN filters and mapItem. - Round 3 (3 findings): TGH source missed migration; stale classifier code block contradicted Sprint 1; helpers leaked via list-disease-outbreaks + bootstrap. - Round 4 (3 findings): replacement classifier still used bare 'return' but real code uses status='X' assignment; mapItem section contradicted strip contract; grep missed _originalPublishedMs. - Round 5 (1 finding): test descriptions still said cached items 'have helper fields'; rewrote to separate pre-publish in-memory layer from post-strip published-canonical layer. Net: ~280 prod LOC + ~250 test LOC for Sprint 1; explicit envelope-writer chain coverage; Sprint 2 disease pilot covers all 3 sources (WHO/RSS/TGH) + helper-field strip via publishTransform + anti-regression tests. Companion: PR koala73#3593 (immediate disease-outbreaks fixes), koala73#3582 (canonical-envelope-mirror).
1 participant
Plan doc capturing the architectural follow-up identified during the 2026-05-04 disease-outbreaks incident.
Problem
/api/healthreports seeder-RUN freshness, not CONTENT freshness. For sparse upstream sources (WHO Disease Outbreak News, IEA OPEC monthly reports, central-bank releases, WB annual indicators) these diverge — seeder runs fine, seed-metafetchedAtstays fresh, but the freshest item the user actually sees in the cache is days or weeks old. Health says OK; UI renders nothing.Proposal
Opt-in content-age contract:
itemTimestamp(item)extractor +maxContentAgeMinbudgetrunSeedwritesnewestItemAt/oldestItemAt/maxContentAgeMininto seed-meta when declaredapi/healthaddsSTALE_CONTENTstatus that fires when content is older than the seeder's content-age budgetitemTimestampkeep current behaviorSprints
Each sprint is an independent PR; staged rollout reduces risk.
Companion incidents this would catch
The pattern: fetched-recently is not the same as fresh-content. This plan makes the distinction first-class.
Plan doc lives at
docs/plans/2026-05-04-001-feat-health-readiness-probe-content-age-plan.md(force-added sincedocs/plans/is gitignored — matches the convention for past plans #2890, #3402).