Summary
scripts/regional-snapshot/snapshot-meta.mjs produces a snapshot_confidence score that is presented to users as a trust signal for regional intelligence snapshots. The score is structurally broken in two ways that compound: stale inputs default to "fresh" when they lack a timestamp, and valid_until is unconditionally set to +6 hours regardless of input freshness. A snapshot built entirely from stale data will display high confidence and remain marked valid for 6 hours.
Defect 1 — Undated inputs treated as fresh (freshness.mjs:73-77)
// freshness.mjs lines 73-77 (classifyInputs):
// 3. If neither yields a timestamp, fall back to "fresh" (cannot prove
// staleness). This fallback remains so we don't regress existing
// keys that have never needed a meta key.
When an upstream seeder crashes and leaves a stale Redis payload that has no timestamp field, classifyInputs() returns "fresh" rather than "stale". The snapshot is built from the old data and snapshot_confidence is computed as if all inputs are current.
This is acknowledged in the codebase itself (todos/181-pending-p2-undated-inputs-treated-as-fresh-confidence-bug.md) but unfixed.
Defect 2 — valid_until hardcoded to +6 hours regardless of input staleness (snapshot-meta.mjs:53)
// snapshot-meta.mjs:53
valid_until: Date.now() + 6 * 60 * 60 * 1000, // 6h
valid_until is computed at snapshot write time, always set 6 hours ahead, and never reduced based on whether any inputs are stale. A snapshot that classifies 80% of inputs as stale still claims full 6-hour validity.
Combined impact
The two defects compound:
- Seeder crashes → stale payload with no timestamp
classifyInputs returns "fresh" for that payload (Defect 1)
snapshot_confidence formula: 0.6 * completeness + 0.4 * freshness computes a high score
valid_until is set to +6h (Defect 2)
- Users see a high-confidence, valid-for-6-hours snapshot built from stale data
There is no observable signal to the user that any of this happened.
Confidence formula
// snapshot-meta.mjs:43
const snapshot_confidence = round(0.6 * cCompleteness + 0.4 * cFreshness);
The weights 0.6 and 0.4 are also undocumented, but the more critical issue is the freshness default that makes cFreshness artificially high.
Suggested remediation
- In
classifyInputs(): flip the default for present-but-undated inputs from "fresh" to "stale". The existing metaKey pattern already handles keys that legitimately lack timestamps — they should be using the metaKey path.
- In
buildPreMeta(): derive valid_until from the oldest fresh input's expected age rather than a fixed offset.
- Expose
stale_inputs and missing_inputs in the UI alongside the confidence score so users can assess the basis for the score.
Summary
scripts/regional-snapshot/snapshot-meta.mjsproduces asnapshot_confidencescore that is presented to users as a trust signal for regional intelligence snapshots. The score is structurally broken in two ways that compound: stale inputs default to "fresh" when they lack a timestamp, andvalid_untilis unconditionally set to+6 hoursregardless of input freshness. A snapshot built entirely from stale data will display high confidence and remain marked valid for 6 hours.Defect 1 — Undated inputs treated as fresh (
freshness.mjs:73-77)When an upstream seeder crashes and leaves a stale Redis payload that has no timestamp field,
classifyInputs()returns"fresh"rather than"stale". The snapshot is built from the old data andsnapshot_confidenceis computed as if all inputs are current.This is acknowledged in the codebase itself (
todos/181-pending-p2-undated-inputs-treated-as-fresh-confidence-bug.md) but unfixed.Defect 2 —
valid_untilhardcoded to+6 hoursregardless of input staleness (snapshot-meta.mjs:53)valid_untilis computed at snapshot write time, always set 6 hours ahead, and never reduced based on whether any inputs are stale. A snapshot that classifies 80% of inputs as stale still claims full 6-hour validity.Combined impact
The two defects compound:
classifyInputsreturns "fresh" for that payload (Defect 1)snapshot_confidenceformula:0.6 * completeness + 0.4 * freshnesscomputes a high scorevalid_untilis set to+6h(Defect 2)There is no observable signal to the user that any of this happened.
Confidence formula
The weights
0.6and0.4are also undocumented, but the more critical issue is the freshness default that makescFreshnessartificially high.Suggested remediation
classifyInputs(): flip the default for present-but-undated inputs from"fresh"to"stale". The existingmetaKeypattern already handles keys that legitimately lack timestamps — they should be using the metaKey path.buildPreMeta(): derivevalid_untilfrom the oldest fresh input's expected age rather than a fixed offset.stale_inputsandmissing_inputsin the UI alongside the confidence score so users can assess the basis for the score.