fix(classify-cache + brief-filter): cap LLM upgrades + brief-side guards (PR 2/2)#3419
Merged
Merged
Conversation
… (3 sites atomic)
Closes U4 of docs/plans/2026-04-26-001-fix-brief-static-page-contamination-
plan.md. The third compounding bug behind the Pentagon static-page brief
contamination: the classify:sebuf:v3:* LLM cache could promote any
keyword classification to any LLM-classified level, regardless of the
gap between them. A title like "About Section 508" with keyword=info
(no-match fallback at confidence 0.3) could end up with cache hit
level=high, which combined with Pentagon's tier-1 source-tier boost
cleared every importance-score threshold downstream.
Two-part fix:
1. Tier cap. capLlmUpgrade(keywordLevel, llmLevel) clamps to
keywordRank + 2. Blocks the contamination class:
- keyword=info + LLM=critical → caps at medium (info+2)
- keyword=info + LLM=high → caps at medium
Preserves legitimate upgrades:
- keyword=medium + LLM=critical → critical (medium+2 = critical)
- keyword=low + LLM=high → high (low+2 = high)
Tradeoff: keyword=low + LLM=critical caps at high (low+2 = high).
Logged on every cap-fire so the loss is measurable, not silent.
Reviewer noted +1 was too aggressive; +2 is the right cut line.
2. Prefix bump v3→v4 across all three sites atomically.
- server/worldmonitor/intelligence/v1/_shared.ts (canonical writer)
- server/worldmonitor/news/v1/list-feed-digest.ts — refactored to
import buildClassifyCacheKey from _shared so the literal lives in
exactly ONE place. Future bumps only need to touch _shared.ts +
the relay's independent inline helper.
- scripts/ais-relay.cjs:3138 — the relay maintains its own helper
because .cjs cannot import from .ts. Updated in lockstep.
Static-analysis test (tests/news-classify-cache-prefix-audit.test.mjs)
grep-asserts zero non-canonical `classify:sebuf:vN` literals across
.ts/.mjs/.cjs/.js/.json — defense against the next bump leaving the
relay site behind. The cache-prefix-bump-propagation-scope learning is
the recipe.
Tier cap tests at tests/news-classify-cache-tier-cap.test.mts cover
within-cap, contamination case, malformed LLM levels, and the
keyword=critical edge case (the existing 0.9-confidence guard at
list-feed-digest.ts:480 already skips cache for keyword=critical;
the cap is layered defense).
16 new tests pass, 75 tests total across the touched + parity-coupled
surface (importance-score-parity preserved — formula untouched).
Plan: docs/plans/2026-04-26-001-fix-brief-static-page-contamination-plan.md
…+U7) Closes U5, U6, U7 of docs/plans/2026-04-26-001-fix-brief-static-page- contamination-plan.md. Three brief-side guards that complement the ingest-side hardening from PR #3417 + the classify-cache fix in the prior commit. U5 — source-topic cap (default 2 per source+category) filterTopStories now drops the 3rd+ story sharing a (source, category) pair within a single brief. Surgical fix for the editorial-clutter case where 2026-04-25-2001 shipped both "Millions under tornado threat" and "Watch tornadoes swirl through Oklahoma" from CBS News — distinct stories the dedup correctly kept separate, but redundant. Cap is applied AFTER applyRankedOrder, so the highest-importance member of each pair survives. Tunable via maxPerSourceTopic param (default 2, pass Infinity to disable). Emits onDrop with reason 'source_topic_cap'. U6 — shared URL classifier + ops audit script Pure helper at shared/url-classifier.js (mirrored to scripts/shared/ byte-identical, enforced by tests/edge-functions.test.mjs per the shared-dir-mirror-requirement learning). isInstitutionalStaticPage(url) is conservative-by-design: must match BOTH .gov/.mil/.int host AND a curated path prefix (/About/, /Section-, /Strategy/, /Strategies/, /Policy/, /Policies/, /Resources/, /Programs/, /Acquisition- Transformation-Strategy). Plus the ops script scripts/audit-static-page-contamination.mjs that SCANs story:track:v1:* with cursor-based batching, classifies each via the helper, and on --apply DELs both the track key and its story:sources:v1:{hash} sibling. Dry run reports per-host + per-path-pattern stats so any patterns the starter regex misses surface in production data and can drive a follow-up tightening PR. U7 — brief-filter URL denylist (defense in depth) After normaliseSourceUrl returns a valid URL, filterTopStories now calls isInstitutionalStaticPage(sourceUrl) and drops with reason 'institutional_static_page' when true. Last line of defense — the ingest gates from PR #3417 should prevent these reaching story:track: v1 in the first place, but a regression in the feed registry or a new date-dialect bypassing U2 could let one through, and the brief surface is the user-visible failure mode. Test additions: - tests/url-classifier.test.mjs (24 cases) — known contamination URLs classify true; legitimate /News/ paths on the same hosts classify false; non-institutional hosts classify false; defensive on bad input (no throws). - tests/brief-filter.test.mjs +9 cases for U5/U7 — within cap, over cap, source/category isolation, ranked-order survival, Multiple-wires/General fallbacks, U7 institutional-static-page drop with onDrop reason, maxPerSourceTopic override. - tests/brief-from-digest-stories.test.mjs — 3 fixtures updated to vary sources so the source-topic cap doesn't dominate tests of other behaviors (maxStories cap, ranking). - tests/edge-functions.test.mjs +1 line — adds url-classifier.js to the explicit-mirrored-files set so future drift fails CI. DropMetricsFn typedef extended with 'source_topic_cap' and 'institutional_static_page' reasons. The reconciliation invariant test (in === out + sum(dropped_*)) updated to include both new reasons. 533/533 tests across the touched + parity-coupled surface (parity test preserved — formula untouched). Plan: docs/plans/2026-04-26-001-fix-brief-static-page-contamination-plan.md
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Greptile SummaryThis PR adds four defense-in-depth layers against static-institutional-page contamination in user briefs: an LLM cache upgrade cap (+2 tiers), a source-topic cap (default 2 per source+category), a shared URL classifier for
Confidence Score: 3/5Safe to merge for production (brief filter and cap logic are correct); the P1 bug is isolated to the one-shot ops audit script and should be fixed before --apply is run post-deploy. A single P1 logic bug exists in the audit script batchHgetAll, which can silently produce false-zero match counts if Upstash returns HGETALL results as an object. All production-path changes (brief-filter, capLlmUpgrade, cache prefix bump) appear correct and well-tested. scripts/audit-static-page-contamination.mjs — batchHgetAll dead-code path must be fixed before the --apply run. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Ingest: RSS/feed parser] -->|story:track:v1 written| B[LLM Classify Cache\nclassify:sebuf:v4:]
B -->|cache hit| C{capLlmUpgrade\nkeyword + 2 tier cap}
C -->|within cap| D[Apply LLM level]
C -->|exceeds cap| E[Clamp to keyword+2\nlog warn]
D --> F[filterTopStories]
E --> F
F -->|isInstitutionalStaticPage?| G{U7: URL denylist\n.gov/.mil/.int + static path}
G -->|true| H[Drop: institutional_static_page]
G -->|false| I{U5: source-topic cap\ndefault 2 per source+category}
I -->|over cap| J[Drop: source_topic_cap]
I -->|within cap| K[Brief surface ✓]
L[audit-static-page-contamination.mjs] -->|SCAN story:track:v1:*| M[Redis / Upstash]
M -->|HGETALL per batch| N{isInstitutionalStaticPage?}
N -->|true| O[Dry-run report / --apply DEL]
N -->|false| P[Skip]
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+114
to
+126
| } | ||
|
|
||
| function summarizePerHostPath(matches) { | ||
| const hostCounts = new Map(); | ||
| const pathPatternCounts = new Map(); | ||
| for (const m of matches) { | ||
| try { | ||
| const u = new URL(m.link); | ||
| hostCounts.set(u.hostname, (hostCounts.get(u.hostname) ?? 0) + 1); | ||
| // First two path segments give a useful pattern bucket without | ||
| // exploding cardinality (e.g., '/About' vs '/About/Section-508'). | ||
| const segs = u.pathname.split('/').filter(Boolean); | ||
| const bucket = '/' + segs.slice(0, 2).join('/'); |
Contributor
There was a problem hiding this comment.
The first guard if (!Array.isArray(arr) || arr.length === 0) return null short-circuits for any non-array value, including a plain object. This means the subsequent check if (typeof arr === 'object' && !Array.isArray(arr)) return arr is unreachable dead code — arr is guaranteed to be an array by the time execution reaches it.
If Upstash ever returns HGETALL results as a plain object {field: value, ...} (a format the code comment explicitly anticipates), every such entry is silently coerced to null, causing the audit to report zero matches even when contaminated entries exist. The --apply path would then delete nothing, defeating the purpose of the script.
Fix: check for null/undefined and the object case before the array check.
Comment on lines
+251
to
+258
| let existingForPair = 0; | ||
| for (const s of out) { | ||
| if (s.source === source && s.category === category) existingForPair++; | ||
| } | ||
| if (existingForPair >= maxPerSourceTopic) { | ||
| if (emit) emit({ reason: 'source_topic_cap', severity: threatLevel, sourceUrl }); | ||
| continue; | ||
| } |
Contributor
There was a problem hiding this comment.
For every story that passes earlier gates, this iterates over the entire out array to count accepted stories with the same (source, category) pair. With the default maxStories = 12 this is at most ~144 comparisons and is harmless today. Consider replacing it with a Map keyed by source + ":" + category initialised before the story loop, so the lookup is O(1) per story. pairCounts.set(key, (pairCounts.get(key) ?? 0) + 1) after out.push(...) keeps the counter in sync without an extra scan.
…ape (review fixes on #3419) Three review findings: 1. scripts/seed-digest-notifications.mjs drop log was missing the new reasons. filterTopStories now emits `source_topic_cap` (U5) and `institutional_static_page` (U7), but the seeder's per-attempt drop line only tabulated severity/headline/url/shape/cap. Operator reconciliation (in === out + sum(dropped_*)) silently broke when either new guard fired. Both reasons added to the dropStats init block AND the structured log format. brief-quality-report.mjs's regex parser is generic so legacy parsing keeps working; adding per-reason rates to the report summary is a small follow-up. 2. scripts/audit-static-page-contamination.mjs HGETALL normalization had a dead branch. `if (!Array.isArray(arr)) return null` ran BEFORE the object-shape branch, so any Upstash response shaped as `{k1:v1, k2:v2}` (newer client versions / direct REST) would normalize to null and the audit would silently miss those rows AND --apply would delete nothing for them. Reordered: object branch runs first (with empty-object check), then array branch for the flat ['k1','v1','k2','v2',...] shape. 3. shared/brief-filter.d.ts was stale. DropMetricsFn was missing `source_topic_cap` and `institutional_static_page` in its reason union, and filterTopStories was missing the `maxPerSourceTopic` parameter. TypeScript consumers couldn't use the new option or exhaustively switch on the new reasons. Both updated with doc comments matching the JS implementation. 311/311 tests pass; importance-score-parity preserved.
…ap lookup Greptile P2 on PR #3419: the U5 source-topic cap iterates over the entire in-flight `out` array per candidate to count survivors with the same (source, category) pair. With the default maxStories=12 this is at most ~144 comparisons — harmless today, but the Map-based version is both faster and more robust to future cap-relaxation experiments. Fix: introduce a `pairCounts: Map<string, number>` initialised before the story loop. The cap check is now O(1) (`pairCounts.get(key) ?? 0`), and the counter is incremented atomically after each successful out.push(). Key construction uses `String.fromCharCode(31)` (ASCII Unit Separator 0x1F) as the delimiter rather than a literal space, preventing a subtle collision class: under a space delimiter, ('Reuters', 'World Politics') and ('Reuters World', 'Politics') both produce the same Map key 'Reuters World Politics' and would share a counter — over- or under-dropping each other depending on encounter order. With 0x1F (which never appears in source/category text), the keys stay distinct. Added a regression test for the collision case so any future delimiter change must demonstrate it preserves the property. 312/312 tests pass; importance-score-parity preserved.
…cap rationale (review fixes on #3419) Two findings from review: P2 — STATIC_PATH_PREFIXES with trailing slashes used a naive startsWith check, so `/About/` matched `/about/section-508` but NOT the bare `/about` form (the canonical landing-page URL on most .gov sites). Same for `/Strategy/`, `/Strategies/`, `/Policy/`, `/Policies/`, `/Resources/`, `/Programs/`. That weakened both U7 brief-side guard and U6 audit cleanup for very common contamination shapes. Fix: pathMatchesPrefix(path, prefix) helper with two rules: - Prefix ends with '/' → segment-bucket. Drop trailing slash, match exact stem OR `stem/...`. So `/About/` matches `/about` AND `/about/section-508`, but NOT `/aboutface` (segment boundary enforced — no over-match on prefix-letter collisions). - Prefix doesn't end with '/' → wildcard literal startsWith. So `/Section-` keeps matching `/section-508`, `/section-504`, etc., where the dash-suffix is part of the bucket name. Wildcard prefixes (`/Section-`, `/Acquisition-Transformation-Strategy`) are unchanged. Segment-bucket prefixes (the 7 `/.../` entries) now match both with-trailing-slash and bare forms. 5 new tests cover bare-segment hits (`/About`, `/Strategy`, `/Policy`) + 2 over-match guards (`/aboutface`, `/strategist`). P3 — LEVEL_RANK doc comment in list-feed-digest.ts:73-76 said "low → critical preserved because low+2=critical". The arithmetic is wrong (low=1, +2=3=high, not critical) and contradicted both the test assertions (low → critical caps to high) and the actual implementation. Comment rewritten to enumerate cap behavior per keyword level and correctly describe the bounded loss (low → critical capped at high, logged on every cap-fire). Behavior unchanged. 284/284 tests pass; importance-score-parity preserved.
…w fix on #3419) Reviewer caught a third stale comment site missed in the prior P3 fix (commit f40b403). The top-level LEVEL_RANK rationale was correct; the inline comment inside enrichWithAiCache (line 525) and the header of tests/news-classify-cache-tier-cap.test.mts both still asserted "low+2 = critical, low → critical preserved." The arithmetic is wrong (low=1, +2=3=high) and contradicts the test that asserts the opposite. Rewritten to match LEVEL_RANK doc + actual implementation. Plan doc test-scenarios block (line 269) also corrected for the same reason — that one is local-only since docs/plans/ is gitignored, but keeping it consistent for future-me. Behavior unchanged. 106/106 tests pass.
4 tasks
koala73
added a commit
that referenced
this pull request
Apr 26, 2026
…ns container (#3423) Production digest-notifications cron has been crashing on startup since PR #3419 deployed: Error [ERR_MODULE_NOT_FOUND]: Cannot find module '/app/shared/url-classifier.js' imported from /app/shared/brief-filter.js PR #3419 added `import { isInstitutionalStaticPage } from './url-classifier.js'` to shared/brief-filter.js. The file IS in the repo at shared/url-classifier.js and the Vercel side picks it up automatically (whole-tree deploy). But Dockerfile.digest-notifications uses a deliberately tight per-file COPY list for shared/* (line 53-56 comment: "Keep this COPY list tight — adding unrelated shared/* files expands the rebuild watch surface"). The url-classifier.js COPY directive was never added in PR #3419, so the Railway container ships brief-filter.js without the new dependency. Other Railway services (Dockerfile.relay, Dockerfile.seed-bundle-*) use whole-dir `COPY shared/ ./shared/` and were not affected. Fix: one-line `COPY shared/url-classifier.js ./shared/` adjacent to its consumer. Comment cites PR #3419 + the recurring shared-dir-mirror / tight-COPY-list pattern (memory: feedback_validation_docker_ship_full_ scripts_dir.md, worldmonitor-scripts-package-json-install-scope). No test changes — this is purely Docker packaging. The classifier itself is covered by tests/url-classifier.test.mjs (24 cases) which run against the source tree, not the container. Operator runbook: 1. Merge → Railway rebuilds digest-notifications service. 2. Verify container starts: Railway logs show "Starting Container" followed by digest-cron init lines, NOT ERR_MODULE_NOT_FOUND. 3. Verify next scheduled cron tick succeeds (digest sends).
koala73
added a commit
that referenced
this pull request
Apr 26, 2026
…e mode (#3422) * fix(brief-ingest): READ-time freshness + direct-RSS migration + U6 age mode Three coupled gap closures discovered after PR #3419 merged. Brief 2026-04-26-0802 still surfaced two months-old Pentagon items DESPITE PR #3417's `when:1d` ingest gate working perfectly (verified live: the gated query returns zero Pentagon items in the last 24h). The cause was post-deploy residue, not the gate. See: skill: ingest-gate-tightening-leaves-residue-in-read-path Fixes (all on the same PR because they share the publishedAt persistence precondition): 1. READ-time freshness floor in buildDigest (seed-digest-notifications.mjs). Drops story:track:v1 rows whose source publishedAt is older than DIGEST_READ_MAX_AGE_HOURS (default 48h). Closes the residue window: when any future ingest-side gate is tightened, pre-deploy entries stop shipping in briefs even before they TTL-expire from the accumulator. Rows missing publishedAt (legacy pre-this-PR entries) fall through — back-compat. Operator kill switch: DIGEST_READ_MAX_AGE_HOURS=999. 2. Persist publishedAt in story:track:v1 HSET (list-feed-digest.ts). Adds 'publishedAt', String(item.publishedAt) to buildStoryTrackHsetFields. Required for #1 AND for U6's age-mode (#3) to actually find the field. Pre-existing rows pick it up on their next mention. Test: tests/news-story-track-description-persistence.test.mts asserts the field is emitted as a numeric string that round-trips through parseInt. 3. U6 audit gains --mode=age|url|both (audit-static-page-contamination.mjs). Original URL-pattern classifier is BLIND to Google-News-routed entries (track.link is news.google.com/rss/articles/CBMi opaque redirect). --mode=age matches by track.publishedAt > max-age-hours, the right primary signal for evicting post-deploy residue. Per-reason rollup surfaces url vs age hits. Operator runbook included in the script header. Default --mode=url is back-compat. 4. Feed swap: 3 of 5 gov sources to direct first-party RSS (_feeds.ts). Verified 2026-04-26 via live curl + end-to-end parseRssXml smoke test: - Pentagon -> https://www.war.gov/DesktopModules/ArticleCS/RSS.ashx?ContentType=1&Site=945 - White House (briefings) -> /briefings-statements/feed/ - White House Actions -> /presidential-actions/feed/ (NEW entry) Direct RSS means the publisher's pubDate is authoritative — no re-indexing of months-old PDFs creating fresh-looking Google entries. State Dept, Treasury, DOJ retain gn(...) with an explanatory comment; no working public RSS at any verified path, Federal Register fallback bot-blocked. The READ-time freshness floor (#1) mitigates the residue gap for those. 5. source-tiers.json: White House Actions = 1 (with scripts/shared mirror). New feed gets the same tier-1 source-boost as the existing White House entry; without it, Actions items would default to tier-4 with no boost. 250/250 tests pass; importance-score-parity preserved. Operator runbook for the immediate post-deploy residue: 1. Wait ~1h for cron tick to start writing publishedAt on existing entries via HSET re-mention. 2. Dry run: node scripts/audit-static-page-contamination.mjs --mode=age 3. Inspect output, confirm matched titles look stale. 4. Apply: --mode=age --apply * fix: address review findings — window-aware floor + residue audit mode (P1×2 + P2/P3) 5 fixes responding to review: P1 (reviewer): READ-time floor + audit --mode=age both fall through on missing publishedAt — the legacy state of EVERY pre-PR-3422 row. So the actual residue this PR was meant to evict (Pentagon items in brief 2026-04-26-0802) wasn't catchable by either gate. Add --mode=residue that matches rows with missing/unparseable publishedAt — the explicit opt-in cleanup signal for the post-deploy one-shot. Operator runbook updated: wait at least 1 cron cycle (so still-active rows have publishedAt re-mentioned in via HSET), then run --mode=residue --apply. P1 (self-review): READ-time floor was hardcoded 48h, breaking weekly users (their 7d window stories all dropped). Replaced with readTimeAgeCutoffMs(windowStartMs) = windowStart - 24h buffer. Daily user (24h window) -> 48h cutoff (matches old behavior); weekly user (7d window) -> 8d cutoff (the broken case). DIGEST_READ_MAX_AGE_HOURS env var removed — there's no scenario where an operator wants to drop items younger than the rule's intended window. P2: Defensive cast in buildStoryTrackHsetFields. Number.isFinite guard on item.publishedAt prevents the literal "undefined"/"NaN" string from ever being persisted; degraded input writes '' which the read-side parseInt treats as legacy back-compat (fall through, not mis-classify as a stale-row). P2: Extract shouldDropTrackByAge + readTimeAgeCutoffMs into scripts/lib/digest-orchestration-helpers.mjs (the existing module for this purpose). 6 new unit tests cover the predicate matrix (within/at-cutoff/before/missing/unparseable/zero/null-track) plus 2 integration cases that reproduce the exact failure modes: - weekly user with 5d-old story is KEPT (window-aware vs naive 48h) - 4-month-old residue (Jan 2026 Pentagon PDF) is DROPPED for daily user P3: Audit parseArgs now rejects unknown args (--mode age space-not-equals) with exit-2 + clear error message. classifyTrack refactored to a pure function (mode + maxAgeMs as options) so tests can exercise it without the script's module-load argv side effects. main() invocation gated behind import.meta.url === file://process.argv[1] check (standard ESM idiom). 10 new unit tests cover the url/age/residue/both modes + parseArgs flag handling. 316/316 tests pass; importance-score-parity preserved. * fix(audit): residue mode safety gate + EOF whitespace (review fixes on #3422) P2 (reviewer): --mode=residue --apply was too broad. Classifier matched ANY row with missing publishedAt — but every pre-PR-3422 row lacks the field, so the recommended one-shot cleanup could delete legitimate recent stories that simply hadn't been re-mentioned in the first post-deploy cron cycle. Fix: residue mode now requires BOTH conditions: 1. publishedAt missing/unparseable (legacy or anomalous), AND 2. lastSeen >= residueMinStaleMs ago (default 24h). Active stories get HSET-touched on every re-mention, so a row that hasn't been touched in 24h is genuinely abandoned — the new ingest gate (when:1d) is correctly excluding it. Fresh-lastSeen rows are PROTECTED from deletion. New flag: --residue-min-stale-hours=N (default 24, positive-int only). Operator runbook updated: 1. Wait >=24h post-deploy. 2. Dry run: node scripts/audit-static-page-contamination.mjs --mode=residue 3. Inspect output, confirm rows are stale + look like residue. 4. Apply: --mode=residue --apply Defensive fall-throughs: - Missing lastSeen: treat as ancient (errs toward eviction). Acceptable in this opt-in destructive mode; rows without lastSeen are anomalous. - publishedAt present (any value): NOT residue (caught by --mode=age). Tests: - 8 residue-mode cases incl. the explicit P2 fix ("fresh lastSeen must protect the row"), boundary at exactly the threshold, missing-lastSeen treated as ancient. - 4 parseArgs cases for the new --residue-min-stale-hours flag. P3 (reviewer): tests/digest-orchestration-helpers.test.mjs had a trailing blank line at EOF. Stripped. 323/323 tests pass; importance-score-parity preserved. * fix(audit): align residue staleness gate with max digest window (P2 round-2) Reviewer flagged that --mode=residue --apply with the 24h staleness default would delete legitimate weekly-user stories. A pre-PR-3422 row that lacks publishedAt with lastSeen 2-7d ago is still ship-able for a weekly digest (whose readTimeAgeCutoffMs in digest-orchestration-helpers.mjs is windowStart - 24h = 8d ago) — but the 24h gate would mark it as residue and delete it. Fix: default --residue-min-stale-hours from 24 to 192 (= 7d max digest window + 24h buffer). Aligns with the readTimeAgeCutoffMs formula so residue mode can never delete a row still legitimately ship-able for ANY user. New explicit regression test: "SAFETY: does NOT match weekly-user 5d-old story" — pre-fix would have failed (deleted the row); now PROTECTED. Operators with confidence the fleet is daily-only can drop to faster cleanup via --residue-min-stale-hours=48 (documented escape hatch in the script header runbook + test case). The other safety guards remain: - publishedAt present → not residue (caught by --mode=age, not residue). - lastSeen fresh (< gate) → protected. - missing lastSeen → ancient (errs toward eviction in opt-in path). 325/325 tests pass; importance-score-parity preserved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why this matters
Follow-up to #3417 (merged). Closes the remaining four units of
docs/plans/2026-04-26-001-fix-brief-static-page-contamination-plan.md:+2 tierupgrade cap + atomic 3-sitev3 → v4prefix bumpUser impact: even if a regression in the feed registry, RSS parser, or LLM cache lets a static page through, the brief surface stays clean. Plus the editorial-clutter case (CBS shipping both Midwest-tornado and Oklahoma-tornado in the same brief) is fixed via the source-topic cap.
Commits
51dc927ee+2tier cap + atomic 3-site prefix bump (_shared.ts,list-feed-digest.ts,ais-relay.cjs) + 16 new testsc3a8b6392scripts/shared/mirror) + ops audit script + test updatesNotable design choices
+2 tiers, not+1. Initial draft was+1but reviewer flagged that as too aggressive —keyword=medium + LLM=critical(real "Markets crash"-style upgrades) would have regressed.+2blocks the contamination class (info → high/criticalcapped at medium) while preservingmedium → criticalandlow → high. The tradeoff (low → criticalcapped athigh) is logged on every cap-fire so the loss is measurable, not silent.list-feed-digest.tsno longer has its ownclassify:sebuf:v3:literal — it usesbuildClassifyCacheKeyfrom_shared.ts. Future bumps only require touching_shared.ts+ the relay's.cjs(which can't import from.ts). Static-analysis test grep-asserts zero non-canonical literals across.ts/.mjs/.cjs/.js/.json.maxPerSourceTopic: Infinitysince their intent is unrelated (severity gate, max-stories cap, ranking). Production callers (composeBriefFromDigestStories) get the default-2 behavior automatically..gov/.mil/.inthost AND a curated path prefix. A bare.govURL or a bare/About/path on a.comdoesn't trigger.--applyruns post-deploy. Script lands here so reviewers can validate the classifier matches expectations; actual--applydeletion happens as a one-shot ops invocation after deploy + 48h soak.filterTopStoriesisolates the brief-surface fix from those side-effect channels.Tests
tests/news-classify-cache-tier-cap.test.mtsinfo → high/criticalclamps tomedium), legitimate upgrades preserved, malformed LLM levels,keyword=criticaledgetests/news-classify-cache-prefix-audit.test.mjstests/url-classifier.test.mjs/News/paths on same hosts classify false; non-institutional hosts classify false; defensive on bad inputtests/brief-filter.test.mjs(+9)maxPerSourceTopicoverridetests/edge-functions.test.mjs(+1)url-classifier.jsnow in explicit-mirrored-files settests/brief-from-digest-stories.test.mjs(3 fixture updates)278/278 tests pass across the touched + parity-coupled surface (rebased onto current
main).tests/importance-score-parity.test.mjscontinues to pass — formula untouched.Operational notes
Rollback:
keywordRank + 2→keywordRank + 99(no cap fires).maxPerSourceTopic: InfinityfromcomposeBriefFromDigestStories(env wiring is a follow-up if needed).Cost-spike monitoring:
classify-event.ts(analyzer RPC) andais-relay.cjs. Warm-up is driven by analyzer + relay traffic, not digest volume — practically slower than the 24h TTL implies. Acceptable: digest behavior degrades gracefully to keyword-only classification during the warm window (no LLM cost spike from the digest path itself).Deploy ordering:
.cjschange won't take effect until the relay container restarts.classify:sebuf:v4:appears in cache key lines (proves the relay picked up the bump).node scripts/audit-static-page-contamination.mjs. Capture the output in this PR thread.node scripts/audit-static-page-contamination.mjs --applyto evict residual contamination. Document remaining count.Test plan
[classify] LLM upgrade capped:log lines appear ONLY for keyword=info upgrades (not legitimate medium→critical cases)classify:sebuf:v4:keys present in Upstash; v3 keys still present but TTL-expiring--applyis rundefense.gov/About/,defense.gov/Strategy/, etc. URLs; no >2-story bursts from same(source, category)pair