fix: Digest pipeline review actions — feed-date, CI guards, AI-queue identity#3872
Merged
Conversation
…rkflow PR CI gains a docker-build smoke for Dockerfile.digest-notifications that catches cross-directory-import + native-dep breakage the static BFS test in tests/dockerfile-digest-notifications-imports.test.mjs cannot see. Gated on path changes under scripts/, shared/, server/_shared/, api/, the Dockerfile itself, or the root package manifest. Feed-registry validation moves OFF pull_request CI (SSRF surface — a hostile PR could rewrite src/config/feeds.ts to make runners hit arbitrary URLs) and INTO a new feed-validation.yml that runs on push-to-main, every-6h schedule, and workflow_dispatch. The new test:feeds:ci script passes --ci to scripts/validate-rss-feeds.mjs which enforces https-only, allowlist-membership (via api/_rss-allowed-domain-match.js, the new extracted predicate the Edge rss-proxy.js also consumes), and per-hop redirect re-checks. Plan: docs/plans/2026-05-23-001-fix-digest-pipeline-review-actions-plan.md
…ints Replaces the hand-maintained ENTRY_POINTS array in tests/scripts-railway-nixpacks-no-escape-import.test.mts (4 entries, already known to drift per PR #3836 retrospective) with a derived list from scripts/railway-services.json. The Dockerfile.digest-notifications BFS test reads the same registry filtered to deployMode === "dockerfile". A new coverage test (tests/railway-services-registry-coverage.test.mts) audits every Dockerfile.* CMD line and every runbook "Start command:" entry against the registry — if a deployment artifact references a script that isn't registered, the test fails naming the file. Self- fixtures pin both regex shapes so a future simplification can't silently stop matching. The registry surfaces 17 nixpacks services (up from 4) and 4 Dockerfile- deployed services. The runbook header now points new-service authors at the registry as the first stop. Plan: docs/plans/2026-05-23-001-fix-digest-pipeline-review-actions-plan.md
…ateMs
The legacy parseFeedDateOrNow silently substituted Date.now() for any
missing/invalid feed pubDate, which produced false freshness across every
ranking and recency gate. The fix replaces it with parseFeedDate (returns
{ date, missing }) and a new effectivePubDateMs helper that returns 0 for
items flagged pubDateMissing — so they sort last in newest-first
comparators and fail any positive-duration recency gate
(Date.now() - 0 = huge), while keeping the synthesized pubDate in place
for display consumers.
Routed through the helper:
- rss.ts AI-candidate top-N selection + min-heap insert (lines 296, 352-369)
- daily-market-brief.ts headline-pool sort (line 217)
- breaking-news-alerts.ts isRecent recency gate + best-alert ranking (lines 145, 260)
- data-loader.ts staleItems sort, hero/digest sorts, time-range filter (6 sites)
- panel-layout.ts time-range filter (caught by the new guardrail audit)
- velocity.ts trend-midpoint partition (lines 58-59)
- analysis-core.ts cluster sort + WINDOW_MS recency gate (lines 292, 440)
Cache prefix bumped feed: → feed:v2: at rss.ts:23, 44-65 because pre-v2
serialized items lack pubDateMissing; old entries are left to TTL out.
Two new tests:
- tests/feed-date.test.mts — parser shapes + helper ranking behavior
- tests/feed-date-ranking-uses-effective.test.mts — static audit that
walks src/services/ + src/app/ and fails if any .pubDate.getTime()
inside a ranking/recency call site bypasses the helper, with a
documented allow-list for metadata/identity uses and self-fixtures
pinning both sort-comparator and recency-gate regex shapes
Plan: docs/plans/2026-05-23-001-fix-digest-pipeline-review-actions-plan.md
Previously canQueueAiClassification(title) collapsed every article sharing
a wire headline into one dedupe slot, so only the first got an AI threat
upgrade and the rest stayed at keyword confidence for the 30-minute
window. The new signature canQueueAiClassification({ link, title }) keys
on the normalized link when present (lowercased host, fragments dropped,
utm_*/fbclid/gclid stripped) and falls back to title when link is empty
or malformed.
Behavioral surface is narrower than it might look: the caller in rss.ts
only invokes the queue for items where threat.source === 'keyword' —
items already keyword-classified that are eligible for an AI confidence
upgrade. A dropped enqueue does NOT drop the article from the feed; it
just leaves its threat label at keyword confidence.
Also adds __resetAiClassifyQueueForTests() following the project's
__…ForTests convention (insights-loader.ts:104) so the new test file
can drive the module-scope dedupe map across scenarios deterministically.
Import switched from '@/config' barrel to '@/config/variant' leaf to
avoid pulling @/utils/proxy.ts (and its import.meta.env.DEV usage) into
the node:test runner.
Plan: docs/plans/2026-05-23-001-fix-digest-pipeline-review-actions-plan.md
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Greptile SummaryThis PR ships four validated fixes from an external pipeline audit: CI hardening with a Docker smoke build and a SSRF-safe scheduled feed validator, a Railway service registry replacing a hand-maintained entry-point list, a feed-date freshness overhaul that replaces the silent
Confidence Score: 4/5Safe to merge; all runtime logic is correct and the changes close real production issues with the false-freshness and dedup-collapse bugs. The four units of change are well-scoped and thoroughly tested. The feed-date helper correctly returns 0 for missing-date items across all seven call sites, the AI-queue identity change is logically sound with good test coverage, and the CI additions close real gaps. Three inline comments contain inaccuracies but none affect runtime behaviour. src/services/rss.ts (migration fallback comment), src/services/breaking-news-alerts.ts (isRecent comment), tests/ai-classify-queue.test.mts (assertion message) Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["RSS Feed Item (raw)"] --> B["parseFeedDate(pubDateStr)"]
B --> C{timestamp parseable?}
C -- yes --> D["{ date: parsed, missing: false }"]
C -- no / missing --> E["{ date: new Date(), missing: true }"]
D & E --> F["NewsItem { pubDate, pubDateMissing }"]
F --> G["effectivePubDateMs(item)"]
G --> H{pubDateMissing?}
H -- true --> I["return 0 (sorts last, fails recency gates)"]
H -- false --> J["return pubDate.getTime()"]
J --> K["Ranking / Recency Consumers"]
I --> K
K --> K1["rss.ts: AI candidate top-N + heap insert"]
K --> K2["breaking-news-alerts.ts: isRecent gate"]
K --> K3["data-loader.ts: 6 sort sites + time-range filter"]
K --> K4["panel-layout.ts, velocity.ts, analysis-core.ts"]
F --> L["canQueueAiClassification({ link, title })"]
L --> M{link present & valid URL?}
M -- yes --> N["normalizeLink: lowercase host, strip trackers, drop fragment"]
M -- no / malformed --> O["titleKey(title) fallback"]
N --> P["link: dedupe slot"]
O --> Q["title: dedupe slot"]
Reviews (1): Last reviewed commit: "fix(rss): use link-keyed identity in AI ..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+143
to
+150
|
|
||
| // ─── Gate checks ─────────────────────────────────────────────────────────── | ||
|
|
||
| function isRecent(pubDate: Date): boolean { | ||
| return pubDate.getTime() >= (Date.now() - RECENCY_GATE_MS); | ||
| function isRecent(item: { pubDate: Date; pubDateMissing?: boolean }): boolean { | ||
| // Routes through effectivePubDateMs so items with pubDateMissing get 0 | ||
| // and fail the recency gate (Date.now() - 0 = a huge positive number, | ||
| // always >= RECENCY_GATE_MS). Without the helper, the synthesized | ||
| // pubDate would pass this gate and fire false-fresh alerts. |
Contributor
There was a problem hiding this comment.
The comment says "Date.now() - 0 = a huge positive number, always >= RECENCY_GATE_MS" but the actual expression being evaluated is 0 >= (Date.now() - RECENCY_GATE_MS). For a missing-date item effectivePubDateMs returns 0, and 0 >= ~1.7 trillion is false — correctly failing the gate. The Date.now() - 0 framing describes a subtraction that doesn't appear in the code and implies the opposite comparison direction, which can confuse someone trying to reason about the gate semantics.
Comment on lines
+102
to
+109
| it('malformed link falls back to title identity without crashing', () => { | ||
| assert.equal(canQueueAiClassification({ link: 'not a url', title: 'Same title' }), true); | ||
| assert.equal( | ||
| canQueueAiClassification({ link: 'also not a url', title: 'Same title' }), | ||
| false, | ||
| 'Malformed links short-circuit on the raw string; identical raw strings dedupe, different ones do not', | ||
| ); | ||
| }); |
Contributor
There was a problem hiding this comment.
The assertion message says "Malformed links short-circuit on the raw string; identical raw strings dedupe, different ones do not." In reality, normalizeLink returns null for both 'not a url' and 'also not a url', so toAiKey falls back to titleKey(identity.title) — the second call dedupes on the shared title 'Same title', not on any form of the raw link string. A developer reading this message could incorrectly infer that { link: 'not a url', title: 'A' } and { link: 'not a url', title: 'B' } would also dedupe (same raw string), when they would in fact produce distinct dedupe slots.
Comment on lines
64
to
75
| const scoped = await readPersistentFeed(scopedKey); | ||
| if (scoped) return scoped; | ||
|
|
||
| // Migration fallback: older builds stored feeds as `feed:<feedName>` without language scope. | ||
| // Only use this for English to avoid mixing cached content across locales. | ||
| // Migration fallback: older builds stored feeds as `feed:v2:<feedName>` without | ||
| // language scope. Only use this for English to avoid mixing cached content | ||
| // across locales. Pre-v2 `feed:<feedScope>` and `feed:<feedName>` entries are | ||
| // deliberately NOT consulted — they predate the pubDateMissing schema and | ||
| // would re-introduce false-freshness for cached items. | ||
| const { feedName, lang } = parseFeedScope(feedScope); | ||
| if (lang !== 'en') return null; | ||
| return readPersistentFeed(`feed:${feedName}`); | ||
| return readPersistentFeed(`${CACHE_PREFIX}${feedName}`); | ||
| } |
Contributor
There was a problem hiding this comment.
The comment says "older builds stored feeds as feed:v2:<feedName> without language scope," but feed:v2: is being introduced for the first time in this PR alongside full language scoping. No prior codebase version ever wrote a feed:v2:<feedName> (language-unscoped) entry. The fallback will always return null in production, making it dead code. The old feed:<feedName> migration path was intentionally dropped, so this is harmless — but the comment creates false archaeological history that can mislead future readers investigating cache-miss behaviour.
…bDateMs Code-review caught a real regex bypass: src/app/country-intel.ts:297 ran `new Date(b.pubDate).getTime() - new Date(a.pubDate).getTime()` over NewsItem[] from ctx.allNews — a freshness-ranking comparator that the U3 guardrail regex /\.pubDate\s*\.getTime\(\)/ did not match. The country-brief panel's news ordering would have surfaced missing-date items at the top despite the U3 fix. Three changes: 1. src/app/country-intel.ts:297 now routes through effectivePubDateMs. 2. PUBDATE_GETTIME_RE broadened to also match `item.pubDate?.getTime()` (optional chain, idiomatic in TS 4+) and `new Date(item.pubDate). getTime()` (the bypass the country-intel violation used). Self-fixtures pin all three shapes plus a negative-control fixture confirming the regex does NOT catch `new Date(item.endDate).getTime()`. Paren-wrapped, aliased, and destructured reads remain residual risk — documented in the regex docblock. 3. effectivePubDateMs now guards NaN/Infinity on every input branch (Date with Invalid value, NaN/Infinity number, unparseable string). Sort comparators on NaN have unspecified behavior per the JS spec; the previous numeric branch returned the value verbatim. Tests cover NaN, ±Infinity, and Invalid Date. Also minor cleanups from review: - Inverted JSDoc on isRecent clarified (`0 >= (large positive) = false`). - Misleading migration-fallback comment in rss.ts (claimed "older builds stored feeds as feed:v2:") corrected to describe the actual scope: language-scope migration carried forward from the pre-v2 era. Plan: docs/plans/2026-05-23-001-fix-digest-pipeline-review-actions-plan.md
F2 — Cache prefix bump for happy-all-items: data-loader.ts:3200 HAPPY_ITEMS_CACHE_KEY bumped 'happy-all-items' → 'happy-all-items:v2' to match the feed: → feed:v2: invalidation. Pre-v2 entries serialize NewsItem without pubDateMissing; under the new helper they would fraudulently claim freshness inside the 24h gate window. Bumping forces refetch; pre-v2 entries are left to TTL out naturally. F3 — Velocity excludes missing-date items from trend math: velocity.ts pre-filters items.filter(i => !i.pubDateMissing) before the midpoint partition. Without this, clustering.ts:116 (allow-listed raw aggregation) would compute firstSeen/lastUpdated from synthesized stamps, then velocity would treat every missing-date item as 'older' via effectivePubDateMs=0, flipping cluster trend to 'falling' for any cluster with several missing-date items. F4/F5 — CI workflow timeouts: feed-validation.yml job gains timeout-minutes: 15 (cap on hung npm ci or stalled feed registry). digest-image job in test.yml gains timeout-minutes: 20 (cap on docker build infra flakes). F6 — Per-hop fetch timeout in --ci validator: validate-rss-feeds.mjs --ci redirect handler now creates a fresh AbortController + setTimeout per hop instead of one budget across all hops. "Timeout (15s)" in the report now means a single 15s network call, not 17s+ chained. F7 — MAX_HOPS off-by-one in --ci validator: Loop bound changed from `hop <= MAX_HOPS` to `hop < MAX_HOPS` so the documented "max 3 hops" intent matches the actual iteration count. Final-iteration redirect now throws 'Too many redirects' consistently. F8 — Dublin Core <dc:date> fallback in RSS parser: rss.ts:267-274 adds dc:date as a final fallback for the RSS branch (after pubDate). ArXiv RSS feeds — "ArXiv AI" and "ArXiv ML" in the feed registry — ship dc:date with no <pubDate>; without this fallback every ArXiv item would be marked pubDateMissing=true and demoted in every freshness ranking. Prior precedent: PR #3417 dc:date strict-drop incident. Observation 1 — Asymmetric tie-break in breaking-news-alerts.ts: Captures a bestEffectiveMs local so the tie-break comparison is symmetric (helper on both sides). Today isRecent excludes missing- date items upstream so the comparison is safe; the local makes the guarantee defense-in-depth against a future refactor that removes the upstream gate. Observation 2 — Document NaN/Infinity exclusion in time-range filters: data-loader.ts filterItemsByTimeRange and panel-layout.ts time-range filter previously wrapped raw pubDate.getTime() with Number.isFinite and fell through to `true` for non-finite, INCLUDING corrupt-stamp items in narrow windows. The helper now sanitizes NaN/Infinity to 0 uniformly with pubDateMissing — corrupt-stamp items are excluded. This is intentional (untrustworthy timestamps shouldn't claim freshness) but undocumented in U3; comments at both sites now call out the behavior shift. Plan: docs/plans/2026-05-23-001-fix-digest-pipeline-review-actions-plan.md
4 tasks
koala73
added a commit
that referenced
this pull request
May 24, 2026
…hird-party state) (#3885) * fix(feeds): unblock feed-validation workflow The Feed Validation workflow (.github/workflows/feed-validation.yml) has been failing 100% of runs since it landed in PR #3872 — every push to main + every scheduled 6h run. Five root causes, all addressed here: 1. fast-xml-parser default entity-expansion limit was tripping on legitimate large feeds (Guardian, Fox, Axios, CISA, WHO, MIT, Defense One, Folha, El País, iefimerida, GitHub Trending, Dev.to, Oryx OSINT, …). We only read date strings from the parsed doc, so processEntities:false is safe and recovers all 17 false-positive DEAD rows. 2. 10 hosts referenced from src/config/feeds.ts were absent from the 5-file allowlist mirror set (shared/rss-allowed-domains.{json,cjs}, scripts/shared/rss-allowed-domains.json, api/_rss-allowed-domains.js, vite.config.ts:RSS_PROXY_ALLOWED_DOMAINS). Added: abcnews.go.com + abcnews.com (feeds.abcnews.com → abcnews.go.com → abcnews.com two-hop chain), www.corriere.it, www.rt.com, www.alarabiya.net, tuoitrenews.vn, www.yonhapnewstv.co.kr, www.chosun.com, rss.libsyn.com, feeds.megaphone.fm, rss.art19.com. The same allowlist gates the prod Edge rss-proxy, so this also silently restores access to these feeds for live users. 3. BBC Persian was declared as plaintext http://, rejected by the --ci https-only guard. Updated to the canonical https://feeds.bbci.co.uk/persian/rss.xml (server-side mirror already had this). 4. Tom's Hardware /feeds/all redirects to http://… on the same host, tripping the per-hop https guard. The canonical https path is /feeds.xml — switched both client and server mirrors. 5. Validator was hard-failing on any STALE-or-DEAD row, which made the workflow noise floor unbearable: 8 stale + 32 dead = 40 reasons to be red, of which only 10 were actionable. Split the exit policy: HARD-FAIL on config/SSRF-guard drift (allowlist miss, plaintext URL, redirect loop) so future drift is loud, SOFT-FAIL (exit 0 with warn) on third-party 4xx/timeouts/STALE so a feed disappearing upstream doesn't page anyone. Promoting third-party failures to hard-fail can wait for a registry grooming PR. Also bumps the scheduled cadence from every-6h to daily-00:00-UTC. 4× the discovery rate added zero value — feed outages don't change faster than once-a-day, and 542 feeds × 4 runs/day was wasted runner-minutes and third-party fetch volume. Local validator result (after the fix): Summary: 512 OK, 10 stale, 6 dead, 13 empty, 1 skipped Exit: 0 (no config drift). 6 remaining DEAD are all genuine third-party state (Brasil Paralelo 404, EIA Reports 404 [duplicate entry], News24 403, Tuoi Tre + Al Arabiya unreachable from this network) — candidates for a future registry-cleanup PR. Test coverage: tests/feeds-client-server-parity.test.mjs, tests/feed-resolution.test.mts, tests/feeds-time-gate.test.mts — all green. Full test:data suite — green. * fix(feeds): address Greptile P1 + P2 on validate-rss-feeds.mjs P1: FAIL message claimed "4 allowlist mirrors" but the codebase enforces 5 (scripts/shared/rss-allowed-domains.json was missing from the list). A developer following the message would skip that mirror and hit a puzzling secondary failure from the `test:data` scripts/shared parity check. Listed all 5 mirrors. P2: The isConfigDrift predicate was prefix-matching against literal copies of the error message strings thrown by assertCiAllowed and fetchFeed. A future reword at either throwing site (e.g. dropping the "(--ci)" annotation, rewording the redirect error) would silently reclassify config drift as third-party rot, demoting a hard fail to a soft warning. Centralised the sentinel strings as a frozen CONFIG_DRIFT_REASONS object that both the throwing sites and the classifier consume — rename a reason and BOTH consumers move in lockstep. INVALID_URL is also now properly classified as config drift (was previously falling through to soft-fail despite being an actor- fixable bug in feeds.ts). Tested: - End-to-end run: 512 OK / 9 stale / 7 dead / 13 empty, EXIT=0 - Classifier unit test: all 8 representative cases correct (4 config-drift reasons → true, 4 third-party reasons → false) * fix(feeds): allowlist idp.nature.com (Nature SSO redirect hop) Nature publishes a session/IP-conditional redirect chain on feeds.nature.com — on some networks the request lands directly at www.nature.com/nature.rss (both already in the allowlist), but on others (apparently GitHub Actions runner IPs) Nature inserts an idp.nature.com SSO/identity-provider hop: feeds.nature.com → idp.nature.com → www.nature.com/nature.rss The validator's per-hop allowlist re-check fails on idp.nature.com. Adding it to all 4 hand-maintained mirrors (+ the .cjs that auto-syncs via require) closes the gap. Same shape as the abcnews.go.com fix on the original PR — the lesson is that allowlist audits done from a developer laptop can miss intermediate redirect hops that only appear under different network egress paths. Documented in worldmonitor-architecture-gotchas/.../multi-hop-redirect- chain-needs-every-host-in-allowlist.md (skill added in PR #3885 first round). Also addresses the reviewer's second P1 finding (Invalid URL being soft-fail instead of hard-fail): already fixed in 8a34058 — the reviewer's audit was against the pre-Greptile commit cbb80e1. INVALID_URL is now in CONFIG_DRIFT_REASONS and isConfigDrift hard-fails malformed registry entries.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
External audit (
deep-research-report (7).md) flagged 11 candidate issues in the WM news/digest pipeline. Verification rejected 7 (stale claims, file-name domain confusion, or fixes already shipped) and validated 4 actionable items. This PR ships all four, plus one additional production violation surfaced during execution.Plan:
docs/plans/2026-05-23-001-fix-digest-pipeline-review-actions-plan.md(Codex-approved after 4 rounds of review).What changed (one commit per unit)
U1 — CI hardening (
27f9ffebf)digest-imagejob in.github/workflows/test.ymlrunsdocker build -f Dockerfile.digest-notificationson every PR touchingscripts/,shared/,server/_shared/,api/,Dockerfile.digest-notifications, or the root package manifest. Catches the native-dep + cross-directory-import breakage class that the static BFS test cannot see..github/workflows/feed-validation.ymlrunsnpm run test:feeds:cionpushto main, every 6h cron, andworkflow_dispatch— but NOTpull_request, closing the SSRF surface where a hostile PR could rewritesrc/config/feeds.tsto make runners hit arbitrary URLs.scripts/validate-rss-feeds.mjsgains a--ciflag enforcing (a) https-only URLs, (b) hostname allowlist via the extracted predicate, (c) per-hop redirect re-check.api/_rss-allowed-domain-match.js(NOTshared/— Edge constraint:api/*.jscannot import from../shared). Bothapi/rss-proxy.jsand the validator consume it.U2 — Railway service registry (
e2099d7a3)scripts/railway-services.jsonis the single source of truth for every Railway-deployed script. Surfaces 17 nixpacks services (the hand-maintainedENTRY_POINTSarray previously covered 4) and 4 Dockerfile-deployed services.tests/scripts-railway-nixpacks-no-escape-import.test.mtsandtests/dockerfile-digest-notifications-imports.test.mjsnow derive their entry lists from the registry, filtered bydeployMode.tests/railway-services-registry-coverage.test.mtsfails if anyDockerfile.*CMD ["node", …]line or runbook "Start command:" entry references a script not in the registry. Self-fixtures pin both regex shapes per the static-grep audit pattern from thetest-ci-gotchasskill.docs/railway-seed-consolidation-runbook.md) header now points new-service authors at the registry.U3 — Feed-date freshness (
9b79fe4c9)parseFeedDateOrNow(which silently substitutedDate.now()for missing/invalid input) is replaced byparseFeedDatereturning{ date, missing }and a neweffectivePubDateMs(item)helper that returns0forpubDateMissingitems. Display consumers still see a valid Date; ranking/recency consumers exclude missing-date items from freshness.rss.tsAI-candidate top-N selection + min-heap insertdaily-market-brief.tsheadline-pool sortbreaking-news-alerts.tsisRecentrecency gate + best-alert rankingdata-loader.ts6 freshness sort sites + time-range filterpanel-layout.ts:1681time-range filter (audit-caught — NOT in the original plan enumeration)velocity.tstrend-midpoint partitionanalysis-core.tscluster sort +WINDOW_MSrecency gatefeed:→feed:v2:because pre-v2 serialized items lackpubDateMissing.tests/feed-date.test.mtscovers parser shapes + helper ranking behavior.tests/feed-date-ranking-uses-effective.test.mtswalkssrc/services/+src/app/and fails if any.pubDate.getTime()inside a ranking/recency call site bypasses the helper. Documented allow-list for metadata/identity uses. Self-fixtures pin both sort-comparator and recency-gate regex shapes.U4 — AI-classify queue identity (
eb141bb5e)canQueueAiClassification(title)→canQueueAiClassification({ link, title }). Identity prefers the normalized link (lowercased host, fragments dropped,utm_*/fbclid/gclidstripped); falls back to title when link is empty or malformed. Stops collapsing distinct articles sharing a wire headline into one dedupe slot.__resetAiClassifyQueueForTests()export following the__…ForTestsconvention frominsights-loader.ts:104so the test file'sbeforeEachcan isolate module state.@/configbarrel to@/config/variantleaf to avoid pullingimport.meta.env.DEVthrough the proxy.ts chain undernode:test.What did NOT change
Audit claims rejected during verification (kept for future-reader visibility):
insights-loader.ts15-min stale gate — already fixed to 60min via PR fix(insights): on-demand bootstrap-key refetch so mobile recovers from hydration miss #3827 (fetchServerInsightson-demand fallback). Report's headline finding was stale.shared/rss-allowed-domains.jsonas editorial admission — it's the SSRF guard for the rss-proxy Edge function, not an editorial gate. (The real four-mirror sync trap is captured in theworldmonitor-architecture-gotchasskill.)brief-envelope.d.tsv4clusterIdsynthesis — no evidence of write failures; current composer populates it.seed-digest-notifications.mjs:1700-1715; Railway log search is the destination.scripts/_pipeline-dedup.mjs/_prediction-scoring.mjsaudit — unrelated to news (GEM oil/gas pipelines + Polymarket prediction markets). Report mis-attributed by filename inference.Test plan
npm run test:data— 9471 passing, 0 failing (39 new tests across U2/U3/U4)npm run typecheck:all— cleantsx --testof new test files passing locallyOut of scope (deferred to follow-ups)
Dockerfile.*CMD lines (would eliminate the manual registry edit step).