fix(news): ground LLM surfaces on real RSS description end-to-end#3370
Merged
Conversation
Add description field to ParsedItem, extract from the first non-empty of description/content:encoded (RSS) or summary/content (Atom), picking the longest after HTML-strip + entity-decode + whitespace-normalize. Clip to 400 chars. Reject empty, <40 chars after strip, or normalize-equal to the headline — downstream consumers fall back to the cleaned headline on '', preserving current behavior for feeds without a description. CDATA end is anchored to the closing tag so internal ]]> sequences do not truncate the match. Preserves cached rss:feed:v1 row compatibility during the 1h TTL bleed since the field is additive. Part of fix: pipe RSS description end-to-end so LLM surfaces stop hallucinating named actors (docs/plans/2026-04-24-001-...). Covers R1, R7.
Append description to the story:track:v1 HSET only when non-empty. Additive — no key version bump. Old rows and rows from feeds without a description return undefined on HGETALL, letting downstream readers fall back to the cleaned headline (R6). Extract buildStoryTrackHsetFields as a pure helper so the inclusion gate is unit-testable without Redis. Update the contract comment in cache-keys.ts so the next reader of the schema sees description as an optional field. Covers R2, R6.
Add two additive proto fields so the article description can ride to every LLM-adjacent consumer without a breaking change: - NewsItem.snippet (field 12): RSS/Atom description, HTML-stripped, ≤400 chars, empty when unavailable. Wired on toProtoItem. - SummarizeArticleRequest.bodies (field 8): optional article bodies paired 1:1 with headlines for prompt grounding. Empty array is today's headline-only behavior. Regenerated TS client/server stubs and OpenAPI YAML/JSON via sebuf v0.11.1 (PATH=~/go/bin required — Homebrew's protoc-gen-openapiv3 is an older pre-bundle-mode build that collides on duplicate emission). Pre-emptive bodies:[] placeholders at the two existing SummarizeArticle call sites in src/services/summarization.ts; U6 replaces them with real article bodies once SummarizeArticle handler reads the field. Covers R3, R5.
…envelope (U4) Digest accumulator reader (seed-digest-notifications.mjs::buildDigest) now plumbs the optional `description` field off each story:track:v1 HGETALL into the digest story object. The brief adapter (brief-compose.mjs:: digestStoryToUpstreamTopStory) prefers the real RSS description over the cleaned headline; when the upstream row has no description (old rows in the 48h bleed, feeds that don't carry one), we fall back to the cleaned headline so today behavior is preserved (R6). This is the upstream half of the description cache path. U5 lands the LLM- side grounding + cache-prefix bump so Gemini actually sees the article body instead of hallucinating a named actor from the headline. Covers R4 (upstream half), R6.
…(U5)
The actual fix for the headline-only named-actor hallucination class:
Gemini 2.5 Flash now receives the real article body as grounding context,
so it paraphrases what the article says instead of filling role-label
headlines from parametric priors ("Iran's new supreme leader" → "Ali
Khamenei" was the 2026-04-24 reproduction; with grounding, it becomes
the actual article-named actor).
Changes:
- buildStoryDescriptionPrompt interpolates a `Context: <body>` line
between the metadata block and the "One editorial sentence" instruction
when description is non-empty AND not normalise-equal to the headline.
Clips to 400 chars as a second belt-and-braces after the U1 parser cap.
No Context line → identical prompt to pre-fix (R6 preserved).
- sanitizeStoryForPrompt extended to cover `description`. Closes the
asymmetry where whyMatters was sanitised and description wasn't —
untrusted RSS bodies now flow through the same injection-marker
neutraliser before prompt interpolation. generateStoryDescription wraps
the story in sanitizeStoryForPrompt before calling the builder,
matching generateWhyMatters.
- Four cache prefixes bumped atomically to evict pre-grounding rows:
scripts/lib/brief-llm.mjs:
brief:llm:description:v1 → v2 (Railway, description path)
brief:llm:whymatters:v2 → v3 (Railway, whyMatters fallback)
api/internal/brief-why-matters.ts:
brief:llm:whymatters:v6 → v7 (edge, primary)
brief:llm:whymatters:shadow:v4 → shadow:v5 (edge, shadow)
hashBriefStory already includes description in the 6-field material
(v5 contract) so identity naturally drifts; the prefix bump is the
belt-and-braces that guarantees a clean cold-start on first tick.
- Tests: 8 new + 2 prefix-match updates on tests/brief-llm.test.mjs.
Covers Context-line injection, empty/dup-of-headline rejection,
400-char clip, sanitisation of adversarial descriptions, v2 write,
and legacy-v1 row dark (forced cold-start).
Covers R4 + new sanitisation requirement.
SummarizeArticle now grounds on per-headline article bodies when callers supply them, so the dashboard "News summary" path stops hallucinating across unrelated headlines when the upstream RSS carried context. Three coordinated changes: 1. SummarizeArticleRequest handler reads req.bodies, sanitises each entry through sanitizeForPrompt (same trust treatment as geoContext — bodies are untrusted RSS text), clips to 400 chars, and pads to the headlines length so pair-wise identity is stable. 2. buildArticlePrompts accepts optional bodies and interleaves a ` Context: <body>` line under each numbered headline that has a non-empty body. Skipped in translate mode (headline[0]-only) and when all bodies are empty — yielding a byte-identical prompt to pre-U6 for every current caller (R6 preserved). 3. summary-cache-key bumps CACHE_VERSION v5→v6 so the pre-grounding rows (produced from headline-only prompts) cold-start cleanly. Extends canonicalizeSummaryInputs + buildSummaryCacheKey with a pair-wise bodies segment `:bd<hash>`; the prefix is `:bd` rather than `:b` to avoid colliding with `:brief:` when pattern-matching keys. Translate mode is headline[0]-only and intentionally does not shift on bodies. Dedup reorder preserved: the handler re-pairs bodies to the deduplicated top-5 via findIndex, so layout matches without breaking cache identity. New tests: 7 on buildArticlePrompts (bodies interleave, partial fill, translate-mode skip, clip, short-array tolerance), 8 on buildSummaryCacheKey (pair-wise sort, cache-bust on body drift, translate skip). Existing summary-cache-key assertions updated v5→v6. Covers R3, R4.
…MCP + audit (U7) Thread the RSS description from the ingestion path (U1-U5) into every user-facing LLM-adjacent surface. Audit the notification producers so RSS-origin and domain-origin events stay on distinct contracts. Dashboard (proto snippet → client → panel): - src/types/index.ts NewsItem.snippet?:string (client-side field). - src/app/data-loader.ts proto→client mapper propagates p.snippet. - src/components/NewsPanel.ts renders snippet as a truncated (~200 chars, word-boundary ellipsis) `.item-snippet` line under each headline. - NewsPanel.currentBodies tracks per-headline bodies paired 1:1 with currentHeadlines; passed as options.bodies to generateSummary so the server-side SummarizeArticle LLM grounds on the article body. Summary plumbing: - src/services/summarization.ts threads bodies through SummarizeOptions → generateSummary → runApiChain → tryApiProvider; cache key now includes bodies (via U6's buildSummaryCacheKey signature). MCP world-brief: - api/mcp.ts pairs headlines with their RSS snippets and POSTs `bodies` to /api/news/v1/summarize-article so the MCP tool surface is no longer starved. Email digest: - scripts/seed-digest-notifications.mjs plain-text formatDigest appends a ~200-char truncated snippet line under each story; HTML formatDigestHtml renders a dim-grey description div between title and meta. Both gated on non-empty description (R6 — empty → today's behavior). Real-time alerts: - src/services/breaking-news-alerts.ts BreakingAlert gains optional description; checkBatchForBreakingAlerts reads item.snippet; dispatchAlert includes `description` in the /api/notify payload when present. Notification relay: - scripts/notification-relay.cjs formatMessage gated on NOTIFY_RELAY_INCLUDE_SNIPPET=1 (default off). When on, RSS-origin payloads render a `> <snippet>` context line under the title. When off or payload.description absent, output is byte-identical to pre-U7. Audit (RSS vs domain): - tests/notification-relay-payload-audit.test.mjs enforces file-level @notification-source tags on every producer, rejects `description:` in domain-origin payload blocks, and verifies the relay codepath gates snippet rendering under the flag. - Tag added to ais-relay.cjs (domain), seed-aviation.mjs (domain), alert-emitter.mjs (domain), breaking-news-alerts.ts (rss). Deferred (plan explicitly flags): InsightsPanel + cluster-producer plumbing (bodies default to [] — will unlock gradually once news:insights:v1 producer also carries primarySnippet). Covers R5, R6.
Final verification for the RSS-description-end-to-end fix:
- docs/architecture.mdx — one-paragraph "News Grounding Pipeline"
subsection tracing parser → story:track:v1.description → NewsItem.snippet
→ brief / SummarizeArticle / dashboard / email / relay / MCP, with the
empty-description R6 fallback rule called out explicitly.
- tests/summarize-reasoning.test.mjs — Fix-4 static-analysis pin updated
to match the v6 bump from U6. Without this the summary cache bump silently
regressed CI's pinned-version assertion.
Final sweep (2026-04-24):
- grep -rn 'brief:llm:description:v1' → only in the U5 legacy-row test
simulation (by design: proves the v2 bump forces cold-start).
- grep -rn 'brief:llm:whymatters:v2/v6/shadow:v4' → no live references.
- grep -rn 'summary:v5' → no references.
- CACHE_VERSION = 'v6' in src/utils/summary-cache-key.ts.
- Full tsx --test sweep across all tests/*.test.{mjs,mts}: 6747/6747 pass.
- npm run typecheck + typecheck:api: both clean.
Covers R4, R6, R7.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Greptile SummaryPipes RSS Confidence Score: 5/5Safe to merge — all remaining findings are P2 style issues; fallback behavior (R6) is preserved on every surface, and all four cache namespace bumps are coordinated in one deploy. The implementation is thorough, with end-to-end test coverage from the RSS parser through every LLM surface. Cache version bumps are clearly motivated and atomically coordinated. Sanitisation is applied at every untrusted-input boundary. The single P2 finding (SNIPPET_PUSH_MAX dead code) has no runtime impact since the snippet-relay path is default-off. scripts/notification-relay.cjs — SNIPPET_PUSH_MAX is declared but never wired into formatMessage; clean up before enabling NOTIFY_RELAY_INCLUDE_SNIPPET in production. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["RSS Feed XML\ndescription / content:encoded\nsummary / content"] --> B["extractDescription\nstrip HTML, decode entities\nclip 400, reject under 40 or dup"]
B --> C["ParsedItem.description"]
C --> D["story:track:v1 HSET\nadditive write, empty omitted"]
C --> E["toProtoItem\nNewsItem.snippet field 12"]
D --> F["digest cron\ndigestStoryToUpstreamTopStory\ndescription or cleanHeadline"]
E --> G["data-loader\nNewsItem.snippet"]
F --> H["brief-llm\nbuildStoryDescriptionPrompt\nContext line injected"]
F --> I["brief-why-matters\nhashBriefStory includes description\ncache v7"]
E --> J["summarize-article\nbodies sanitised and clipped\nContext lines in prompt\ncache v6 plus body hash"]
G --> K["NewsPanel\ncurrentBodies paired 1:1\ngenerateSummary with bodies"]
G --> L["breaking-news-alerts\nBreakingAlert.description\nnotify payload"]
G --> M["MCP world-brief\nbodies array to SummarizeArticle"]
F --> N["seed-digest-notifications\nemail plain and HTML snippet"]
L --> O["notification-relay\nSNIPPET gate default off\nformatMessage with snippet"]
Reviews (1): Last reviewed commit: "docs+test: grounding-path note + bump pi..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+641
to
+642
| // RSS-origin events (source: rss, e.g. from src/services/breaking-news-alerts.ts) | ||
| // MUST set `payload.description` when their upstream NewsItem carried a |
Contributor
There was a problem hiding this comment.
SNIPPET_PUSH_MAX declared but never used
SNIPPET_PUSH_MAX = 150 is defined alongside SNIPPET_TELEGRAM_MAX = 400, but formatMessage only ever calls truncateForDisplay(..., SNIPPET_TELEGRAM_MAX) — the push constant is dead code. When NOTIFY_RELAY_INCLUDE_SNIPPET is eventually enabled, push notifications will be truncated at 400 chars instead of the documented 150-char OS clip boundary. Either wire the constant into a push-specific path or remove it to avoid future confusion.
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
14 fixes from structured code review across 13 reviewer personas.
Correctness-critical (P1 — fixes that prevent R6/U7 contract violations):
- NewsPanel signature covers currentBodies so view-mode toggles that leave
headlines identical but bodies different now invalidate in-flight summaries.
Without this, switching renderItems → renderClusters mid-summary let a
grounded response arrive under a stale (now-orphaned) cache key.
- summarize-article.ts re-pairs bodies with headlines BEFORE dedup via a
single zip-sanitize-filter-dedup pass. Previously bodies[] was indexed by
position in light-sanitized headlines while findIndex looked up the
full-sanitized array — any headline that sanitizeHeadlines emptied
mispaired every subsequent body, grounding the LLM on the wrong story.
- Client skips the pre-chain cache lookup when bodies are present, since
client builds keys from RAW bodies while server sanitizes first. The
keys diverge on injection content, which would silently miss the
server's authoritative cache every call.
Test + audit hardening:
- Legacy v1 eviction test now uses the real hashBriefStory(story()) suffix
instead of a literal "somehash", so a bug where the reader still queried
the v1 prefix at the real key would actually be caught.
- tests/summary-cache-key.test.mts adds 400-char clip identity coverage so
the canonicalizer's clip and any downstream clip can't silently drift.
- tests/news-rss-description-extract.test.mts renames the well-formed
CDATA test and adds a new test documenting the malformed-]]> fallback
behavior (plain regex captures, article content survives).
Safe_auto cleanups:
- Deleted dead SNIPPET_PUSH_MAX constant in notification-relay.cjs.
- BETA-mode groq warm call now passes bodies, warming the right cache slot.
- seed-digest shares a local normalize-equality helper for description !=
headline comparison, matching the parser's contract.
- Pair-wise sort in summary-cache-key tie-breaks on body so duplicate
headlines produce stable order across runs.
- buildSummaryCacheKey gained JSDoc documenting the client/server contract
and the bodies parameter semantics.
- MCP get_world_brief tool description now mentions RSS article-body
grounding so calling agents see the current contract.
- _shared.ts `opts.bodies![i]!` double-bang replaced with `?? ''`.
- extractRawTagBody regexes cached in module-level Map, mirroring the
existing TAG_REGEX_CACHE pattern.
Deferred to follow-up (tracked for PR description / separate issue):
- Promote shared MAX_BODY constant across the 5 clip sites
- Promote shared truncateForDisplay helper across 4 render sites
- Collapse NewsPanel.{currentHeadlines, currentBodies} → Array<{title, snippet}>
- Promote sanitizeStoryForPrompt to shared/brief-llm-core.js
- Split list-feed-digest.ts parser helpers into sibling -utils.ts
- Strengthen audit test: forward-sweep + behavioral gate test
Tests: 6749/6749 pass. Typecheck clean on both configs.
Addresses the second of two Codex-raised findings on PR #3370: The PR threaded bodies through the server-side API provider chain (Ollama → Groq → OpenRouter → /api/news/v1/summarize-article) but the local browser T5 path at tryBrowserT5 was still summarising from headlines alone. In BETA_MODE that ungrounded path runs BEFORE the grounded server providers; in normal mode it remains the last fallback. Whenever T5-small won, the dashboard summary surface regressed to the headline-only path — the exact hallucination class this PR exists to eliminate. Fix: tryBrowserT5 accepts an optional `bodies` parameter and interleaves each body with its paired headline via a `headline — body` separator in the combined text (clipped to 200 chars per body to stay within T5-small's ~512-token context window). All three call sites (BETA warm, BETA cold, normal-mode fallback) now pass the bodies threaded down from generateSummary options.bodies. When bodies is empty/omitted, the combined text is byte-identical to pre-fix (R6 preserved). On Codex finding #1 (story:track:v1 additive-only HSET keeps a body from an earlier mention of the same normalized title), declining to change. The current rule — "if this mention has a body, overwrite; otherwise leave the prior body alone" — is defensible: a body from mention A is not falsified by mention B being body-less (a wire reprint doesn't invalidate the original source's body). A feed that publishes a corrected headline creates a new normalized-title hash, so no stale body carries forward. The failure window is narrow (live story evolving while keeping the same title through hours of body-less wire reprints) and the 7-day STORY_TTL is the backstop. Opening a follow-up issue to revisit semantics if real-world evidence surfaces a stale-grounding case.
…s (Codex #1) Revisiting Codex finding #1 on PR #3370 after re-review. The previous response declined the fix with reasoning; on reflection the argument was over-defending the current behavior. Problem: buildStoryTrackHsetFields previously wrote `description` only when non-empty. Because story:track:v1 rows are collapsed by normalized-title hash, an earlier mention's body would persist for up to STORY_TTL (7 days) on subsequent body-less mentions of the same story. Consumers reading `track.description` via HGETALL could not distinguish "this mention's body" from "some mention's body from the last week," silently grounding brief / whyMatters / SummarizeArticle LLMs on text the current mention never supplied. That violates the grounding contract advertised to every downstream surface in this PR. Fix: HSET `description` unconditionally on every mention — empty string when the current item has no body, real body when it does. An empty value overwrites any prior mention's body so the row is always authoritative for the current cycle. Consumers continue to treat empty description as "fall back to cleaned headline" (R6 preserved). The 7-day STORY_TTL and normalized-title hash semantics are unchanged. Trade-off accepted: a valid body from Feed A (NYT) is wiped when Feed B (AP body-less wire reprint) arrives for the same normalized title, even though Feed A's body is factually correct. Rationale: the alternative — keeping Feed A's body indefinitely — means the user sees Feed A's body attributed (by proximity) to an AP mention at a later timestamp, which is at minimum misleading and at worst carries retracted/corrected details. Honest absence beats unlabeled presence. Tests: new stale-body overwrite sequence test (T0 body → T1 empty → T2 new body), existing "writes description when non-empty" preserved, existing "omits when empty" inverted to "writes empty, overwriting." cache-keys.ts contract comment updated to mark description as always-written rather than optional.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
News24's
Iran's new supreme leader seriously woundedheadline surfaced in a 2026-04-24-0801 PRO brief with the wrong name: Gemini 2.5 Flash substituted its parametric prior (Ali Khamenei) for the role label because it only saw the headline + 4 metadata fields. The article body namedMojtaba Khamenei, but the parser atserver/worldmonitor/news/v1/list-feed-digest.tsdropped the RSS<description>at parse time and no downstream LLM surface ever saw article context.This is the
llm-analyst-context-starvationpattern: every LLM-backed surface in the product (brief description card, briefwhyMattersanalyst blurb, dashboardSummarizeArticle, email digest, notification relay, MCP world-brief) was starved by the same missing-data root cause.Fix
Pipe the RSS article description through the entire pipeline so every grounded surface paraphrases what the article says instead of hallucinating from the headline.
Fallback rule everywhere: empty description → consumer behavior is byte-identical to today. No regression path.
8 implementation units
story:track:v1HSET additivedescription(no key version bump; old rows return undefined naturally)NewsItem.snippet = 12,SummarizeArticleRequest.bodies = 8; sebuf v0.11.1 codegendigestStoryToUpstreamTopStoryforward real body, fall back to clean headline (R6)sanitizeStoryForPromptextended to description + 4 cache prefixes bumped atomically:brief:llm:description:v1 → v2(Railway)brief:llm:whymatters:v2 → v3(Railway fallback)brief:llm:whymatters:v6 → v7(edge primary)brief:llm:whymatters:shadow:v4 → shadow:v5(edge shadow)req.bodies, sanitises viasanitizeForPrompt, interleavesContext:lines;summary-cache-keyv5 → v6+ pair-wise:bd<hash>body-digest segment@notification-sourcetags on every producerWhy the 4 cache prefixes bump in one PR
Per
feedback_webhook_envelope_version_needs_cross_producer_coord+ explicit plan note: both Railway cron and Vercel edge read/write these prefixes. Splitting the bumps across PRs leaves a corrected description paired with a whyMatters sentence still asserting the old hallucinated name for up to 24h. One deploy, one tick of extra Gemini LLM cost while the new prefixes cold-start, prior prefixes age out naturally over 24h.Test plan
npm run typecheck+npm run typecheck:api— both greennpm run test:data— 6747/6747 passnpm run lint— 0 errors (205 pre-existing warnings)npm run lint:md— 0 errorsnode --test tests/edge-functions.test.mjs— 177/177 passnpm run version:check— cleanMojtabanotAli Khameneibrief:llm:description:v2:*,:whymatters:v3:*,:v7:*,:shadow:v5:*growing; prior prefixes trending to zero over 24hsummary:v6:*growing;summary:v5:*aging outDeferred to follow-up
parseStoryDescription(re-evaluate after 7 days of metrics on the grounded path)primarySnippetplumbing (cluster producer change unlocks grounding on that surface automatically)