fix(brief): synthesis-boundary adapter — un-starve the digest prompt + restore the grounding gate#3688
Merged
Merged
Conversation
… parity regression The May 14 0802 brief tripped `PARITY REGRESSION` for a real user: the email channel lead (403 chars) and the magazine envelope lead (373 chars) were different syntheses. The canonical-brain promise from PR #3396 ("one synthesis, every channel byte-identical") had regressed — there were THREE synthesis call sites, not one: 1. Compose pass — runSynthesisWithFallback (seed-digest:1530), ctx-aware, spliced into the envelope. Its output never survived. 2. enrichBriefEnvelopeWithLLM — generateDigestProse (brief-llm.mjs) with a 4-arg NO-ctx call that OVERWROTE envelope.data.digest. The comment claimed "this pass only fills per-story rationales"; it lied — it re-synthesised the whole digest prose. 3. Send pass — runSynthesisWithFallback (seed-digest:1949) against a different `stories` pool, for the channel bodies. So compose built a ctx-aware synthesis, enrich threw it away and re-rolled context-free, and send rolled a third time. At temperature 0.4 the three rolls diverge. Fix — collapse to ONE synthesis, produced by the compose pass: - enrichBriefEnvelopeWithLLM gains `opts.skipDigestProse`. When set, it does ONLY per-story whyMatters/description enrichment and leaves envelope.data.digest untouched — no second generateDigestProse call. The compose path passes skipDigestProse:true (the canonical synthesis is already spliced in). Call site 2 is gone. - composeAndStoreBriefForUser now returns the `synthesis` object on its briefByUser entry (alongside the existing synthesisLevel). - The send pass no longer calls runSynthesisWithFallback. It reads briefSynthesis / briefLead / synthesisLevel from the compose-pass result on the briefByUser entry. Call site 3 is gone. - L3 / opt-out gate preserved: briefSynthesis/briefLead are only set when AI_DIGEST_ENABLED && rule.aiDigestEnabled !== false && synthesisLevel !== 3. The persisted envelope always carries a digest.lead (the L3 stub included), so reading from the briefByUser entry — NOT the envelope — is what keeps L3/opt-out users from a fake "Executive Summary". - Parity log: channels_equal is now `n/a` (not a misleading `false`) when there is no channel synthesis (L3 / opt-out / compose-miss), so those users stop tripping a false PARITY REGRESSION every tick. The remaining PARITY REGRESSION branch fires only on a genuine post-compose envelope mutation — which after this change should be unreachable for L1/L2. Legitimate non-violations preserved: generateDigestProsePublic stays (separate public/share synthesis), and the L1→L2 fallback inside runSynthesisWithFallback may still call generateDigestProse twice within one canonical attempt. Tests: 2 new enrichBriefEnvelopeWithLLM cases — skipDigestProse:true suppresses the generateDigestProse call + leaves digest untouched + still enriches whyMatters; default (no opts) still synthesises (back-compat). 93/93 brief-llm tests, 8702/8702 full unit suite, typecheck clean. Plan: docs/plans/2026-05-14-001-fix-brief-pipeline-parity-grounding-opinion-plan.md (F1, Phase 1). Codex-reviewed (3 rounds, APPROVED). Phase 1.5 (F5 root cause) finding: a harness proved orderBriefCandidates cannot produce the May 14 interleaved severity walk under any topic assignment — H2 (comparator bug) ruled out, H1 confirmed. The comparator is untouched; F5 gets a post-deploy verification step.
…REGRESSION Review catch on PR #3685. The Phase-1 change fixed the channels_equal false-positive for L3/opt-out, but compose-miss still tripped the OTHER alarm: when briefByUser has no entry, `brief` is undefined, so the parity block computed winnerVariant='' → winnerMatch=false → fired `PARITY REGRESSION winner_match=false` every compose-miss tick. Split the parity block: when `!brief` (compose-miss), emit an informational parity line with winner_match=n/a channels_equal=n/a reason=compose-miss and skip BOTH alarms — the compose-miss is already logged separately via `[digest] compose-miss user=…`. The winner_match alarm now only fires when `brief` exists, which is the genuine bug case (canonical-rule filter bypass / chosenVariant drift). Updated the stale "unreachable in practice" comment to note it is reached only with a present `brief`. Pre-existing bug (compose-miss always set winnerVariant=''), but this PR owns the parity-log block and Phase 1 step 5 of the plan explicitly scopes "no compose-pass result → n/a, not a false regression" — compose-miss is the same family as L3/opt-out. typecheck clean, 143/143 brief-llm + digest-orchestration tests pass.
…+ restore the grounding gate PR B of the brief-pipeline plan (Phase 2, F2 + F8). Stacked on PR A (#3685). ESCALATED FINDING. The plan scoped F2 as "grounding gate inert." A direct probe proved it is far worse: the synthesis prompt receives ZERO story content. `buildDigest` pushes stories shaped { hash, title, link, severity, currentScore, mentionCount, phase, sources, description }. `digestFor` → `buildDigest` with no transform, so `runSynthesisWithFallback` gets that raw shape. But `buildDigestPrompt`, `checkLeadGrounding`, and `hashDigestInput` all read { headline, threatLevel, source, category, country } — fields the raw shape does not have. The title→headline / severity→threatLevel mapping only happens inside `digestStoryToUpstreamTopStory`, which runs at COMPOSE time, AFTER synthesis. Probe — feeding the verbatim buildDigest shape into buildDigestPrompt: 01. [h:abc12345] [] undefined — undefined · undefined · undefined 02. [h:def67890] [] undefined — undefined · undefined · undefined Every story line is "undefined". The model gets the sensitivity level, the greeting, and 12 lines of "[h:hash] [] undefined" — no story content. It confabulates the entire lead/threads/signals from training-data priors. This — not F1 alone — is the direct cause of the May 12 Biden+crypto hallucination and the May 14 "Ukraine attacks Russian energy" confabulation. checkLeadGrounding seeing an empty headline and skipping is the SAME mismatch, downstream. Fix — one synthesis-boundary adapter: - `digestStoryToSynthesisShape` (brief-compose.mjs) maps the raw buildDigest story → { headline, threatLevel, source, category, country, hash }. The headline gets the same prefix/suffix cleanup the magazine headline gets (so the lead grounds against the same text the reader sees). category/country default to General/Global (matching digestStoryToUpstreamTopStory + filterTopStories). - F8: every free-text field (headline, source, category, country) is run through `sanitizeForPrompt` — the digest-prose prompt carries the reader's profile context, so an unsanitised hostile RSS <title> was a prompt-injection vector. threatLevel (enum) and hash (digest) are not sanitised. - Wired at the single synthesis boundary in composeAndStoreBriefForUser: winnerStories → synthesisStories once, passed to BOTH runSynthesisWithFallback (L1 + L2-slice inherit) AND generateDigestProsePublic. composeBriefFromDigestStories keeps the raw winnerStories — digestStoryToUpstreamTopStory expects it. Tests: - 6 digestStoryToSynthesisShape unit tests (mapping, headline cleanup, source fallback, explicit category/country, F8 sanitization, missing-input safety) - 4 integration tests through the FULL live-path raw→adapted shape: buildDigestPrompt produces real content not "undefined"; checkLeadGrounding RUNS (non-empty storyTokens) and both accepts a grounded lead and rejects a fabricated one; the May 12 Biden+crypto hallucination is rejected; a hostile <title> is sanitized before the prompt. 97/97 brief-llm, 76/76 brief-from-digest-stories, 8712/8712 full unit suite, typecheck clean, biome clean (2 pre-existing complexity warnings unchanged). Note: cache key prefix not bumped — the synthesis cache material (headline/threatLevel/category/country/source) goes from all-empty to real values, so old v5 rows naturally miss; one tick of re-synthesis on rollout, same cost as any shape fix.
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Contributor
Greptile SummaryThis PR introduces
Confidence Score: 4/5The core fix is correct and well-tested; two minor issues in the adapter are non-blocking but worth addressing before the next release cycle. The synthesis-boundary adapter correctly resolves the field-name mismatch and is wired at exactly one call site, with raw stories properly preserved for the compose path. The two open issues — using the full sanitizeForPrompt on headline text (which can silently strip legitimate instruction-phrased news headlines) and the missing empty-string guard on sources[0] — are unlikely to surface in current production feeds but represent real gaps given the codebase's own sanitization conventions and the brief quality goals of this PR. scripts/lib/brief-compose.mjs — the sanitizer choice for the headline field and the empty-source guard in digestStoryToSynthesisShape. Important Files Changed
Reviews (1): Last reviewed commit: "fix(brief): synthesis-boundary adapter —..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| const rawTitle = typeof s?.title === 'string' ? s.title : ''; | ||
| const cleanTitle = stripHeadlineSuffix(stripHeadlinePrefix(rawTitle), primarySource); | ||
| return { | ||
| headline: sanitizeForPrompt(cleanTitle), |
Contributor
There was a problem hiding this comment.
sanitizeForPrompt may mangle legitimate news headlines
The adapter applies sanitizeForPrompt to the headline field, but llm-sanitize.js specifically documents that function as reserved for "free-form geoContext" and introduces a lighter sanitizeHeadline (structural-delimiters only) precisely to "avoid mangling legitimate tech/security news headlines that quote injection phrases as news subjects." Using the full sanitizer strips semantic phrases like "ignore all previous instructions" from a headline such as "Senator urges Trump to ignore all previous instructions on tariff policy," producing "Senator urges Trump to on tariff policy" — silently degrading the synthesis that this PR is trying to restore. Swapping sanitizeForPrompt(cleanTitle) for sanitizeHeadline(cleanTitle) on the headline field preserves injection-phrase quoting while still stripping structural delimiters.
Owner
Author
There was a problem hiding this comment.
Fixed in 55a619b — switched the headline field to sanitizeHeadline (structural-only), so a headline whose subject IS an injection phrase survives intact. One addition beyond the suggestion: structural-only sanitisation left a gap — a multi-line hostile <title> could still break the per-story prompt line into a line-start assistant: role turn — so the title is also normalised to a single line up front. source/category/country keep the full sanitizeForPrompt (metadata, not headlines).
Comment on lines
+361
to
+363
| const primarySource = sources.length > 0 && typeof sources[0] === 'string' | ||
| ? sources[0] | ||
| : 'Multiple wires'; |
Contributor
There was a problem hiding this comment.
sources: ['']) pass the typeof === 'string' guard and produce an empty source field rather than falling back to 'Multiple wires'. A prompt line rendered as 01. [h:...] [CRITICAL] Headline — General · Global · (trailing empty source) is cosmetically odd and may confuse the model's story-attribution reasoning. A simple && sources[0].length > 0 closes the gap.
Suggested change
| const primarySource = sources.length > 0 && typeof sources[0] === 'string' | |
| ? sources[0] | |
| : 'Multiple wires'; | |
| const primarySource = sources.length > 0 && typeof sources[0] === 'string' && sources[0].length > 0 | |
| ? sources[0] | |
| : 'Multiple wires'; |
Owner
Author
There was a problem hiding this comment.
Fixed in 55a619b — added && sources[0].trim().length > 0 so an empty or whitespace-only first entry falls back to 'Multiple wires' instead of rendering a blank attribution. Regression cases ([''], [' ']) added to the source-fallback test.
8 tasks
…ce guard
Greptile P2 review of digestStoryToSynthesisShape:
1. Headline sanitizer. The adapter ran the headline through the full
`sanitizeForPrompt`, which strips SEMANTIC injection phrases — it
would mangle a legitimate headline whose subject IS such a phrase
("Senator urges Trump to ignore all previous instructions on
tariffs" → "Senator urges Trump to on tariffs"), silently degrading
the synthesis this PR restores. The codebase already has
`sanitizeHeadline` (structural-delimiters-only) for exactly this.
Switched the headline field to it.
Structural-only sanitisation left one gap: a multi-line hostile
`<title>` could still break the per-story prompt line into a
line-start "assistant:" role turn. Closed by normalising the title
to a single line up front (a headline is one line by definition).
Net: structural injection (newline role-turn, model-delimiter
tokens) is neutralised; a phrase quoted as a news SUBJECT survives.
2. Empty-source guard. `sources: ['']` (and whitespace-only) passed
the `typeof === 'string'` guard and produced a blank `source`
field — a prompt line rendered "… — General · Global · " with a
trailing empty attribution. Added a `.trim().length > 0` check so
it falls back to "Multiple wires".
Non-headline free-text fields (source/category/country) keep the full
`sanitizeForPrompt` — they are metadata, not headlines. Tests updated:
the F8 headline test now asserts the corrected behaviour (semantic
phrase preserved, structural injection neutralised) and the
source-fallback test covers empty/whitespace entries.
418 brief-compose/llm tests pass; biome clean.
koala73
added a commit
that referenced
this pull request
May 14, 2026
…erence telemetry PR C of the brief-pipeline plan (Phase 3 F3 + Phase 4 F4). Stacked on PR B (#3688). ── F3: opinion/analysis exclusion ─────────────────────────────────── The May 14 brief ranked a Le Monde opinion column ("'Russia's invasion of Ukraine could have warned Trump…'", by columnist Gilles Paris) as story #1, tagged Critical, ahead of a nuclear ICBM test. The brief is event-driven intelligence — an op-ed column is not an event. - New `server/_shared/opinion-classifier.js` — single shared classifier `classifyOpinion({ title, link, description })`. Tiered: STRONG signals (URL /opinion//views//commentary//editorial//op-ed/ /columnists/ segments; explicit Opinion:/Analysis:/Commentary:/ Op-Ed: headline prefix) classify alone; CORROBORATING signals (whole-headline quote-wrap, description framing words, /analysis/ URL) need a STRONG signal OR two corroborating. Conservative — a false negative ships one op-ed; a false positive silently drops a real event. Note: the plan expected ingest-time byline/section metadata, but the parsed RSS item and the story:track:v1 row carry neither — only title, link, description. Both layers classify from the same three signals. - Two-layer filter: · Ingest (list-feed-digest.ts) — ParsedItem gains `isOpinion`, set by classifyOpinion in parseRssXml; buildStoryTrackHsetFields stamps `isOpinion` ('1'|'0') on the story:track:v1 row. · Read-time (buildDigest) — drops rows where `isOpinion === '1'`; for residue rows ingested BEFORE the stamp shipped (no isOpinion field), re-classifies from the persisted title/link/description. Emits `[digest] buildDigest opinion filter dropped N` telemetry. Stamps rather than drops at ingest — story:track rows feed more than the brief; only buildDigest (the brief's read path) filters. ── F4: lead ↔ final-card-#1 coherence ─────────────────────────────── The synthesis emits `lead` and `rankedStoryHashes` independently, with no constraint that the lead is ABOUT the rendered first story. And `rankedStoryHashes[0]` ≠ `data.stories[0]` — orderBriefCandidates re-sorts by severity/topic/score after the LLM rank. On May 14 the lead was about the Ukraine-energy story while data.stories[0] was the Le Monde column. - New `leadGroundsAgainstStory(lead, headline)` in brief-llm.mjs — reuses the checkLeadGrounding anchor machinery (capitalised, ≥4 chars, stopword-filtered headline anchors; token-set membership) but with a FIXED threshold of 1. checkLeadGrounding is the wrong fit: scoped to one story, a single headline can carry ≥4 anchors, tripping its size-based threshold up to 2 — too strict for a coherence question ("same story?", not "how grounded?"). - `composeAndStoreBriefForUser` runs the check AFTER composeBriefFromDigestStories, against `envelope.data.stories[0]` (the final ordered first card) — NEVER `rankedStoryHashes[0]`. It runs in the orchestration layer, not the pure composer, so composeBriefFromDigestStories stays I/O-free. Measure-first (plan F4 option b): emits `[digest] lead card1 coherence` every brief + a `LEAD/CARD-#1 INCOHERENCE` warn on mismatch; ships the brief as-is. Once the mismatch rate is known, decide between regenerating the lead bound to stories[0] or a separate leadStoryHash. ── Tests ──────────────────────────────────────────────────────────── - 11 classifyOpinion unit tests: STRONG signals alone, two- corroborating, one-corroborating-NOT-enough, the verbatim May 14 Le Monde column → opinion, /analysis/ hard-news NOT dropped, quoted-PHRASE (not whole-wrap) hard-news NOT dropped, bare-noun "Opinion polls…" NOT caught, input safety. - parseRssXml carries isOpinion (hard-news false / op-ed true). - buildStoryTrackHsetFields stamps isOpinion '1'/'0' + legacy missing-field → '0'. - 3 leadGroundsAgainstStory tests incl. the May 14 F4 regression (lead about Netanyahu vs Le Monde card-#1 → incoherent) and the degenerate no-anchor → skip. - Dockerfile.digest-notifications COPYs the new opinion-classifier.js (the transitive-import-closure test caught the missing COPY). 8728/8728 full unit suite, typecheck clean (both configs), biome clean (3 pre-existing warnings: buildDigest/main complexity already 5x over limit, one useOptionalChain on untouched list-feed-digest code — none introduced by this PR). Note: no cache-prefix bump needed — F3 changes the digest POOL membership (fewer stories), not the synthesis cache material; F4 is pure telemetry.
koala73
added a commit
that referenced
this pull request
May 14, 2026
…erence telemetry (#3690) * fix(brief): exclude opinion/analysis from the pool + lead↔card-#1 coherence telemetry PR C of the brief-pipeline plan (Phase 3 F3 + Phase 4 F4). Stacked on PR B (#3688). ── F3: opinion/analysis exclusion ─────────────────────────────────── The May 14 brief ranked a Le Monde opinion column ("'Russia's invasion of Ukraine could have warned Trump…'", by columnist Gilles Paris) as story #1, tagged Critical, ahead of a nuclear ICBM test. The brief is event-driven intelligence — an op-ed column is not an event. - New `server/_shared/opinion-classifier.js` — single shared classifier `classifyOpinion({ title, link, description })`. Tiered: STRONG signals (URL /opinion//views//commentary//editorial//op-ed/ /columnists/ segments; explicit Opinion:/Analysis:/Commentary:/ Op-Ed: headline prefix) classify alone; CORROBORATING signals (whole-headline quote-wrap, description framing words, /analysis/ URL) need a STRONG signal OR two corroborating. Conservative — a false negative ships one op-ed; a false positive silently drops a real event. Note: the plan expected ingest-time byline/section metadata, but the parsed RSS item and the story:track:v1 row carry neither — only title, link, description. Both layers classify from the same three signals. - Two-layer filter: · Ingest (list-feed-digest.ts) — ParsedItem gains `isOpinion`, set by classifyOpinion in parseRssXml; buildStoryTrackHsetFields stamps `isOpinion` ('1'|'0') on the story:track:v1 row. · Read-time (buildDigest) — drops rows where `isOpinion === '1'`; for residue rows ingested BEFORE the stamp shipped (no isOpinion field), re-classifies from the persisted title/link/description. Emits `[digest] buildDigest opinion filter dropped N` telemetry. Stamps rather than drops at ingest — story:track rows feed more than the brief; only buildDigest (the brief's read path) filters. ── F4: lead ↔ final-card-#1 coherence ─────────────────────────────── The synthesis emits `lead` and `rankedStoryHashes` independently, with no constraint that the lead is ABOUT the rendered first story. And `rankedStoryHashes[0]` ≠ `data.stories[0]` — orderBriefCandidates re-sorts by severity/topic/score after the LLM rank. On May 14 the lead was about the Ukraine-energy story while data.stories[0] was the Le Monde column. - New `leadGroundsAgainstStory(lead, headline)` in brief-llm.mjs — reuses the checkLeadGrounding anchor machinery (capitalised, ≥4 chars, stopword-filtered headline anchors; token-set membership) but with a FIXED threshold of 1. checkLeadGrounding is the wrong fit: scoped to one story, a single headline can carry ≥4 anchors, tripping its size-based threshold up to 2 — too strict for a coherence question ("same story?", not "how grounded?"). - `composeAndStoreBriefForUser` runs the check AFTER composeBriefFromDigestStories, against `envelope.data.stories[0]` (the final ordered first card) — NEVER `rankedStoryHashes[0]`. It runs in the orchestration layer, not the pure composer, so composeBriefFromDigestStories stays I/O-free. Measure-first (plan F4 option b): emits `[digest] lead card1 coherence` every brief + a `LEAD/CARD-#1 INCOHERENCE` warn on mismatch; ships the brief as-is. Once the mismatch rate is known, decide between regenerating the lead bound to stories[0] or a separate leadStoryHash. ── Tests ──────────────────────────────────────────────────────────── - 11 classifyOpinion unit tests: STRONG signals alone, two- corroborating, one-corroborating-NOT-enough, the verbatim May 14 Le Monde column → opinion, /analysis/ hard-news NOT dropped, quoted-PHRASE (not whole-wrap) hard-news NOT dropped, bare-noun "Opinion polls…" NOT caught, input safety. - parseRssXml carries isOpinion (hard-news false / op-ed true). - buildStoryTrackHsetFields stamps isOpinion '1'/'0' + legacy missing-field → '0'. - 3 leadGroundsAgainstStory tests incl. the May 14 F4 regression (lead about Netanyahu vs Le Monde card-#1 → incoherent) and the degenerate no-anchor → skip. - Dockerfile.digest-notifications COPYs the new opinion-classifier.js (the transitive-import-closure test caught the missing COPY). 8728/8728 full unit suite, typecheck clean (both configs), biome clean (3 pre-existing warnings: buildDigest/main complexity already 5x over limit, one useOptionalChain on untouched list-feed-digest code — none introduced by this PR). Note: no cache-prefix bump needed — F3 changes the digest POOL membership (fewer stories), not the synthesis cache material; F4 is pure telemetry. * fix(brief): remove unbounded "/opinion-" URL match — false-positives on hard-news slugs PR #3690 review catch. STRONG_URL_SEGMENTS included `/opinion-` (no closing slash) as a substring match — it caught hard-news ARTICLE SLUGS like `/world/opinion-polls-tighten-election`, classifying real election/polling coverage as opinion. Since buildDigest drops rows where classifyOpinion() is true, that silently removed those stories from briefs. Every other STRONG_URL_SEGMENTS entry is slash-delimited on both sides — a real path segment. `/opinion-` was the lone unbounded prefix. Removed it. Genuine opinion sections (`/opinion/`, `/opinions/`, `/commentary/`, `/editorial/`, `/op-ed/`, `/columnists/`, `/columns/`) are still caught. My original test had the gap that let this through: it checked the "Opinion polls tighten" HEADLINE with link `/world/polls` — never the `/world/opinion-polls-…` URL form. Added a regression test for the exact slug, plus a counter-control confirming a genuine `/opinion/` section still classifies. 12/12 classifier tests, biome clean, typecheck clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR B of the brief-pipeline plan (
docs/plans/2026-05-14-001-…, Phase 2 — F2 + F8). Stacked on PR A (#3685) — base isfix/brief-parity-regression; will auto-retarget tomainwhen PR A merges.Escalated finding 🔴
The plan scoped F2 as "grounding gate inert." A direct probe proved it's far worse: the synthesis prompt receives ZERO story content.
buildDigestpushes stories shaped{ hash, title, link, severity, currentScore, mentionCount, phase, sources, description }.digestFor→buildDigestwith no transform, sorunSynthesisWithFallbackgets that raw shape. ButbuildDigestPrompt,checkLeadGrounding, andhashDigestInputall read{ headline, threatLevel, source, category, country }— fields the raw shape doesn't have. Thetitle→headlinemapping only happens indigestStoryToUpstreamTopStory, which runs at COMPOSE time, after synthesis.Probe — feeding the verbatim
buildDigestshape intobuildDigestPrompt:Every story line is
undefined. The model gets the sensitivity level, the greeting, and 12 lines of[h:hash] [] undefined— no story content. It confabulates the entire lead/threads/signals from training-data priors. This — not F1 alone — is the direct cause of the May 12 Biden+crypto hallucination and the May 14 "Ukraine attacks Russian energy" confabulation.checkLeadGroundingseeing an empty headline and skipping is the same mismatch, downstream.Fix — one synthesis-boundary adapter
digestStoryToSynthesisShape(brief-compose.mjs) maps the rawbuildDigeststory →{ headline, threatLevel, source, category, country, hash }. The headline gets the same prefix/suffix cleanup the magazine headline gets (so the lead grounds against the same text the reader sees).category/countrydefault toGeneral/Global(matchingdigestStoryToUpstreamTopStory+filterTopStories).headline,source,category,country) runs throughsanitizeForPrompt. The digest-prose prompt carries the reader's profile context, so an unsanitised hostile RSS<title>was a prompt-injection vector.threatLevel(enum) andhash(digest) are not sanitised.composeAndStoreBriefForUseradaptswinnerStories → synthesisStoriesonce, passed to BOTHrunSynthesisWithFallback(L1 + L2-slice inherit) ANDgenerateDigestProsePublic.composeBriefFromDigestStorieskeeps the rawwinnerStories(digestStoryToUpstreamTopStoryexpects it).Test coverage
digestStoryToSynthesisShapeunit tests — mapping, headline cleanup, source fallback, explicit category/country, F8 sanitization, missing-input safetybuildDigestPromptproduces real content notundefined;checkLeadGroundingRUNS (non-emptystoryTokens) and both accepts a grounded lead and rejects a fabricated one; the May 12 Biden+crypto hallucination is rejected; a hostile<title>is sanitized before the promptTest plan
node --test tests/brief-llm.test.mjs→ 97/97node --test tests/brief-from-digest-stories.test.mjs→ 76/76tsx --test tests/*.test.mjs tests/*.test.mts→ 8712/8712 (one pre-existing flaky test,news-classify-cache-prefix-audit, races with concurrent temp-file tests under parallel run — passes in isolation and on re-run)npx tsc --noEmitcleanbiome check— 2 pre-existing complexity warnings onbuildDigest/main, unchangedgrounding gate skippedcount drops from ~115/window to <5; synthesis prompts contain real headlinesNotes
Cache key prefix not bumped — the synthesis cache material (
headline/threatLevel/category/country/source) goes from all-empty to real values, so old v5 rows naturally miss; one tick of re-synthesis on rollout, same cost as any shape fix.Depends on PR A (#3685): the adapter's call-site scope assumes Phase 1 already removed "call site 2" (
enrichBriefEnvelopeWithLLM's digest-prose re-synthesis). PR C (opinion exclusion + lead/card coherence) is next in the plan.