fix(brief): handle publisher-naming variants in stripHeadlineSuffix (DOJ / Bulletin)#3709
Conversation
Three structural variant classes between the headline-suffix's publisher form and the configured `source` field were observed live on the May 15 brief. Layer 1's strict word-prefix test (the load-bearing PR #3673 asymmetric protection) misses all three: 1. Article insertion — tail "Bulletin of the Atomic Scientists" vs source "Bulletin of Atomic Scientists" (the/no-the). 2. Trailing wire-suffix word — tail "BBC News" vs source "BBC". 3. Abbreviation ↔ long-form — tail "Department of Justice (.gov)" vs source "DOJ". Layer 2 adds two source-aware paths after Layer 1, all preserving the asymmetric protection: Path 2a — `normalizePublisher` on both sides + the same asymmetric prefix test. Lowercases, strips trailing domain paren, removes article words globally, removes wire-suffix words ONLY from the trailing position iteratively. Trailing-only is load-bearing: stripping `news`/`press`/`daily` globally would corrupt "Daily Mail", "News Corp", "Press TV", "The Press Democrat". Path 2b — acronym-shape-gated initials equivalence. Two gates: (a) original publisher matches /^[A-Z]{1,5}$/ — explicit opt-in via ALL-CAPS configured field, so DOJ/NPR/AP/BBC qualify but Title-Case names like Time/Wired/Axios do not; (b) tail passes a Title-Case-or-connector shape check so lowercase editorial text like "trump says that" cannot initials-match. Initials use a SEPARATE `tailForInitials` (not `normalizePublisher`) so wire-suffix words like `Press` in "Associated Press" survive to count toward `ap`. A speculative source-blind shape fallback ("Layer 3") was scoped and rejected on Codex review — feed-controlled text could force user-visible truncation, an integrity risk not worth the speculative benefit. The PR #3673 asymmetric protection (S21 — "AP News analysis" vs "AP News" must NOT strip) holds at every layer and is locked in a regression test. 26 new test scenarios in tests/brief-from-digest-stories.test.mjs; 77 existing stripHeadlineSuffix tests unchanged. 350 brief/digest tests pass under `tsx --test`. typecheck:api + biome clean. Plan: docs/plans/2026-05-15-001-fix-headline-suffix-strip-publisher-naming-variants-plan.md
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR adds Layer 2 publisher-naming variant matching to
Confidence Score: 4/5Safe to merge; the new matching paths preserve the existing asymmetric protection and the test suite is thorough for the targeted cases. The implementation is carefully layered and well-commented. The only gap is that Path 2a normalizes trailing wire-suffix tokens off both the tail and the publisher before comparing, so two publications sharing the same core name but using different wire-suffix words (e.g., configured Fox News vs a tail of Fox Daily) would be treated as equivalent — a case not documented or tested. This is unlikely to surface in practice since RSS feeds are consistent in their attribution, but it is an undocumented behavior boundary. The core logic in scripts/lib/brief-compose.mjs around Path 2a publisher-side normalization deserves a second read; the cross-wire-suffix edge case lives there. Important Files Changed
Reviews (1): Last reviewed commit: "fix(brief): handle publisher-naming vari..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| ); | ||
| }); | ||
|
|
||
| it('S10: strips "The Press Democrat" via Layer 1 equality — verifies article stripped + middle "press" preserved', () => { |
There was a problem hiding this comment.
press preserved", but the test exercises Layer 1 (exact case-insensitive equality between tail and publisher) and never reaches normalizePublisher at all. No article-stripping or wire-suffix logic runs here. A developer reading the test suite could incorrectly conclude that normalizePublisher's handling of leading/middle press is covered by this test, when it isn't.
| it('S10: strips "The Press Democrat" via Layer 1 equality — verifies article stripped + middle "press" preserved', () => { | |
| it('S10: strips "The Press Democrat" via Layer 1 equality — middle "press" is preserved but Layer 1 never invokes normalizePublisher', () => { |
| // Layer 2 — Path 2a: source-aware fuzzy match via normalised | ||
| // asymmetric prefix. normalizePublisher strips articles globally and | ||
| // wire-suffix words trailing-only, so "Bulletin of the Atomic | ||
| // Scientists" matches "Bulletin of Atomic Scientists" and "BBC News" | ||
| // matches "BBC" — without mangling "Daily Mail" or "News Corp". | ||
| const normTail = normalizePublisher(tail); | ||
| const normPub = normalizePublisher(publisher); | ||
| if ( | ||
| normTail.length > 0 | ||
| && normPub.length > 0 | ||
| && isPublisherWordPrefix(normTail, normPub) | ||
| ) { | ||
| return stripped; | ||
| } |
There was a problem hiding this comment.
normalizePublisher strips trailing wire-suffix tokens from both the tail and the publisher before the prefix test. This means two publications that share the same core name but differ only in their trailing wire-suffix word are treated as equivalent. Concretely: if the configured publisher is "Fox News" and the headline tail is "Fox Daily", both normalize to "fox" and the strip fires. The same applies to any {Core} {WireSuffix-A} publisher paired with a {Core} {WireSuffix-B} tail. While RSS feeds are generally consistent in their attribution (making this unlikely in production), the gap is undocumented and unguarded by any test.
1 participant
Summary
Three structural variant classes between the headline-suffix's publisher form and the configured
sourcefield were observed live on the May 15 brief — Layer 1's strict word-prefix test (PR #3673's load-bearing asymmetric protection) misses all three:Bulletin of the Atomic Scientistsvs sourceBulletin of Atomic ScientistsBBC Newsvs sourceBBCDepartment of Justice (.gov)vs sourceDOJApproach
Layer 2 adds two source-aware paths after Layer 1, all preserving the asymmetric protection:
normalizePublisheron both sides + the same asymmetric prefix test. Strips trailing domain paren, removes article words globally, removes wire-suffix words only from the trailing position iteratively. Trailing-only is load-bearing: strippingnews/press/dailyglobally would mangleDaily Mail,News Corp,Press TV,The Press Democrat.publishermatches/^[A-Z]{1,5}$/— explicit opt-in via ALL-CAPS configured field; (b) tail passes Title-Case-or-connector shape check. Initials use a separatetailForInitials(notnormalizePublisher) so wire-suffix words likePressinAssociated Presssurvive to count towardap.A speculative source-blind shape fallback ("Layer 3") was scoped and rejected on Codex review — feed-controlled text could force user-visible truncation, an integrity risk not worth the speculative benefit. See plan for the full design rationale.
What this is NOT
stripHeadlinePrefixpipe-separator boilerplate (e.g."Office of Public Affairs |") — separate problem, deferred.<channel><title>capture — the architecturally pure long-term fix; documented in plan as the next step if the source-aware layer leaves edge cases.Test plan
stripHeadlineSuffixtests pass unchanged — Layer 1 is untouched; Layer 2 only fires when Layer 1 returns no match.tsx --test(broader sweep includingbrief-llm,brief-magazine-render,digest-orchestration-helpers,brief-edge-route-smoke).typecheck:apiclean,biome checkclean (no new warnings on touched files).Codex review history
The plan was reviewed across 3 rounds before this PR (max 5; Codex hit its usage limit before Round 4 could verdict). Key Codex catches that hardened the design:
normalizePublisherfor initials would corruptAssociated Press → AP(Pressgot stripped). Fix: separatetailForInitialshelper. Test S14 locks this.normPubwould accept ordinary Title-Case publishers likeTime/Wired/Axiosas acronyms. Fix: gate on the original publisher matching/^[A-Z]{1,5}$/. Tests S18b/S18c/S18d lock this.Plan:
docs/plans/2026-05-15-001-fix-headline-suffix-strip-publisher-naming-variants-plan.md(gitignored, not in this PR).