fix(brief): severity/topic-cluster ordering + Video:/Reuters headline cleanup#3673
Merged
Conversation
…d-name in suffix Two real headline-noise issues observed in production briefs. #1 — Editorial-format prefixes shipped to the magazine. May 12 brief, page 16/18: "Video: Philippine senator flees ICC arrest over role in drug war" The "Video: " tells the user nothing the magazine card body doesn't already convey (every card has its own source line and body block). Same shape applies to Watch / Live / Photos / Photo / Gallery / Listen / Podcast / Breaking / Exclusive / Opinion / Analysis / Update. New stripHeadlinePrefix helper handles all of the above with a REQUIRED trailing colon — preserves legitimate headlines that use one of these words as a noun ("Video game regulator fines...", "Watch list updated by sanctions office...", "Live broadcasts paused..."). #2 — Wire-name suffix not stripped when feed-name is the longer form. May 13 brief, story 5: "Putin says Russia will deploy new Sarmat nuclear missile this year - Reuters" primarySource: "Reuters World" stripHeadlineSuffix used strict equality: "reuters" !== "reuters world", so the suffix shipped. Fix is asymmetric word-boundary prefix-match: tail must be a SHORTER prefix of publisher (with a trailing space delimiter), e.g. tail "Reuters" against publisher "Reuters World" → strip. The inverse direction (publisher prefix of tail) is rejected ON PURPOSE — "Story - AP News analysis" with publisher "AP News" must NOT strip because "analysis" is editorial content extending the publisher name, not a desk-name suffix. Both helpers wired into digestStoryToUpstreamTopStory in two-stage order: prefix first, then suffix (handles "Video: Story - Al Jazeera" correctly). Test coverage: - REGRESSION (May 13 brief) for the Sarmat/Reuters/Reuters World case - REGRESSION (May 12 brief) for the Video: Philippine senator case - REJECTS-the-inverse coverage for AP News / AP News analysis - Word-boundary stem rejection ("reuter" / "Reuters", "iran" / "iranian press") - All 12 prefix variants individually tested Tests: 69/69 brief-from-digest-stories pass (was 56), 8669/8669 full unit suite pass, typecheck clean, biome clean.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
Greptile SummaryThis PR fixes two production headline-noise bugs: editorial-format prefixes (e.g.,
Confidence Score: 4/5Safe to merge — both helpers are additive, the asymmetric suffix logic matches the documented and tested intent, and no existing behavior is regressed. The implementation is correct and well-tested (13 new cases including both production regressions, all 12 prefix variants, and the load-bearing asymmetric inverse rejection). The only issue is one inline comment describing the suffix match as bidirectional when the code and PR description both document the asymmetry as intentional and critical — a future maintainer reading only the comment could misapply or remove the asymmetry. The inline comment on lines 287-292 of Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["rawTitle (from digest story)"] --> B["stripHeadlinePrefix(rawTitle)"]
B --> C{"Matches HEADLINE_PREFIX_RE?\n^(video|watch|live|…)\\s*:\\s*"}
C -- Yes --> D["Remove prefix → prefixStripped"]
C -- No --> D2["Return unchanged → prefixStripped"]
D --> E["stripHeadlineSuffix(prefixStripped, primarySource)"]
D2 --> E
E --> F{"Matches HEADLINE_SUFFIX_RE_PART?\n\\s+[-–—|]\\s+([^\\s].*)$"}
F -- No --> G["Return trimmed title → cleanTitle"]
F -- Yes --> H["Extract tail"]
H --> I{"isPublisherWordPrefix(tail, publisher)?"}
I -- "tail === publisher OR tail is word-boundary prefix" --> J["Slice off suffix → cleanTitle"]
I -- "tail longer than publisher OR not a word prefix" --> K["Return trimmed (keep editorial suffix) → cleanTitle"]
J --> L["digestStoryToUpstreamTopStory output"]
G --> L
K --> L
Reviews (1): Last reviewed commit: "fix(brief): strip "Video:"/"Watch:" pref..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+287
to
+292
| // Case-insensitive equality OR word-boundary prefix in either | ||
| // direction. The prefix relaxation handles wire-name vs feed-name | ||
| // mismatch (Reuters / Reuters World, AP / AP News). A tail that | ||
| // merely CONTAINS the publisher (e.g. "- AP News analysis") still | ||
| // stays — that's editorial content, not a wire suffix. | ||
| if (!isPublisherWordPrefix(tail.toLowerCase(), publisher.toLowerCase())) return trimmed; |
Contributor
There was a problem hiding this comment.
isPublisherWordPrefix(tail, publisher) accepts tail as a shorter prefix of publisher, never the reverse. The PR description explicitly calls this asymmetry the "load-bearing trick" (preventing "AP News analysis" from stripping), so the comment actively contradicts the design intent and would mislead anyone reading the code without the PR context.
Suggested change
| // Case-insensitive equality OR word-boundary prefix in either | |
| // direction. The prefix relaxation handles wire-name vs feed-name | |
| // mismatch (Reuters / Reuters World, AP / AP News). A tail that | |
| // merely CONTAINS the publisher (e.g. "- AP News analysis") still | |
| // stays — that's editorial content, not a wire suffix. | |
| if (!isPublisherWordPrefix(tail.toLowerCase(), publisher.toLowerCase())) return trimmed; | |
| // Case-insensitive equality OR asymmetric word-boundary prefix: tail | |
| // must be a SHORTER prefix of publisher (Reuters → Reuters World), | |
| // never the reverse. The asymmetry is intentional — it blocks | |
| // editorial suffixes like "AP News analysis" from being stripped | |
| // when publisher is "AP News". A tail that merely CONTAINS the | |
| // publisher (e.g. "- AP News analysis") still stays. | |
| if (!isPublisherWordPrefix(tail.toLowerCase(), publisher.toLowerCase())) return trimmed; |
Owner
Author
There was a problem hiding this comment.
Valid catch — fixed in commit 1d91206. The "either direction" wording was a stale leftover from round-1 of this PR's internal iteration. Updated comment now names the asymmetry explicitly:
Case-insensitive equality OR ASYMMETRIC word-boundary prefix: tail must be a SHORTER prefix of publisher (Reuters → Reuters World), never the reverse. The asymmetry is intentional and load-bearing — it blocks editorial suffixes like "AP News analysis" (tail) from stripping when publisher is "AP News".
Behaviour unchanged. Tests still 69/69 pass; biome clean. Thanks for the catch.
…ic one-directional match (PR #3673 review) The inline comment in stripHeadlineSuffix said 'word-boundary prefix in either direction' but the actual implementation (and the isPublisherWordPrefix docstring) is one-directional only: tail must be a SHORTER prefix of publisher, never the reverse. The asymmetry is intentional and load-bearing — it's what blocks editorial suffixes like 'AP News analysis' from stripping when publisher is 'AP News'. The 'either direction' wording was left over from round-1 of the same PR's internal iteration. Updated comment names the asymmetry explicitly and cross-references the full rationale on isPublisherWordPrefix. Comment-only change. Behaviour unchanged: 69/69 brief-from-digest tests still pass, biome clean.
koala73
changed the title
4 tasks
koala73
added a commit
that referenced
this pull request
May 15, 2026
…3709) Three structural variant classes between the headline-suffix's publisher form and the configured `source` field were observed live on the May 15 brief. Layer 1's strict word-prefix test (the load-bearing PR #3673 asymmetric protection) misses all three: 1. Article insertion — tail "Bulletin of the Atomic Scientists" vs source "Bulletin of Atomic Scientists" (the/no-the). 2. Trailing wire-suffix word — tail "BBC News" vs source "BBC". 3. Abbreviation ↔ long-form — tail "Department of Justice (.gov)" vs source "DOJ". Layer 2 adds two source-aware paths after Layer 1, all preserving the asymmetric protection: Path 2a — `normalizePublisher` on both sides + the same asymmetric prefix test. Lowercases, strips trailing domain paren, removes article words globally, removes wire-suffix words ONLY from the trailing position iteratively. Trailing-only is load-bearing: stripping `news`/`press`/`daily` globally would corrupt "Daily Mail", "News Corp", "Press TV", "The Press Democrat". Path 2b — acronym-shape-gated initials equivalence. Two gates: (a) original publisher matches /^[A-Z]{1,5}$/ — explicit opt-in via ALL-CAPS configured field, so DOJ/NPR/AP/BBC qualify but Title-Case names like Time/Wired/Axios do not; (b) tail passes a Title-Case-or-connector shape check so lowercase editorial text like "trump says that" cannot initials-match. Initials use a SEPARATE `tailForInitials` (not `normalizePublisher`) so wire-suffix words like `Press` in "Associated Press" survive to count toward `ap`. A speculative source-blind shape fallback ("Layer 3") was scoped and rejected on Codex review — feed-controlled text could force user-visible truncation, an integrity risk not worth the speculative benefit. The PR #3673 asymmetric protection (S21 — "AP News analysis" vs "AP News" must NOT strip) holds at every layer and is locked in a regression test. 26 new test scenarios in tests/brief-from-digest-stories.test.mjs; 77 existing stripHeadlineSuffix tests unchanged. 350 brief/digest tests pass under `tsx --test`. typecheck:api + biome clean. Plan: docs/plans/2026-05-15-001-fix-headline-suffix-strip-publisher-naming-variants-plan.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes three brief-quality issues observed on the May 12 / May 13 sends:
Editorial prefixes shipping to the magazine. May 12 brief, page 16/18: "Video: Philippine senator flees ICC arrest…". Strip
Video:,Watch:,Live:,Photos:,Breaking:,Exclusive:,Opinion:,Analysis:,Update:, and similar editorial-format prefixes (12 variants) at compose time. The trailing colon is REQUIRED so legitimate noun-form headlines ("Video game regulator…", "Watch list updated…", "Live broadcasts paused…") survive intact.Wire-name suffixes surviving when feed-name is the longer form. May 13 brief, story 5: "Putin says Russia will deploy new Sarmat nuclear missile this year - Reuters";
primarySourcewasReuters World. The old strict-equality check ("reuters" !== "reuters world") shipped the suffix to the magazine. Fix: asymmetric word-boundary prefix match — tail must be a SHORTER prefix of publisher (ReutersagainstReuters World→ strip). The inverse direction is rejected ON PURPOSE so "Story - AP News analysis" with publisherAP Newskeeps the editorialanalysisword.Severity / topic-cluster ordering breaking down to a chaotic interleaved walk. May 13 brief order was 🔴🟠🟠🟠🔴🔴🔴🔴🔴🟠🟠🟠 with Nigeria stories split by Syria and the Iran arc fragmented across four non-adjacent positions. Root cause:
applyRankedOrderlet LLMrankedStoryHashesdominate the global sort, overriding the topic blocks fromgroupTopicsPostDedupand ignoring severity entirely. Replace withorderBriefCandidates— a block-based sort keyed by (topic block's highest eligible severity, count at that severity, eligible block size, max score, then LLM rank as tie-breaker). Critical clusters now stay contiguous; LLM still chooses between similarly severe candidates.Ordering trade-off
The ordering model intentionally favors concentrated top severity before breadth: a block with two critical stories sorts ahead of a block with one critical plus several high stories. Once top-severity count ties, broader eligible block size wins, then max score, then LLM rank. Each tiebreak rung is individually tested so a future contributor reordering the keys will fail a named test, not a vibes check.
Rows without
briefTopicIdorclusterRepHashdegrade to per-row singleton blocks. That preserves severity ordering but loses topic adjacency for legacy/raw rows — deliberate, to avoid false-grouping unrelated stories under a guessed key.Ops note
dropped_captelemetry now counts only otherwise-renderable stories skipped aftermaxStories. Invalid or sensitivity-excluded tail items keep their root-cause reason (severity,headline,url, etc.). The reconciliation invariant still holds (in === out + Σ dropped_*), but absolutedropped_capvalues may be lower post-deploy. Worth recalibrating any alerts keyed on the absolute count.Test coverage
Video:prefix strippingreuter/Reuters,iran/iranian press)groupTopicsPostDedupthrough composeTest plan
node --test tests/brief-filter.test.mjsnode --test tests/brief-from-digest-stories.test.mjsnode --test tests/brief-dedup-embedding.test.mjsnpm run typecheckbiome checkclean (verified locally + on CI)Video:/Watch:/Live:prefixes on magazine cards- Reuterssuffix when story source isReuters WorldNotes
Independent of PR #3667 (digest grounding validation). PR #3667 guards synthesis content grounding (catches hallucinated leads); this PR handles headline cleanup and deterministic story ordering before render. Both touch the brief but address orthogonal bug classes.
For the full pipeline context, see
docs/internal/brief130526.md— the §7 signal-conflict matrix names every offending signal interaction this PR closes.