feat(digest): topic-grouped brief ordering (size-first)#3247
Merged
Conversation
Brief composer currently surfaces top-N stories in raw currentScore DESC order. On topic-dominant news days (e.g. 2026-04-20 20:00 brief) related stories scatter — 4 Hormuz angles at positions 1/3/8/11 with unrelated stories wedged between them. Secondary clustering pass on already-sliced top-30 reps at a looser cosine threshold (default 0.45), then re-orders by a total key: (topicSize DESC, topicMax DESC, repScore DESC, titleHashHex ASC). The dominant thread leads; within-thread order is score DESC; ties are deterministic. Hidden behind DIGEST_DEDUP_TOPIC_GROUPING (default 1 — kill switch = '0', no deploy). Design notes: - Post-slice placement bounds work to N ≤ 30 (~0.4 ms) and avoids reshuffling reps that never surface. - Sidecar Map<hash, number[]> returned from deduplicateStories — no hidden __embedding fields on the user-facing Rep (would otherwise risk leaking into the brief envelope). - groupTopicsPostDedup is pure: no I/O, no logging, injected clusterFn for testability. Errors are RETURNED (not thrown) so a helper bug cannot cascade into the outer Jaccard fallback boundary. - Caller owns logging: deduplicateStories returns logSummary; caller splices ` topics=N ` after `clusters=M ` via a simple regex, emits one log line per tick. Env: - DIGEST_DEDUP_TOPIC_GROUPING = '0' disables (default on) - DIGEST_DEDUP_TOPIC_THRESHOLD = float in (0,1], default 0.45 Tests: 55 in tests/brief-dedup-embedding.test.mjs (was 33, +22 new): size-first ordering, topicMax tiebreak, within-topic score, titleHash determinism, kill switch, permutation invariance, empty/singleton, injected clusterer throws, missing embedding, materialized-rep keying, envelope cleanliness (JSON.stringify has no _embedding / __ / embeddingByHash), log-line splice regex, pre-slice input size, and 6 env-parsing cases. Full verification: npm run test:data (5913 pass), typecheck, typecheck:api, biome check on changed files — all clean. Pre-existing main() complexity (74) unchanged.
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Contributor
Greptile SummaryAdds a secondary cosine-clustering pass ( One edge case in the sort comparator is worth addressing: when two topics share the same Confidence Score: 5/5Safe to merge; the two remaining findings are P2 edge-case quality issues that don't affect the primary path. Both findings are P2: one is a corner case (same-size + same-topicMax topic tie causing interleaving, which is rare and degrades gracefully) and one is a defensive-coding suggestion (sparse array sentinel). The core fallback/kill-switch/error-return design is correct, the sidecar doesn't leak, and the 55-test suite covers the important boundaries. No data loss, security, or crash risk introduced. scripts/lib/brief-dedup.mjs — sort comparator in groupTopicsPostDedup (lines 333–342) Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[buildDigest: raw stories] --> B[deduplicateStories]
B --> C{mode?}
C -- jaccard --> D[Jaccard fallback\nembeddingByHash = empty Map]
C -- embed --> E[embedBatch + singleLinkCluster\nbuild embeddingByHash sidecar]
E --> F[materializeCluster per cluster\nstore winning rep embedding]
D --> G[dedupedAll + logSummary]
F --> G
G --> H[score floor filter]
H --> I[slice 0..30]
I --> J[groupTopicsPostDedup]
J --> K{topicGroupingEnabled?}
K -- false --> L[pass-through\ntopicCount=input.length]
K -- true --> M[singleLinkCluster\nloose cosine=0.45\nno entity veto]
M --> N{all embeddings present?}
N -- no --> O[return error\npreserve primary order]
N -- yes --> P[compute topicSize/topicMax\nper cluster]
P --> Q[sort: topicSize DESC\ntopicMax DESC\nrepScore DESC\ntitleHashHex ASC]
Q --> R[grouped reps]
L --> S[top reps for dispatch]
O --> S
R --> S
S --> T[splice topics=N into logSummary\nconsole.log]
T --> U[source attribution + channel dispatch]
Reviews (1): Last reviewed commit: "feat(digest): topic-grouped brief orderi..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+333
to
+342
| const order = [...top.keys()].sort((a, b) => { | ||
| const tA = topicOf[a]; | ||
| const tB = topicOf[b]; | ||
| if (topicSize[tA] !== topicSize[tB]) return topicSize[tB] - topicSize[tA]; | ||
| if (topicMax[tA] !== topicMax[tB]) return topicMax[tB] - topicMax[tA]; | ||
| const sA = Number(top[a].currentScore ?? 0); | ||
| const sB = Number(top[b].currentScore ?? 0); | ||
| if (sA !== sB) return sB - sA; | ||
| return hashOf[a] < hashOf[b] ? -1 : hashOf[a] > hashOf[b] ? 1 : 0; | ||
| }); |
Contributor
There was a problem hiding this comment.
When two topics share the same topicSize AND the same topicMax, the comparator falls through to individual repScore DESC for cross-topic pairs. This breaks the "contiguous blocks" guarantee: a lower-scoring member of topic B can sort before a higher-scoring member of topic A, interleaving the two topics.
Concrete example — Topic A = [score 80, score 70], Topic B = [score 80, score 75] (both size 2, both max 80):
A₈₀vsB₈₀: hash tiebreak → sayA₈₀firstA₇₀vsB₇₅:repScorefallback putsB₇₅aboveA₇₀- Final output:
A₈₀, B₈₀, B₇₅, A₇₀— interleaved ✗
A deterministic topic-level tiebreaker (e.g. the tA/tB index itself, which is stable after the earlier score-DESC pre-sort) should be inserted before the individual repScore comparison so all members of one topic remain contiguous regardless of intra-topic score distribution.
// After topicMax tiebreak, use topic index before individual rep score:
if (tA !== tB) return tA - tB; // stable cross-topic order
const sA = Number(top[a].currentScore ?? 0);
const sB = Number(top[b].currentScore ?? 0);
if (sA !== sB) return sB - sA;
return hashOf[a] < hashOf[b] ? -1 : hashOf[a] > hashOf[b] ? 1 : 0;The test suite covers topicMax ties but has no fixture for same-size + same-max topics, so this gap is currently undetected.
| vetoFn: null, | ||
| }); | ||
|
|
||
| const topicOf = new Array(top.length); |
Contributor
There was a problem hiding this comment.
topicOf array — silent contract violations
new Array(top.length) creates a sparse (hole-filled) array. Every index is assigned during the clusters.forEach loop as long as clusterFn covers every input index — which singleLinkCluster's union-find guarantees today.
If a future injected clusterFn omits an index, topicOf[i] is undefined, causing topicSize[undefined] and topicMax[undefined] to both return undefined and the sort comparator to silently degrade to individual scores rather than surfacing the contract violation.
Suggested change
| const topicOf = new Array(top.length); | |
| const topicOf = new Array(top.length).fill(-1); |
Fixes 5 findings raised by the multi-agent review of PR #3247: - #234 P2: Drop dead `deps.log` shim from `deduplicateStories` — caller now owns the log line so the param rotted (no test used it post-#3247). Removed from JSDoc + signature logic. - #236 P3: Add defensive warn when `winningIdx === undefined` during sidecar `embeddingByHash` population. Shouldn't fire with the current `materializeCluster` contract, but catches a future refactor where a synthesised rep would silently skip topic grouping. - #237 P3: Skip `groupTopicsPostDedup` when `cfg.mode === 'jaccard'` — the kill-switch path returns an empty `embeddingByHash`, and running the secondary pass on it would log a noisy "missing embedding" warn every tick. Gate the call site; passthrough primary order. - #240a P3: Remove dead `top ?? []` fallback after `!Array.isArray(top)` already handled the falsy case. Replaced with an explicit Array check for the rare "falsy but also not-array" input (defence-in-depth). - #240b P3: Delete two redundant test blocks — the `titleHashHex tiebreak` 2-rep fixture (permutation-invariance at 15-rep scale already covers this invariant) and the `caller log-line format (regex splice)` describe block (the regex lives in seed-digest-notifications.mjs, not brief-dedup; the full-flow envelope-cleanliness test exercises the caller end-to-end). Deferred to follow-up: #235 (structured logParts vs regex splice), #238 (plumb cfg to avoid double env-read), #239 (repo-wide env .trim pattern). Tests: 5910 pass (was 5913; -3 redundant tests removed). typecheck, typecheck:api, biome all clean.
Two P1/P2 bugs found in post-PR-#3247 review: P1: groupTopicsPostDedup did NOT guarantee contiguous topic blocks. The global sort key (topicSize, topicMax, repScore, titleHashHex) fell through to per-rep repScore when two topics tied on size and max, interleaving members. Two same-size-same-max topics {A90,A80} and {B90,B70} output as [A90,B90,A80,B70] — broken contiguity, breaking the editorial promise of the PR. Fix: two-phase sort. Phase 1 orders TOPICS by (topicSize DESC, topicMax DESC, topicTieHash ASC) where topicTieHash = min titleHash among the topic's members — a topic-level invariant, not a rep-level one. Phase 2 orders members within each topic by (repScore DESC, titleHashHex ASC). Concatenate in topic order. Members of the same topic CANNOT interleave with any other topic's members. Added regression test with the exact fixture. P2: DIGEST_DEDUP_MODE typos failed OPEN to embed, not Jaccard. File header documented "non-{embed,jaccard} → jaccard with warn" but readOrchestratorConfig mapped typos to mode='embed'. Operator scenario: during an embed outage sets DIGEST_DEDUP_MODE=jacard — kill switch silently stays off. Fix: typo / unrecognised value resolves to mode='jaccard' (safer), matching the documented contract. invalidModeRaw warn still fires so operators see the typo. Added 3 parsing tests (typo, garbage, empty). Tests: 5914 pass (was 5910 + 4 new). typecheck, typecheck:api, biome clean.
Two P2 bugs from round-3 review: P2a: warn text lied. After the typo→jaccard fix, the warn still said "defaulting to embed", which is the opposite of what the code now does. During an outage, operators reading the warn get told the wrong thing. Updated to "falling back to jaccard (safe rollback path)" to match the actual behavior. P2b: shouldGroupTopics gate used stale signal. Gating on cfg.mode === 'embed' worked for configured-jaccard (kill switch) but NOT for runtime Jaccard fallback. When the embed path throws inside deduplicateStories, it falls back to Jaccard but cfg.mode is still 'embed'. The gate passed, groupTopicsPostDedup ran with an empty embeddingByHash, and the caller emitted a misleading "topic grouping failed: missing embedding" warn ON TOP of the legitimate "falling back to Jaccard" warn. Ground-truth fix: gate on `embeddingByHash.size > 0`. The Map is the authoritative signal for "primary embed path produced vectors" — populated only on success, empty in both fallback paths (configured and runtime). One gate, both paths clean. Added regression test: "runtime Jaccard fallback returns empty embeddingByHash + empty logSummary" — proves the ground-truth invariant the caller relies on, so a future change can't re-introduce the leak. Tests: 5915 pass (+1 new). typecheck, typecheck:api, biome clean.
…ices Greptile P2 on PR #3247: `new Array(top.length)` creates a sparse array. If a future injected clusterFn doesn't cover every input index, topicOf[i] would be undefined, which then silently poisons the phase-1 aggregates (topicSize[undefined] / topicMax[undefined]) and degrades the topic sort without any observable failure. Fill with -1 so absence is unambiguous, then validate after clusterFn runs and throw if any index is still -1. The outer try/catch captures the error and returns {reps: top, topicCount: top.length, error} matching the existing contract — primary order is preserved, no crash. No behavior change today: singleLinkCluster's union-find guarantees every index is covered. This just guards the invariant for future clusterFn injections.
koala73
added a commit
that referenced
this pull request
Apr 21, 2026
Fixes 5 findings from multi-agent review, 2 of them P1: - #241 P1: `.gitignore !api/internal/**` was too broad — it re-included `.env`, `.env.local`, and any future secret file dropped into that directory. Narrowed to explicit source extensions (`*.ts`, `*.js`, `*.mjs`) so parent `.env` / secrets rules stay in effect inside api/internal/. - #242 P1: `Dockerfile.digest-notifications` did not COPY `shared/brief-llm-core.js` + `.d.ts`. Cron would have crashed at container start with ERR_MODULE_NOT_FOUND. Added alongside brief-envelope + brief-filter COPY lines. - #243 P2: Cron dropped the endpoint's source/producedBy ground-truth signal, violating PR #3247's own round-3 memory (feedback_gate_on_ground_truth_not_configured_state.md). Added structured log at the call site: `[brief-llm] whyMatters source=<src> producedBy=<pb> hash=<h>`. Endpoint response now includes `hash` so log + shadow-record pairs can be cross-referenced. - #244 P2: Defense-in-depth prompt-injection hardening. Story fields flowed verbatim into both LLM prompts, bypassing the repo's sanitizeForPrompt convention. Added sanitizeStoryFields helper and applied in both analyst and gemini paths. - #245 P2: Removed redundant `validate` option from callLlmReasoning. With only openrouter configured in prod, a parse-reject walked the provider chain, then fell through to the other path (same provider), then the cron's own fallback (same model) — 3x billing on one reject. Post-call parseWhyMatters check already handles rejection cleanly. Deferred to P3 follow-ups (todos 246-248): singleflight, v2 sunset, misc polish (country-normalize LOC, JSDoc pruning, shadow waitUntil, auto-sync mirror, context-assembly caching). Tests: 6022 pass. typecheck + typecheck:api clean.
koala73
added a commit
that referenced
this pull request
Apr 21, 2026
…nt (#3248) * feat(brief): route whyMatters through internal analyst-context endpoint The brief's "why this is important" callout currently calls Gemini on only {headline, source, threatLevel, category, country} with no live state. The LLM can't know whether a ceasefire is on day 2 or day 50, that IMF flagged >90% gas dependency in UAE/Qatar/Bahrain, or what today's forecasts look like. Output is generic prose instead of the situational analysis WMAnalyst produces when given live context. This PR adds an internal Vercel edge endpoint that reuses a trimmed variant of the analyst context (country-brief, risk scores, top-3 forecasts, macro signals, market data — no GDELT, no digest-search) and ships it through a one-sentence LLM call with the existing WHY_MATTERS_SYSTEM prompt. The endpoint owns its own Upstash cache (v3 prefix, 6h TTL), supports a shadow mode that runs both paths in parallel for offline diffing, and is auth'd via RELAY_SHARED_SECRET. Three-layer graceful degradation (endpoint → legacy Gemini-direct → stub) keeps the brief shipping on any failure. Env knobs: - BRIEF_WHY_MATTERS_PRIMARY=analyst|gemini (default: analyst; typo → gemini) - BRIEF_WHY_MATTERS_SHADOW=0|1 (default: 1; only '0' disables) - BRIEF_WHY_MATTERS_SHADOW_SAMPLE_PCT=0..100 (default: 100) - BRIEF_WHY_MATTERS_ENDPOINT_URL (Railway, optional override) Cache keys: - brief:llm:whymatters:v3:{hash16} — envelope {whyMatters, producedBy, at}, 6h TTL. Endpoint-owned. - brief:llm:whymatters:shadow:v1:{hash16} — {analyst, gemini, chosen, at}, 7d TTL. Fire-and-forget. - brief:llm:whymatters:v2:{hash16} — legacy. Cron's fallback path still reads/writes during the rollout window; expires in ≤24h. Tests: 6022 pass (existing 5915 + 12 core + 36 endpoint + misc). typecheck + typecheck:api + biome on changed files clean. Plan (Codex-approved after 4 rounds): docs/plans/2026-04-21-001-feat-brief-why-matters-analyst-endpoint-plan.md * fix(brief): address /ce:review round 1 findings on PR #3248 Fixes 5 findings from multi-agent review, 2 of them P1: - #241 P1: `.gitignore !api/internal/**` was too broad — it re-included `.env`, `.env.local`, and any future secret file dropped into that directory. Narrowed to explicit source extensions (`*.ts`, `*.js`, `*.mjs`) so parent `.env` / secrets rules stay in effect inside api/internal/. - #242 P1: `Dockerfile.digest-notifications` did not COPY `shared/brief-llm-core.js` + `.d.ts`. Cron would have crashed at container start with ERR_MODULE_NOT_FOUND. Added alongside brief-envelope + brief-filter COPY lines. - #243 P2: Cron dropped the endpoint's source/producedBy ground-truth signal, violating PR #3247's own round-3 memory (feedback_gate_on_ground_truth_not_configured_state.md). Added structured log at the call site: `[brief-llm] whyMatters source=<src> producedBy=<pb> hash=<h>`. Endpoint response now includes `hash` so log + shadow-record pairs can be cross-referenced. - #244 P2: Defense-in-depth prompt-injection hardening. Story fields flowed verbatim into both LLM prompts, bypassing the repo's sanitizeForPrompt convention. Added sanitizeStoryFields helper and applied in both analyst and gemini paths. - #245 P2: Removed redundant `validate` option from callLlmReasoning. With only openrouter configured in prod, a parse-reject walked the provider chain, then fell through to the other path (same provider), then the cron's own fallback (same model) — 3x billing on one reject. Post-call parseWhyMatters check already handles rejection cleanly. Deferred to P3 follow-ups (todos 246-248): singleflight, v2 sunset, misc polish (country-normalize LOC, JSDoc pruning, shadow waitUntil, auto-sync mirror, context-assembly caching). Tests: 6022 pass. typecheck + typecheck:api clean. * fix(brief-why-matters): ctx.waitUntil for shadow write + sanitize legacy fallback Two P2 findings on PR #3248: 1. Shadow record was fire-and-forget without ctx.waitUntil on an Edge function. Vercel can terminate the isolate after response return, so the background redisPipeline write completes unreliably — i.e. the rollout-validation signal the shadow keys were supposed to provide was flaky in production. Fix: accept an optional EdgeContext 2nd arg. Build the shadow promise up front (so it starts executing immediately) then register it with ctx.waitUntil when present. Falls back to plain unawaited execution when ctx is absent (local harness / tests). 2. scripts/lib/brief-llm.mjs legacy fallback path called buildWhyMattersPrompt(story) on raw fields with no sanitization. The analyst endpoint sanitizes before its own prompt build, but the fallback is exactly what runs when the endpoint misses / errors — so hostile headlines / sources reached the LLM verbatim on that path. Fix: local sanitizeStoryForPrompt wrapper imports sanitizeForPrompt from server/_shared/llm-sanitize.js (existing pattern — see scripts/seed-digest-notifications.mjs:41). Wraps story fields before buildWhyMattersPrompt. Cache key unchanged (hash is over raw story), so cache parity with the analyst endpoint's v3 entries is preserved. Regression guard: new test asserts the fallback prompt strips "ignore previous instructions", "### Assistant:" line prefixes, and `<|im_start|>` tokens when injection-crafted fields arrive. Typecheck + typecheck:api clean. 6023 / 6023 data tests pass. * fix(digest-cron): COPY server/_shared/llm-sanitize into digest-notifications image Reviewer P1 on PR #3248: my previous commit (4eee220) added `import sanitizeForPrompt from server/_shared/llm-sanitize.js` to scripts/lib/brief-llm.mjs, but Dockerfile.digest-notifications cherry- picks server/_shared/* files and doesn't copy llm-sanitize. Import is top-level/static — the container would crash at module load with ERR_MODULE_NOT_FOUND the moment seed-digest-notifications.mjs pulls in scripts/lib/brief-llm.mjs. Not just on fallback — every startup. Fix: add `COPY server/_shared/llm-sanitize.js server/_shared/llm-sanitize.d.ts` next to the existing brief-render COPY line. Module is pure string manipulation with zero transitive imports — nothing else needs to land. Cites feedback_validation_docker_ship_full_scripts_dir.md in the comment next to the COPY; the cherry-pick convention keeps biting when new cross-dir imports land in scripts/lib/ or scripts/shared/. Can't regression-test at build time from this branch without a docker-build CI job, but the symptom is deterministic — local runs remain green (they resolve against the live filesystem); only the container crashes. Post-merge, Railway redeploy of seed-digest- notifications should show a clean `Starting Container` log line instead of the MODULE_NOT_FOUND crash my prior commit would have caused.
4 tasks
koala73
added a commit
that referenced
this pull request
Apr 23, 2026
…link (#3331) DIGEST_DEDUP_CLUSTERING previously fell to 'single' on unrecognised values, which silently defeated the documented kill switch. A typo like `DIGEST_DEDUP_CLUSTERING=complet` during an over-merge incident would stick with the aggressive single-link merger instead of rolling back to the conservative complete-link algorithm. Mirror the DIGEST_DEDUP_MODE typo pattern (PR #3247): - Unrecognised value → fall to 'complete' (SAFE / conservative). - Surface the raw value via new `invalidClusteringRaw` config field. - Emit a warn line on the dedup orchestrator's entry path so operators see the typo alongside the kill-switch-took-effect message. Valid values 'single' (default), 'complete', unset, empty, and any case variation all behave unchanged. Only true typos change behaviour — and the new behaviour is the kill-switch-safe one. Tests: updated the existing case that codified the old behaviour plus added coverage for (a) multiple typo variants falling to complete with invalidClusteringRaw set, (b) case-insensitive valid values not triggering the typo path, and (c) the orchestrator emitting the warn line even on the jaccard-kill-switch codepath (since CLUSTERING intent applies to both modes). 81/81 dedup tests pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Secondary clustering pass on already-sliced top-30 dedup reps at a looser cosine (default
0.45) + re-order by a total key:(topicSize DESC, topicMax DESC, repScore DESC, titleHashHex ASC). Dominant thread leads; within-thread order is score DESC; ties are deterministic. Hidden behindDIGEST_DEDUP_TOPIC_GROUPING(default on, kill switch ='0', no deploy).Why
Current
currentScore DESCorder scatters related stories on topic-dominant news days. The 2026-04-20 20:00 brief had 4 Hormuz stories at positions 1/3/8/11 with unrelated events wedged between. User: "if there's a massive news getting coverage the most, and we surfaced it, might as well let it show first."Before (score DESC)
After (topic-grouped)
Louisiana (score 95 singleton) appears after Iran-war-max-91 (3-story thread) — exactly the editorial intent.
Design
slice(0, 30)reps → ~435 cosine comparisons × 512-dim ≈ 0.4 ms. Avoids reshuffling reps that never surface.Map<hash, number[]>returned fromdeduplicateStories— no hidden__embeddingfields on the user-facing Rep (prevents leaking into the brief envelope).groupTopicsPostDedupis pure: no I/O, no logging, injectedclusterFnfor testability. Errors are returned (not thrown) so a helper bug cannot cascade into the outer Jaccard fallback boundary.deduplicateStoriesreturnslogSummary; caller splicestopics=Nbetweenclusters=Mandveto_fires=Vvia a targeted regex, emits one structured log line per tick.Env
DIGEST_DEDUP_TOPIC_GROUPING1'0'disablesDIGEST_DEDUP_TOPIC_THRESHOLD0.45Read at call time. Railway flip takes effect next cron tick. No new Railway env vars required for this PR — defaults are correct.
Log line delta
Before:
After (when grouping enabled AND succeeded):
When
topicErrpresent, a separate warn line is emitted BEFORE the structured log andtopics=Nis omitted:Testing
55 tests in
tests/brief-dedup-embedding.test.mjs(was 33, +22 new):topicMaxtiebreak between same-size topicsrepScore DESCtitleHashHexdeterministic final tiebreak (permutation-stable)topicGroupingEnabled=false→ output === input referenceitems[0](regression guard)JSON.stringify(envelope)contains no_embedding,__-prefixed keys, orembeddingByHash(regression guard against leaking the sidecar into user-facing JSON)'yes','foo','1.5','0','0.55', defaults)Full verification:
npm run test:data→ 5913 pass.typecheck,typecheck:api, biome — all clean. Pre-existingmain()complexity (74) unchanged.Post-Deploy Monitoring & Validation
topics=on the[digest] dedup mode=embed …line (digest-notifications Railway service)ms=field in the same log line; secondary pass should add <2 msrailway logs --service digest-notifications --lines 500 | grep "topics="— expecttopicsin[5, 10]on normal ticks/api/brief/<user>/<slot>— related stories should appear in contiguous blockscurl "/api/brief/<user>/<slot>?…" | grep -c "embedding"→ expect0(envelope leak check)topics=Nline every tick (5–10 typically)[digest] topic grouping failed …warn linesfalling back to Jaccardlines attributable to topic groupingtopics=1) across multiple consecutive ticks → threshold too loose → bumpDIGEST_DEDUP_TOPIC_THRESHOLD=0.50on Railwaytopic grouping failed, preserving primary orderpersists → inspect Sentry + flipDIGEST_DEDUP_TOPIC_GROUPING=0falling back to JaccardWITHtopic grouping failedon same tick → the nested boundary leaked; bug that must be fixedDIGEST_DEDUP_TOPIC_GROUPING=0on Railway digest-notifications service (instant, no deploy).Plan:
docs/plans/2026-04-20-001-feat-topic-grouped-brief-ordering-plan.md(Codex-approved after 3 rounds).🤖 Generated with Claude Opus 4.7 (1M context, extended thinking) via Claude Code