fix(brief): date-ground the LLM system prompts (plan F6 / PR D) by koala73 · Pull Request #3692 · koala73/worldmonitor

koala73 · 2026-05-14T11:48:39Z

Summary

PR D of the brief-pipeline plan (docs/plans/2026-05-14-001-…), Phase 5 / Finding F6 — date-grounds the brief's LLM system prompts. Independent of PRs A/B/C; branched from main.

Every brief LLM system prompt — digest synthesis, the whyMatters analyst path, and the whyMatters legacy/fallback path — is a static string with no notion of "now". With no date anchor the model fills date/year gaps from training-data priors: a May 2026 brief shipped a whyMatters claiming a deploy "in 2024". The proper-noun grounding gate (#3667) is anchor-based and does not catch numeric/date fabrication.

Changes

New shared helper briefDateLine(todayIso?) in shared/brief-llm-core.js — returns Today is YYYY-MM-DD. Do not state any year or date that contradicts the dates in the stories below; when a date is not given, omit it rather than guess.
Appended to the system prompt by all three prompt builders:
- buildDigestPrompt (DIGEST_PROSE_SYSTEM_BASE)
- buildWhyMattersUserPrompt (WHY_MATTERS_SYSTEM — legacy / cron + edge fallback)
- buildAnalystWhyMattersPrompt (WHY_MATTERS_ANALYST_SYSTEM_V2 — live analyst path)
todayIso is injectable for deterministic tests; production callers pass nothing and get the current UTC date — fully backward compatible, no production call-site changes. A malformed override falls back to today rather than interpolating garbage.
Mirrored byte-for-byte into scripts/shared/brief-llm-core.{js,d.ts} per the file's edge-mirror contract.

Deliberately out of scope

Cache keys are not touched. The date line guards against contradictory years — it does not emit a literal date into the prose — so a cache hit across midnight is still correctly grounded. The per-run story-pool hash already rotates cache keys; adding the date would only force a daily all-rows miss (extra LLM spend) for no correctness gain.

Test plan

briefDateLine unit tests — injected date verbatim + malformed-input fallback to today
Updated the 2 assert.equal(system, …) assertions broken by the appended line; added date-line presence assertions for all 3 builders
404 brief/digest tests pass under tsx --test (incl. brief-edge-route-smoke, brief-magazine-render, digest-orchestration-helpers, digest-cache-key-sensitivity)
typecheck:api clean, biome lint clean
shared/ ↔ scripts/shared/ mirror byte-identical

vercel · 2026-05-14T11:48:44Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
worldmonitor	Ready	Preview, Comment	May 14, 2026 2:16pm

greptile-apps · 2026-05-14T11:52:50Z

Greptile Summary

This PR date-grounds all three brief LLM system prompts (buildDigestPrompt, buildWhyMattersUserPrompt, and buildAnalystWhyMattersPrompt) by appending a new shared briefDateLine helper that emits a "Today is YYYY-MM-DD…" instruction with a strict no-fabrication rule. The helper accepts an injectable todayIso for deterministic tests while defaulting to the current UTC date in production.

briefDateLine in shared/brief-llm-core.js: new pure helper with format-regex validation and a safe fallback to new Date(); mirrored byte-for-byte to scripts/shared/brief-llm-core.js per the edge-mirror contract.
Three prompt builders updated: buildDigestPrompt (digest cron), buildWhyMattersUserPrompt (legacy/edge fallback), and buildAnalystWhyMattersPrompt (live analyst path) each append the date line to their system prompt; all changes are backward-compatible (optional parameter, no production call-site edits required).
Tests: briefDateLine unit tests cover injected-date verbatim use and malformed-input fallback; all three builder tests updated with deterministic todayIso injection.

Confidence Score: 4/5

Safe to merge; all production paths are backward-compatible and no call sites change.

The implementation is clean, the mirror parity is verified, and test coverage is thorough. Two minor test-quality issues exist: a UTC-midnight timing race in the fallback assertions, and a dynamic import inside a test body where a static import would be simpler. Neither affects production behaviour.

tests/brief-llm-core.test.mjs — the fallback-date assertion loop has a latent midnight-UTC flake worth fixing before it shows up as a spurious CI failure.

Important Files Changed

Filename	Overview
shared/brief-llm-core.js	Adds `briefDateLine` helper and injects it into `buildWhyMattersUserPrompt`; format-only date regex accepts calendar-invalid values, but impact is limited to test injection only.
scripts/shared/brief-llm-core.js	Byte-for-byte mirror of shared/brief-llm-core.js (confirmed by diff); all changes are identical.
scripts/lib/brief-llm.mjs	Imports and appends `briefDateLine` in `buildDigestPrompt`; uses `ctx?.todayIso` where optional chaining is redundant because `ctx` defaults to `{}`.
server/worldmonitor/intelligence/v1/brief-why-matters-prompt.ts	Adds optional `todayIso` to `buildAnalystWhyMattersPrompt` and appends the date-grounding line; backward-compatible change with no impact on existing callers.
tests/brief-llm-core.test.mjs	Adds `briefDateLine` unit tests including malformed-input fallback; the fallback test captures `today` before calling `briefDateLine`, introducing a theoretical UTC-midnight flake.
tests/brief-llm.test.mjs	Tests `buildDigestPrompt` date-grounding; uses a dynamic import inside the test body to get `briefDateLine` when a static import would suffice.
tests/brief-why-matters-analyst.test.mjs	Updated assertion to include the date-grounding suffix in the expected system prompt; passes deterministic `todayIso` for stability.
shared/brief-llm-core.d.ts	Adds `briefDateLine` export and optional `todayIso` parameter to `buildWhyMattersUserPrompt`; types match implementation.
scripts/shared/brief-llm-core.d.ts	Mirror of shared/brief-llm-core.d.ts; identical to the canonical declaration file.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[briefDateLine todayIso?] --> B{valid YYYY-MM-DD?}
    B -- yes --> C[use injected date]
    B -- no / absent --> D[new Date UTC slice 0,10]
    C --> E[Today is ISO. Do not state...]
    D --> E

    E --> F[buildDigestPrompt\nDIGEST_PROSE_SYSTEM_BASE + date line]
    E --> G[buildWhyMattersUserPrompt\nWHY_MATTERS_SYSTEM + date line]
    E --> H[buildAnalystWhyMattersPrompt\nWHY_MATTERS_ANALYST_SYSTEM_V2 + date line]

    F --> I[LLM — digest prose]
    G --> J[LLM — legacy whyMatters]
    H --> K[LLM — analyst whyMatters]

_{Reviews (1): Last reviewed commit: "fix(brief): date-ground the LLM system p..." | Re-trigger Greptile}

greptile-apps · 2026-05-14T11:52:54Z

+  it('falls back to the current UTC date for missing / malformed input', () => {
+    const today = new Date().toISOString().slice(0, 10);
+    for (const bad of [undefined, null, '', 'not-a-date', '2026/05/14', 14]) {
+      assert.match(
+        briefDateLine(bad),
+        new RegExp(`^Today is ${today}\\.`),
+        `malformed input ${JSON.stringify(bad)} must fall back to today`,
+      );
+    }
+  });


UTC-midnight test flake

today is captured once before the loop, but each briefDateLine(bad) call inside the loop independently calls new Date(). If the test suite happens to execute these lines across a UTC-midnight boundary, the two new Date() calls can return different calendar days and every assertion in the loop will fail with a date mismatch. The window is tiny but reproducible in CI pipelines that run near midnight UTC.

greptile-apps · 2026-05-14T11:52:55Z

+  // explicit date the model fabricates years (a May 2026 brief shipped
+  // a "deploy ... in 2024" line). briefDateLine pins the current date.
+  return {
+    system: `${DIGEST_PROSE_SYSTEM_BASE}\n${briefDateLine(ctx?.todayIso)}`,


ctx is declared with a default parameter of {}, so it is never null or undefined inside the function body. The ?. optional-chaining operator on ctx is harmless but unnecessary and signals to readers that ctx might be absent at this point, which it cannot be.

Suggested change

system: `${DIGEST_PROSE_SYSTEM_BASE}\n${briefDateLine(ctx?.todayIso)}`,

system: `${DIGEST_PROSE_SYSTEM_BASE}\n${briefDateLine(ctx.todayIso)}`,

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Keeping ctx?.todayIso as-is: buildDigestPrompt already uses ctx?.isPublic, ctx?.profile, and ctx?.greeting a few lines up. A lone bare ctx.todayIso would be inconsistent within the same function — local convention wins over the micro-optimization here. The other two comments (midnight flake, dynamic import) are fixed in 666b060.

greptile-apps · 2026-05-14T11:52:56Z

+  it('appends the date-grounding line to the system prompt (F6)', async () => {
+    const { briefDateLine } = await import('../shared/brief-llm-core.js');


The async keyword and dynamic import() are used here solely to retrieve briefDateLine. A static top-level import avoids the extra async ceremony and keeps the test body synchronous, which is easier to read and marginally faster.

Suggested change

it('appends the date-grounding line to the system prompt (F6)', async () => {

const { briefDateLine } = await import('../shared/brief-llm-core.js');

it('appends the date-grounding line to the system prompt (F6)', () => {

const { briefDateLine } = require('../shared/brief-llm-core.js'); // or hoist to top-level static import

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

…eploy PR #3692 review (High). The date-grounding line is appended to the prompts, but the cache keys were left unchanged — so on a cache hit all three paths return pre-F6 prose that was generated WITHOUT the date line and may contain exactly the fabricated year (e.g. "2024" in a May 2026 brief) this PR exists to prevent. Stale rows would keep shipping for the full TTL: 24h (fallback whyMatters), 6h (edge whyMatters), 4h (digest). `validateDigestProseShape` revalidates digest cache hits, but its grounding gate is proper-noun based and does NOT catch date/numeric fabrication — a stale v5 row re-passes and ships. Bump (same mechanism #3667 used v4→v5 for the grounding gate): - brief:llm:whymatters:v3 → v4 (Railway fallback, brief-llm.mjs) - brief:llm:digest:v5 → v6 (digest synthesis, brief-llm.mjs) - brief:llm:whymatters:v7 → v8 (edge/analyst, brief-why-matters.ts) - brief:llm:whymatters:shadow:v5 → v6 (shadow record, same reason) Prefix bump (not adding the date to the hash material) is the right mechanism: it forces a one-time cold-start through the date-grounded prompt on first tick after deploy, then steady-state resumes — no daily all-rows cache churn. Also refreshed the now-doubly-stale cache-version references in the brief-llm.mjs header block and the generateWhyMatters JSDoc, and the test assertions pinning the old prefixes. 243 brief/digest tests pass under tsx --test; typecheck:api + biome clean.

…c import Greptile P2 review: - brief-llm-core.test.mjs / brief-llm.test.mjs fallback assertions captured `today` once but briefDateLine() reads new Date() inside the call — a UTC-midnight rollover mid-test desynced the two. Now bracket each call with before/after and accept either; the date is parsed out of the line rather than string-built into a regexp. - brief-llm.test.mjs: replaced the dynamic `await import()` of briefDateLine with a static top-level import; the test is sync again. The third comment (redundant `ctx?.todayIso` in buildDigestPrompt) is left as-is: the same function already uses `ctx?.isPublic`, `ctx?.profile`, `ctx?.greeting` — matching that local convention beats a lone bare `ctx.todayIso`. Replied on the thread. 178 brief tests pass; biome clean.

The brief's source stories are dated, but every LLM system prompt (digest synthesis, whyMatters analyst path, whyMatters legacy path) is a static string with no notion of "now". With no anchor the model fills date/year gaps from training-data priors — a May 2026 brief shipped a whyMatters claiming a deploy "in 2024". The proper-noun grounding gate (#3667) is anchor-based and does not catch numeric/date fabrication. Fix: a shared `briefDateLine(todayIso?)` helper in brief-llm-core.js returns "Today is YYYY-MM-DD. Do not state any year or date that contradicts the dates in the stories below…". It is appended to the system prompt by all three prompt builders: - buildDigestPrompt (DIGEST_PROSE_SYSTEM_BASE) - buildWhyMattersUserPrompt (WHY_MATTERS_SYSTEM, legacy/fallback) - buildAnalystWhyMattersPrompt (WHY_MATTERS_ANALYST_SYSTEM_V2) `todayIso` is injectable for deterministic tests; production callers pass nothing and get the current UTC date — fully backward compatible, no production call site changes. A malformed override falls back to today rather than interpolating garbage. Cache keys are deliberately NOT touched. The date line guards against *contradictory* years, it does not emit a literal date into the prose, so a cache hit across midnight is still correctly grounded. The per-run story-pool hash already rotates cache keys; adding the date would only force a daily all-rows miss (extra LLM spend) for no correctness gain. briefDateLine is mirrored byte-for-byte into scripts/shared/ per the file's edge-mirror contract. 404 brief/digest tests pass under tsx --test; typecheck:api and biome lint clean.

…eploy PR #3692 review (High). The date-grounding line is appended to the prompts, but the cache keys were left unchanged — so on a cache hit all three paths return pre-F6 prose that was generated WITHOUT the date line and may contain exactly the fabricated year (e.g. "2024" in a May 2026 brief) this PR exists to prevent. Stale rows would keep shipping for the full TTL: 24h (fallback whyMatters), 6h (edge whyMatters), 4h (digest). `validateDigestProseShape` revalidates digest cache hits, but its grounding gate is proper-noun based and does NOT catch date/numeric fabrication — a stale v5 row re-passes and ships. Bump (same mechanism #3667 used v4→v5 for the grounding gate): - brief:llm:whymatters:v3 → v4 (Railway fallback, brief-llm.mjs) - brief:llm:digest:v5 → v6 (digest synthesis, brief-llm.mjs) - brief:llm:whymatters:v7 → v8 (edge/analyst, brief-why-matters.ts) - brief:llm:whymatters:shadow:v5 → v6 (shadow record, same reason) Prefix bump (not adding the date to the hash material) is the right mechanism: it forces a one-time cold-start through the date-grounded prompt on first tick after deploy, then steady-state resumes — no daily all-rows cache churn. Also refreshed the now-doubly-stale cache-version references in the brief-llm.mjs header block and the generateWhyMatters JSDoc, and the test assertions pinning the old prefixes. 243 brief/digest tests pass under tsx --test; typecheck:api + biome clean.

…c import Greptile P2 review: - brief-llm-core.test.mjs / brief-llm.test.mjs fallback assertions captured `today` once but briefDateLine() reads new Date() inside the call — a UTC-midnight rollover mid-test desynced the two. Now bracket each call with before/after and accept either; the date is parsed out of the line rather than string-built into a regexp. - brief-llm.test.mjs: replaced the dynamic `await import()` of briefDateLine with a static top-level import; the test is sync again. The third comment (redundant `ctx?.todayIso` in buildDigestPrompt) is left as-is: the same function already uses `ctx?.isPublic`, `ctx?.profile`, `ctx?.greeting` — matching that local convention beats a lone bare `ctx.todayIso`. Replied on the thread. 178 brief tests pass; biome clean.

vercel Bot deployed to Preview May 14, 2026 11:52 View deployment

greptile-apps Bot reviewed May 14, 2026

View reviewed changes

vercel Bot deployed to Preview May 14, 2026 12:21 View deployment

koala73 added 3 commits May 14, 2026 18:11

koala73 force-pushed the fix/brief-date-grounding branch from 666b060 to cf0178f Compare May 14, 2026 14:13

vercel Bot deployed to Preview May 14, 2026 14:16 View deployment

koala73 merged commit 9262d39 into main May 14, 2026
11 checks passed

koala73 deleted the fix/brief-date-grounding branch May 14, 2026 14:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(brief): date-ground the LLM system prompts (plan F6 / PR D)#3692

fix(brief): date-ground the LLM system prompts (plan F6 / PR D)#3692
koala73 merged 3 commits into
mainfrom
fix/brief-date-grounding

koala73 commented May 14, 2026

Uh oh!

vercel Bot commented May 14, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented May 14, 2026

Uh oh!

greptile-apps Bot May 14, 2026

Uh oh!

greptile-apps Bot May 14, 2026

Uh oh!

koala73 May 14, 2026

Uh oh!

greptile-apps Bot May 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	system: `${DIGEST_PROSE_SYSTEM_BASE}\n${briefDateLine(ctx?.todayIso)}`,
	system: `${DIGEST_PROSE_SYSTEM_BASE}\n${briefDateLine(ctx.todayIso)}`,

		it('appends the date-grounding line to the system prompt (F6)', async () => {
		const { briefDateLine } = await import('../shared/brief-llm-core.js');

Conversation

koala73 commented May 14, 2026

Summary

Changes

Deliberately out of scope

Test plan

Uh oh!

vercel Bot commented May 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps Bot commented May 14, 2026

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Flowchart

Uh oh!

greptile-apps Bot May 14, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot May 14, 2026

Choose a reason for hiding this comment

Uh oh!

koala73 May 14, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot May 14, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented May 14, 2026 •

edited

Loading