Repository navigation
fix(owner-goals): drop unverifiable quotes from a fresh save instead of refusing it whole - #5992
Conversation
…of refusing it whole A first goals map's answer carries 60-90 verbatim quotes, and verbatim quoting gets less accurate as the window grows, so one inexact quote refused every verified quote beside it and no snapshot was ever saved (thedotmack#5969). Without --carried-before, save_goals.py now drops a quote that fails verification and any item left with none, reports the drops as dropped_unverified_quotes, and still refuses when no item verifies. Updates saved with --carried-before keep the strict refusal for new quotes.
[Medium impact] The PR appears safe to merge; no actionable issues remain. SummaryFresh saves now drop quotes that fail verification and omit items left without evidence.
Reviews (3) · Last reviewed commit: "docs(owner-goals): tell a manual update ..." · Reviewed by Greptile |
Without it, an update run by hand or by an agent takes the fresh-save path and drops a bad new quote instead of refusing it. A test pins that an update with --carried-before still refuses one bad new quote beside a good one. Co-Authored-By: Claude Opus 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_019FbY8c2BSHbdhJhnWYMGaY
|
Pushed a small follow-up on your branch, for the Greptile note on SKILL.md:
|
|
Merged, thank you @0xABCD01! On a first goals map, |
Fixes #5969.
On a first My Work run with
CLAUDE_MEM_WORK_VIEW_AGENT=api, the goals step maps the whole window in one answer, andsave_goals.pythen refused the whole answer when any one of its 60-90 evidence quotes was not a verbatim substring of the prompt at that minute. The reporter measured a 2-18% per-quote miss rate that grows with window size, so a large first run never saved a snapshot, and every later refresh took the same full-window path and failed again.Change: on a fresh save — the refresh passes no
--carried-beforewhen there is no saved snapshot to carry quotes from —save_goals.pynow drops a quote that fails verification, and drops an item left with no quote, instead of refusing every verified quote beside it. The dropped quotes are reported in the save result asdropped_unverified_quotes(item and minute, the same shape asdropped_carried_quotes). Every saved quote still verifies verbatim, and every saved item still holds at least one verified quote. If every item is dropped, the save is still refused with the failing quotes listed, so a wholly unverified answer cannot become an empty snapshot. Updates saved with--carried-beforeare unchanged: a carried quote that no longer matches is dropped, and a newly written quote that fails still refuses the save. The skill's step-3 text is updated to match.Notes:
generated_at, so those weeks would never be mapped. Chronological batching is a larger change and is left out rather than guessed at here.Verified without an API key at the validator level: a synthetic first-run answer with the reporter's full-window numbers — 90 quotes, 74 verbatim and 16 unverifiable, one item supported only by unverifiable quotes — was refused whole by the base script (exit 1, all 16 listed) and is saved by the fixed script with the 74 verified quotes on 29 items, the unsupported item dropped, and 16 drops reported. Tests: the new regression case in
tests/work-view/skill-scripts-python.test.tscovers all three item kinds plus the all-unverified refusal;tests/work-view/passes (280 tests), the agent-runner extraction and shared work-view writer suites pass (12 tests), and the root TypeScript check is clean.