fix: close three ways a day plan could reference work it should not#3596
Conversation
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
|
Warning Review limit reached
Next review available in: 3 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (7)
📝 WalkthroughWalkthroughDaily OS day-plan validation now filters task references by journal visibility, rejects unavailable calendar blocks, and prevents new work from being scheduled in already-passed time. Tool schemas, prompts, tests, and release notes reflect these rules. ChangesDaily OS plan validation
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
All three were found by the day-planning eval and reproduced against a real model before being fixed. decidedTaskIds was a permission set the model wrote itself. The parser accepts a block's taskId when it appears either in decidedTaskIds or in allowedExistingTaskIds — but only the latter was resolved against the journal and filtered by the agent's allowed categories. A model could therefore reference a deleted task, a non-existent one, or one in a category this agent may not touch, simply by echoing the id into its own draft_day_plan call, defeating the check the sibling branch applies. Both sets now go through the same resolution. Legitimate decided tasks are unaffected: they are hydrated for the prompt through hydrateDecidedTasks, which already applies exactly this filter, so the only ids dropped are ones the model was never given. The prompt offered a cal exemption whose precondition can never hold. It told the model that "only cal blocks mirroring real calendar events may span" the current time, while the agent is shown no calendar at all — calendarBlocks is documented as deferred and RealDayAgent drops it. So the exemption was a standing invitation to plan the past, and any cal block that persisted left the user with a block the plan editor refuses to edit and their calendar has never heard of. Closed at all three levels: the parser rejects a model-emitted cal block, cal is gone from both tool schemas, and the prompt no longer promises it. All three come back together whenever calendar events are actually wired into the drafting context. Block state bypassed the same-day guard. The guard fired only for state == drafted, so writing a new block as committed placed work before the current time — the same probing as the earlier trick of relabelling a past-starting block buffer, one field over. The guard now keys on what a state means: drafted and committed are plans and are guarded; inProgress, completed and dropped are history a re-draft legitimately carries forward.
45c8642 to
f26cb02
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 45c8642353
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codecov Report❌ Patch coverage is
❌ Your patch check has failed because the patch coverage (90.27%) is below the target coverage (99.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #3596 +/- ##
==========================================
- Coverage 99.15% 99.15% -0.01%
==========================================
Files 1784 1784
Lines 130978 131033 +55
==========================================
+ Hits 129872 129920 +48
- Misses 1106 1113 +7
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Review found the first fix half-done: it closed the draft route and left the refine route open, which is the same mistake twice rather than two mistakes. propose_plan_diff accepted `to.taskId` with no validation at all — never resolved, never category-checked — and acceptPlanDiff copies it onto the live plan. The approval summary the user reads carries the title and times but not the task reference, so an approved diff could attach a deleted task, a non-existent one, or one from a category this agent may not touch, and the user would never have seen it described. Both routes now share one resolveAllowedTaskIds; two implementations with only one of them resolving the id is what produced the original hole. The same asymmetry applied to `cal`: removing it from the tool schema does not stop a diff supplying it, and optionalBlockSnapshot happily parsed it. The diff path now refuses it with the same reasoning, and for the same user-facing consequence — an approved block whose type was never shown and which the plan editor afterwards refuses to edit. And a regression I introduced: a legacy `agreed` plan can hold `committed` blocks the user already approved, so once one had started, guarding `committed` rejected the entire redraft for faithfully repeating what was already on the plan. A block that matches a baseline block's id *and* start is carried forward and exempt; reusing a known id at a new time is still planning the past.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 55bf5fa1eb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…he request arrives Review found the same shape three more times: a check placed at the point a request is parsed, while the data reaches the plan somewhere else entirely. A ChangeSet is durable and synced, so proposal-time validation does not bind. It can be accepted long after it was written — by which point the task may have been deleted or moved out of the agent's categories, and the part of the day it aimed at may already have passed — or arrive from a peer on an older build that never ran the check at all. resolvePlanDiff now re-resolves task references against the journal and re-derives the clock at apply time, and the apply-path batch validator refuses `cal` and past starts as well as the categories and timestamps it already checked. propose_plan_diff gains the past-start guard too, so refining today's plan cannot place work in a part of the day that has gone, any more than drafting it can. Carrying a baseline block forward now requires the same state as well as the same id and start. Matching id and start alone let a known 09:00 id be reused to drop a brand-new committed block into that slot, rewriting approved work without the refinement approval that gates it. Deliberately not tightened: a past-starting block in a *history* state still needs no baseline. The first draft of a day can happen in the afternoon, and the capture is where the agent learns what the morning held, so requiring baseline evidence would forbid recording real work. Whether a model should be able to invent that history is a live question the eval already measures; it is a product decision about what a plan may record, not one to settle by tightening a guard mid-fix.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 860a7a21a1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
The past-start guard on the refine path was too blunt. A full `to` snapshot carries the block's unchanged original start, so extending the end of a block that began at 09:00 was refused at noon for saying 09:00 — and a pending change accepted after its block started hit the same wall on the apply path. Both sites now compare against the live block and refuse only an actual relocation. `add_block` has no live block, so any past start there is still a genuine placement into the past. Also documents the state match a carried-forward block requires, which the README described as id and start alone.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c6bc2390d7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| Preserve already-started baseline blocks only when they represent existing | ||
| in-progress, completed, or dropped history. |
There was a problem hiding this comment.
Tell the planner to retain committed baseline blocks
On a same-day drafting wake over a legacy agreed plan, the baseline may contain an already-started committed block. The new parser exception expressly accepts that block when its id, start, and state match, but this prompt still tells the model to preserve past baseline blocks only when they are inProgress, completed, or dropped. Because draft_day_plan replaces the full block list, following this instruction omits and silently deletes previously approved committed work; include unchanged committed baseline blocks in the preservation rule.
Useful? React with 👍 / 👎.
Closes
lotti3-80m,lotti3-3r1andlotti3-abp. All three were found by the day-planning eval, reproduced against a real model, and are verified against one again here.1.
decidedTaskIdswas a permission set the model wrote itselfparsePlannedBlockaccepts a block'staskIdwhen it appears in eitherdecidedTaskIdsorallowedExistingTaskIds. Only the second was resolved:while
decidedTaskIdscame straight off the wire —stringListArg(args['decidedTaskIds'])in the tool dispatcher — and went into the parser untouched. So a model could reference a deleted task, a non-existent one, or one in a category this agent is not allowed to touch, just by echoing the id into its owndraft_day_plancall. That is the category check the sibling branch applies being bypassed one line over.Both sets now go through the same
_resolveTaskIds. Legitimate decided tasks are unaffected: they are hydrated for the prompt throughhydrateDecidedTasks, which already applies exactly this filter, so the only ids dropped are ones the model was never given.The test churn is the clearest evidence of what changed — three existing tests failed because they had encoded
decidedTaskIdsas trusted, and fixing them meant telling the journal those tasks exist, which is what production now requires.2. The prompt offered a
calexemption whose precondition can never holdThe drafting rules said "only
calblocks mirroring real calendar events may span" the current time. The day agent is shown no calendar at all:DayAgentInterface.draftDayPlandocumentscalendarBlocksas deferred,RealDayAgentdrops the argument, and no context section renders an event. So the exemption was a standing invitation to plan the past with a type that could never be legitimate — and anycalblock that persisted left the user with a blockDayAgentPlanEditorrefuses to edit ("edit it in the source calendar") which their calendar has never heard of.Closed at all three levels, because closing one would have left the other two lying: the parser rejects a model-emitted
calblock,calis removed from both tool schemas so the option is not offered, and the prompt no longer promises it. The parser comment records that all three come back together when calendar events are actually wired into the drafting context.3. Block
statebypassed the same-day guardThe guard fired only for
state == drafted, so writing a new block ascommittedplaced work before the current time. Same probing as the earlier trick of relabelling a past-starting blockbufferto slip an ai/manual-only guard — one field over.It now keys on what a state means rather than enumerating types:
inProgress,completedanddroppedstay exempt — they are history a re-draft legitimately carries forward, which is what the original exemption existed for.Verified against a live model
Ran the eval before and after. Overall 96% (89/93), the best of six runs, and the one remaining rejection is the new guard firing on
lateStart:blocks planned for today must not start before current time.Then three samples of
bindingDirective, the scenario where the abuse originally showed up. The model still wants to represent the directive's 60 already-scheduled minutes as a block — but now writes it asbuffer/committedrather thancal, and that persists correctly: it is a future-day plan so the past-start guard is legitimately inert, and a committed buffer claims neither a calendar import nor fabricated history. The fix removed the harmful expression while leaving a legitimate one available, which is the outcome you want from a guard.Two honest caveats:
calattempt appears in any post-fix run, so there is no live evidence of the parser rejection firing. That is expected rather than lucky —calis gone from the schema, so the model cannot emit it; the parser check is defence in depth for the diff path and future callers. The parser test proves that branch.noFabricatedCalendarBlocksandnoHistoryFabricationread 100%, but they also read 100% in the last pre-fix run — the behaviour was intermittent. Those constraints now cannot fail by construction, which is a stronger statement than a green run, and the construction is what the unit tests assert.Verification
fvm dart analyzeclean; 2115 tests indaily_os_next.committedis guarded, and acalblock is refused with or without the guard active.Summary by CodeRabbit