Skip to content

fix: close three ways a day plan could reference work it should not#3596

Merged
matthiasn merged 4 commits into
mainfrom
fix/day-agent-plan-reference-guards
Jul 25, 2026
Merged

fix: close three ways a day plan could reference work it should not#3596
matthiasn merged 4 commits into
mainfrom
fix/day-agent-plan-reference-guards

Conversation

@matthiasn

@matthiasn matthiasn commented Jul 25, 2026

Copy link
Copy Markdown
Owner

Closes lotti3-80m, lotti3-3r1 and lotti3-abp. All three were found by the day-planning eval, reproduced against a real model, and are verified against one again here.

1. decidedTaskIds was a permission set the model wrote itself

parsePlannedBlock accepts a block's taskId when it appears in either decidedTaskIds or allowedExistingTaskIds. Only the second was resolved:

final entities = await journalDb.journalEntityMapForIds(referenced.toList());
return { for (final entry in entities.entries)
  if (entry.value is Task && ...deletedAt == null &&
      categoryAllowed((entry.value as Task).meta.categoryId, allowedCategoryIds))
    entry.key };

while decidedTaskIds came straight off the wire — stringListArg(args['decidedTaskIds']) in the tool dispatcher — and went into the parser untouched. So a model could reference a deleted task, a non-existent one, or one in a category this agent is not allowed to touch, just by echoing the id into its own draft_day_plan call. That is the category check the sibling branch applies being bypassed one line over.

Both sets now go through the same _resolveTaskIds. Legitimate decided tasks are unaffected: they are hydrated for the prompt through hydrateDecidedTasks, which already applies exactly this filter, so the only ids dropped are ones the model was never given.

The test churn is the clearest evidence of what changed — three existing tests failed because they had encoded decidedTaskIds as trusted, and fixing them meant telling the journal those tasks exist, which is what production now requires.

2. The prompt offered a cal exemption whose precondition can never hold

The drafting rules said "only cal blocks mirroring real calendar events may span" the current time. The day agent is shown no calendar at all: DayAgentInterface.draftDayPlan documents calendarBlocks as deferred, RealDayAgent drops the argument, and no context section renders an event. So the exemption was a standing invitation to plan the past with a type that could never be legitimate — and any cal block that persisted left the user with a block DayAgentPlanEditor refuses to edit ("edit it in the source calendar") which their calendar has never heard of.

Closed at all three levels, because closing one would have left the other two lying: the parser rejects a model-emitted cal block, cal is removed from both tool schemas so the option is not offered, and the prompt no longer promises it. The parser comment records that all three come back together when calendar events are actually wired into the drafting context.

3. Block state bypassed the same-day guard

The guard fired only for state == drafted, so writing a new block as committed placed work before the current time. Same probing as the earlier trick of relabelling a past-starting block buffer to slip an ai/manual-only guard — one field over.

It now keys on what a state means rather than enumerating types:

const plannedBlockStatesGuardedFromThePast = {drafted, committed};

inProgress, completed and dropped stay exempt — they are history a re-draft legitimately carries forward, which is what the original exemption existed for.

Verified against a live model

Ran the eval before and after. Overall 96% (89/93), the best of six runs, and the one remaining rejection is the new guard firing on lateStart: blocks planned for today must not start before current time.

Then three samples of bindingDirective, the scenario where the abuse originally showed up. The model still wants to represent the directive's 60 already-scheduled minutes as a block — but now writes it as buffer/committed rather than cal, and that persists correctly: it is a future-day plan so the past-start guard is legitimately inert, and a committed buffer claims neither a calendar import nor fabricated history. The fix removed the harmful expression while leaving a legitimate one available, which is the outcome you want from a guard.

Two honest caveats:

  • No cal attempt appears in any post-fix run, so there is no live evidence of the parser rejection firing. That is expected rather than lucky — cal is gone from the schema, so the model cannot emit it; the parser check is defence in depth for the diff path and future callers. The parser test proves that branch.
  • noFabricatedCalendarBlocks and noHistoryFabrication read 100%, but they also read 100% in the last pre-fix run — the behaviour was intermittent. Those constraints now cannot fail by construction, which is a stronger statement than a green run, and the construction is what the unit tests assert.

Verification

  • fvm dart analyze clean; 2115 tests in daily_os_next.
  • New parser tests: only history states are exempt from the past-start guard, committed is guarded, and a cal block is refused with or without the guard active.
  • CHANGELOG and flatpak metainfo updated — all three are user-visible.

Summary by CodeRabbit

  • Bug Fixes
    • Improved daily day-plans by resolving task references only against what the agent is allowed to see, preventing attachment to inaccessible categories and references to deleted/unavailable tasks.
    • Eliminated “phantom” imported calendar blocks when calendar context isn’t available.
    • Prevented planning blocks from being placed in the already-passed portion of today, while allowing only genuinely started/finished/dropped items to remain earlier.
  • Tests
    • Updated validation and tool schema expectations for the tightened day-plan parsing rules.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@coderabbitai

coderabbitai Bot commented Jul 25, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@matthiasn, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 3 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a1187e50-fddd-4d76-b342-5915d1f4639d

📥 Commits

Reviewing files that changed from the base of the PR and between f26cb02 and c6bc239.

📒 Files selected for processing (7)
  • lib/features/daily_os_next/README.md
  • lib/features/daily_os_next/agents/service/day_agent_plan_diff.dart
  • lib/features/daily_os_next/agents/service/day_agent_plan_editor.dart
  • lib/features/daily_os_next/agents/service/day_agent_plan_parser.dart
  • lib/features/daily_os_next/agents/service/day_agent_plan_writer.dart
  • test/features/daily_os_next/agents/service/day_agent_plan_parser_test.dart
  • test/features/daily_os_next/agents/service/day_agent_plan_service_test.dart
📝 Walkthrough

Walkthrough

Daily OS day-plan validation now filters task references by journal visibility, rejects unavailable calendar blocks, and prevents new work from being scheduled in already-passed time. Tool schemas, prompts, tests, and release notes reflect these rules.

Changes

Daily OS plan validation

Layer / File(s) Summary
Task visibility during plan persistence
lib/features/daily_os_next/agents/service/day_agent_plan_writer.dart, test/features/daily_os_next/agents/service/day_agent_plan_service_test.dart
Decided and referenced task IDs are resolved through journal entities and allowed categories before persistence; tests seed resolvable task entities.
Calendar and past-start validation
lib/features/daily_os_next/agents/service/day_agent_plan_parser.dart, lib/features/daily_os_next/agents/tools/day_agent_tools.dart, lib/features/daily_os_next/agents/workflow/day_agent_prompt_builder.dart, test/features/daily_os_next/agents/service/day_agent_plan_parser_test.dart, test/features/daily_os_next/agents/tools/day_agent_tools_test.dart
Calendar blocks are rejected, only in-progress/completed/dropped blocks may start before the current-time boundary, and schemas, prompts, and tests use the updated constraints.
Release notes
CHANGELOG.md, flatpak/com.matthiasn.lotti.metainfo.xml
Version 0.9.1069 notes describe task visibility, phantom calendar, and past-scheduling fixes.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: it summarizes the three fixes that prevent day plans from referencing invalid work.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/day-agent-plan-reference-guards

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

All three were found by the day-planning eval and reproduced against a
real model before being fixed.

decidedTaskIds was a permission set the model wrote itself. The parser
accepts a block's taskId when it appears either in decidedTaskIds or in
allowedExistingTaskIds — but only the latter was resolved against the
journal and filtered by the agent's allowed categories. A model could
therefore reference a deleted task, a non-existent one, or one in a
category this agent may not touch, simply by echoing the id into its
own draft_day_plan call, defeating the check the sibling branch
applies. Both sets now go through the same resolution. Legitimate
decided tasks are unaffected: they are hydrated for the prompt through
hydrateDecidedTasks, which already applies exactly this filter, so the
only ids dropped are ones the model was never given.

The prompt offered a cal exemption whose precondition can never hold.
It told the model that "only cal blocks mirroring real calendar events
may span" the current time, while the agent is shown no calendar at
all — calendarBlocks is documented as deferred and RealDayAgent drops
it. So the exemption was a standing invitation to plan the past, and
any cal block that persisted left the user with a block the plan
editor refuses to edit and their calendar has never heard of. Closed at
all three levels: the parser rejects a model-emitted cal block, cal is
gone from both tool schemas, and the prompt no longer promises it. All
three come back together whenever calendar events are actually wired
into the drafting context.

Block state bypassed the same-day guard. The guard fired only for
state == drafted, so writing a new block as committed placed work
before the current time — the same probing as the earlier trick of
relabelling a past-starting block buffer, one field over. The guard now
keys on what a state means: drafted and committed are plans and are
guarded; inProgress, completed and dropped are history a re-draft
legitimately carries forward.
@matthiasn
matthiasn force-pushed the fix/day-agent-plan-reference-guards branch from 45c8642 to f26cb02 Compare July 25, 2026 21:44

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 45c8642353

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lib/features/daily_os_next/agents/tools/day_agent_tools.dart
@codecov

codecov Bot commented Jul 25, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.27778% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 99.15%. Comparing base (4acf2c2) to head (c6bc239).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
...ly_os_next/agents/service/day_agent_plan_diff.dart 75.00% 5 Missing ⚠️
..._os_next/agents/service/day_agent_plan_editor.dart 88.23% 2 Missing ⚠️

❌ Your patch check has failed because the patch coverage (90.27%) is below the target coverage (99.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3596      +/-   ##
==========================================
- Coverage   99.15%   99.15%   -0.01%     
==========================================
  Files        1784     1784              
  Lines      130978   131033      +55     
==========================================
+ Hits       129872   129920      +48     
- Misses       1106     1113       +7     
Flag Coverage Δ
glados 14.07% <26.38%> (+<0.01%) ⬆️
standard 98.90% <90.27%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Review found the first fix half-done: it closed the draft route and
left the refine route open, which is the same mistake twice rather
than two mistakes.

propose_plan_diff accepted `to.taskId` with no validation at all —
never resolved, never category-checked — and acceptPlanDiff copies it
onto the live plan. The approval summary the user reads carries the
title and times but not the task reference, so an approved diff could
attach a deleted task, a non-existent one, or one from a category this
agent may not touch, and the user would never have seen it described.
Both routes now share one resolveAllowedTaskIds; two implementations
with only one of them resolving the id is what produced the original
hole.

The same asymmetry applied to `cal`: removing it from the tool schema
does not stop a diff supplying it, and optionalBlockSnapshot happily
parsed it. The diff path now refuses it with the same reasoning, and
for the same user-facing consequence — an approved block whose type was
never shown and which the plan editor afterwards refuses to edit.

And a regression I introduced: a legacy `agreed` plan can hold
`committed` blocks the user already approved, so once one had started,
guarding `committed` rejected the entire redraft for faithfully
repeating what was already on the plan. A block that matches a
baseline block's id *and* start is carried forward and exempt; reusing
a known id at a new time is still planning the past.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 55bf5fa1eb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lib/features/daily_os_next/agents/service/day_agent_plan_diff.dart
Comment thread lib/features/daily_os_next/agents/service/day_agent_plan_parser.dart Outdated
Comment thread lib/features/daily_os_next/agents/service/day_agent_plan_parser.dart Outdated
…he request arrives

Review found the same shape three more times: a check placed at the
point a request is parsed, while the data reaches the plan somewhere
else entirely.

A ChangeSet is durable and synced, so proposal-time validation does not
bind. It can be accepted long after it was written — by which point the
task may have been deleted or moved out of the agent's categories, and
the part of the day it aimed at may already have passed — or arrive
from a peer on an older build that never ran the check at all.
resolvePlanDiff now re-resolves task references against the journal and
re-derives the clock at apply time, and the apply-path batch validator
refuses `cal` and past starts as well as the categories and timestamps
it already checked.

propose_plan_diff gains the past-start guard too, so refining today's
plan cannot place work in a part of the day that has gone, any more
than drafting it can.

Carrying a baseline block forward now requires the same state as well
as the same id and start. Matching id and start alone let a known 09:00
id be reused to drop a brand-new committed block into that slot,
rewriting approved work without the refinement approval that gates it.

Deliberately not tightened: a past-starting block in a *history* state
still needs no baseline. The first draft of a day can happen in the
afternoon, and the capture is where the agent learns what the morning
held, so requiring baseline evidence would forbid recording real work.
Whether a model should be able to invent that history is a live
question the eval already measures; it is a product decision about what
a plan may record, not one to settle by tightening a guard mid-fix.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 860a7a21a1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lib/features/daily_os_next/agents/service/day_agent_plan_editor.dart Outdated
Comment thread lib/features/daily_os_next/README.md
The past-start guard on the refine path was too blunt. A full `to`
snapshot carries the block's unchanged original start, so extending
the end of a block that began at 09:00 was refused at noon for saying
09:00 — and a pending change accepted after its block started hit the
same wall on the apply path.

Both sites now compare against the live block and refuse only an
actual relocation. `add_block` has no live block, so any past start
there is still a genuine placement into the past.

Also documents the state match a carried-forward block requires, which
the README described as id and start alone.
@matthiasn
matthiasn merged commit f28055d into main Jul 25, 2026
27 of 28 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c6bc2390d7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +80 to +81
Preserve already-started baseline blocks only when they represent existing
in-progress, completed, or dropped history.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Tell the planner to retain committed baseline blocks

On a same-day drafting wake over a legacy agreed plan, the baseline may contain an already-started committed block. The new parser exception expressly accepts that block when its id, start, and state match, but this prompt still tells the model to preserve past baseline blocks only when they are inProgress, completed, or dropped. Because draft_day_plan replaces the full block list, following this instruction omits and silently deletes previously approved committed work; include unchanged committed baseline blocks in the preservation rule.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant