Skip to content

perf: make day-scoped capture and status-event reads indexed#3569

Merged
matthiasn merged 3 commits into
mainfrom
feat/day-scoped-capture-reads
Jul 25, 2026
Merged

perf: make day-scoped capture and status-event reads indexed#3569
matthiasn merged 3 commits into
mainfrom
feat/day-scoped-capture-reads

Conversation

@matthiasn

Copy link
Copy Markdown
Owner

Third of the "day planner must not degrade as history accumulates" series,
after #3564 (outbox → table) and #3566 (priority claiming).

The problem

Opening a day loaded every capture and every day-status event the
owning agent had ever recorded, then filtered them in Dart:

  • day_agent_provider.dartgetEntitiesByAgentId(ownerId, type: capture)
    for both owners, no limit, day-filtered afterwards. Its own comment conceded
    "an owner can hold many days' captures (the coordinator always does)".
  • day_agent_persona_provider.dart — every dayStatusEvent for the owner,
    filtered to the day in Dart. Status events drive the activity indicator and
    are the fastest-growing entity type in the feature.

Both are Riverpod providers on the main day surface, so the cost of opening one
day grew with the user's entire history.

Severity was not uniform, which is worth stating: a day_agent:<dayId> is
self-limiting by construction — it holds one day's rows and then goes cold
forever. The coordinator is the accumulator, and it owns every pre-cutover day
(day_agent_service.dart leaves those under the monolith permanently).

Cause

The projected subtype column. capture stored the capture's own id —
redundant with the primary key — and dayStatusEvent stored its status name,
so neither could serve a day-scoped lookup. Meanwhile dayPlan, daySummary
and dayDirective already stored the day and were served by
idx_agent_entities_agent_type_sub.

Neither old value was used as a query key anywhere, so repurposing them is
safe. subtype is a locally-derived projection, not part of the synced
payload, so a device on an older build keeps deriving its own — no sync skew.

What changed

  • entitySubtype writes the day workspace for both types; the two providers
    read through getEntitiesByAgentIdAndSubtype.
  • localDay/dayAgentIdForDate move to lib/classes/day_plan.dart beside
    dayPlanId, so the agent persistence layer can derive a capture's day
    without depending on a feature package.
  • A capture with no explicit dayId (synced from a peer predating it) derives
    one from capturedAt in local time — the same rule captureDayId
    applies on read — so legacy rows land on the right day instead of needing a
    Dart-side fallback scan.
  • Schema v17 backfills existing rows through that same Dart projection rather
    than a SQL json_extract update. That is deliberate: the local-time
    derivation cannot be expressed faithfully in SQL, and a backfilled value that
    disagreed with the writer would silently drop rows out of their day.
    Undecodable rows are skipped rather than aborting the upgrade.

Testing

1902 daily_os_next + 5066 agents tests pass; analyzer and formatter clean.

New coverage: day-scoped reads return only the requested day; a legacy capture
with no dayId is found by its derived day; an explicit dayId wins over the
derivation (a capture recorded after midnight can belong to the previous
planning day); and the conversion tests now pin the day subtype for both types.

No CHANGELOG entry — no runtime behavior a user would notice beyond the day
view getting cheaper.

Still open in this area

_plannerOwnsDay (day_agent_service.dart) still scans the planner's whole
capture history via getCaptureEventMetaByAgentId on every day-agent
resolution, which runs ahead of both providers. It is now a one-line query
change on top of this index and I left it out to keep the schema migration
reviewable on its own — happy to fold it in here if you would rather.

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@matthiasn, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 18 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 3df82bab-d710-4a53-8e35-56266d122d56

📥 Commits

Reviewing files that changed from the base of the PR and between 22047b1 and 791cfdd.

📒 Files selected for processing (22)
  • lib/classes/day_plan.dart
  • lib/features/agents/database/agent_database.dart
  • lib/features/agents/database/agent_db_conversions.dart
  • lib/features/agents/database/agent_repo_core.dart
  • lib/features/agents/database/agent_repository.dart
  • lib/features/daily_os_next/README.md
  • lib/features/daily_os_next/agents/domain/day_agent_identity.dart
  • lib/features/daily_os_next/agents/domain/day_agent_slots.dart
  • lib/features/daily_os_next/agents/domain/week_context.dart
  • lib/features/daily_os_next/agents/domain/week_rollup.dart
  • lib/features/daily_os_next/agents/service/day_agent_plan_diff.dart
  • lib/features/daily_os_next/agents/service/day_agent_plan_parser.dart
  • lib/features/daily_os_next/logic/real_day_agent.dart
  • lib/features/daily_os_next/state/day_agent_persona_provider.dart
  • lib/features/daily_os_next/state/day_agent_provider.dart
  • test/features/agents/database/agent_db_conversions_test.dart
  • test/features/agents/database/agent_repo_core_test.dart
  • test/features/agents/database/agent_repository_test.dart
  • test/features/daily_os_next/logic/real_day_agent_test.dart
  • test/features/daily_os_next/state/day_agent_persona_provider_test.dart
  • test/features/daily_os_next/state/day_agent_provider_test.dart
  • test/features/daily_os_next/ui/widgets/day_timeline_test.dart
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/day-scoped-capture-reads

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov

codecov Bot commented Jul 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.22%. Comparing base (22047b1) to head (791cfdd).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3569   +/-   ##
=======================================
  Coverage   99.22%   99.22%           
=======================================
  Files        1780     1780           
  Lines      130297   130324   +27     
=======================================
+ Hits       129283   129311   +28     
+ Misses       1014     1013    -1     
Flag Coverage Δ
glados 14.14% <21.62%> (-0.01%) ⬇️
standard 98.97% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Opening a day loaded every capture and every day-status event the owning
agent had ever recorded and filtered them in Dart, so the cost of the main
day surface grew with the user's entire history. It bit hardest on
planner-owned days: a day agent is self-limiting, but the coordinator
accumulates forever and owns every pre-cutover day.

The cause was the projected subtype column — capture stored its own id
(redundant with the primary key) and dayStatusEvent its status name, so
neither could serve a day-scoped query, while dayPlan/daySummary/
dayDirective already stored the day and were indexed. Both now store the
day workspace and read through idx_agent_entities_agent_type_sub.

- localDay/dayAgentIdForDate move to lib/classes/day_plan.dart beside
  dayPlanId so the agent persistence layer can derive a capture's day
  without depending on a feature package.
- A capture with no explicit dayId derives one from capturedAt in local
  time, the same rule captureDayId applies on read, so legacy rows synced
  from older peers land on the right day.
- Schema v17 backfills through the same Dart projection rather than a SQL
  json_extract update: a backfilled value disagreeing with the writer
  would silently drop rows out of their day.
Codecov flagged 9 uncovered lines: the schema v17 backfill and the
repository delegator.

The backfill is exposed via @VisibleForTesting so the migration logic can
be driven directly with rows written at their pre-v17 subtype. Covers the
capture-id and status-name rewrites, the local-time derivation for a
legacy capture with no dayId, that an undecodable row is skipped rather
than costing every other row its index entry, and that types which
already stored the day are left alone.
@matthiasn
matthiasn force-pushed the feat/day-scoped-capture-reads branch from 529f99f to ad6fbb9 Compare July 24, 2026 23:56
The test pins its draft to the real calendar day, then pumps a minute so
the widget's per-minute timer fires. That callback re-reads the *real*
clock, not the test binding's, so a run that straddles midnight moves
"now" out of the draft's day and correctly retires the badge — CI hit
this at 23:59:29 UTC.

Assert the badge only while the wall clock is still on the day the draft
was built for. The survives-a-tick property the test is named for is
carried by the anchor-block and still-mounted checks regardless.
@matthiasn
matthiasn merged commit 4438260 into main Jul 25, 2026
26 of 27 checks passed
@matthiasn
matthiasn deleted the feat/day-scoped-capture-reads branch July 25, 2026 00:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant