Repository navigation
perf(db): bound since-limited events reads with the month-floor literal - #9197
Conversation
A newest-first read whose own `since` is at or above the recent-pass floor was read whole, bounded only by the `since` bind. A cached generic plan prunes a bind only at execution, after it has locked every `events` partition and its indexes, so live REQs with a recent `since` (`#p` mention reads from reconnecting agents) contended on `LWLock:LockManager`. Such a read now runs as one recent pass: the floor literal (and the ceiling for regular kinds) bounds it at plan time, and no older pass follows since no row below the floor can match. Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: Luke Tornquist <[email protected]>
🔐 Codex Security Review
Review SummaryOverall Risk: NONE
FindingsNo concrete security, correctness, or reliability findings were identified. Notes
Generated by Codex Security Review | |
The lock checks built the recent pass directly, so routing recent-since production reads back to the unsplit read would have kept the test green. Assert the production read prepared exactly the recent pass's statement. Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: Luke Tornquist <[email protected]>
An inline pattern made the lookup query match itself in pg_prepared_statements. Co-authored-by: Elrond <28d6302a099e5225b02c4155ac4236e4912603df2ab08dbfc2f4fef08ce598c8@buzz.block.builderlab.xyz> Co-Authored-By: Claude Opus 5.5 <[email protected]> Signed-off-by: Luke Tornquist <[email protected]>
wpfleger96
left a comment
There was a problem hiding this comment.
looks good. the recent pass with since at or above the floor returns the same rows (the floor is implied by since, and the ceiling only applies to regular kinds, which can't sit past it), and older: false keeps it to one statement. the transaction cost doesn't change either, since the unsplit read already opens one for per-read deadlines and the session-deadline callers are inside a transaction anyway. one nonblocking nit inline.
* origin/main: fix(relay): route live REQ events by each filter's own channel scope (#8519) fix(relay): deliver multi-node fan-out concurrently and never drop silently (#9195) perf(db): bound since-limited events reads with the month-floor literal (#9197) fix(mobile): match Agents sign-in to Home’s reconnect capsule (#9204) fix(cli): clearer desktop pairing errors, paired default relay, and auth desktop list (#9196) Signed-off-by: Brad Seiler <[email protected]>
Summary
A newest-first
eventsread whose ownsinceis at or above the recent-pass floor now runs as a single recent pass. Postgres prunes it to the recent monthlies at plan time instead of locking every partition. Results are unchanged.Why.
split_forskipped any read whosesincewas at or above the floor (since_prunes), on the theory that itssincealready prunes. It does, but only as a bind parameter. Once a prepared statement switches to a cached generic plan, Postgres prunes bind-parameter bounds at execution, afterAcquireExecutorLockshas already locked everyeventspartition and its indexes. Live REQs with a recentsinceare this shape, for example#pmention reads from reconnecting agents. Under load they spill past the 16 fast-path lock slots and contend onLWLock:LockManager. The ceiling PR (#9157) named this gap as identified debt.What changes. Such a read now gets the same
Splitas other recent reads, so the recent pass carries the floor literal (plus the ceiling literal for regular-kind reads) one.created_atand onm.event_created_atfor#p.Splitgainsolder: bool. It is false when the read's ownsinceis at or above the floor, because no row below the floor can match, so no older pass follows a short recent pass. A read whoseuntilfalls below the floor stays unsplit, as before.No protocol, schema, index or client change.
Related issue
Follow-up to #9157 (its identified debt: bind-parameter
sincereads).Testing
recent_pass::tests::split_eligibility: the table now records not split / ceiled / open and whether an older pass follows. New rows:sinceat the floor, a recentsincewith regular, any and addressable-only kinds, and a recentsincewithuntilbelow the floor.recent_pass::testsin-memory exactness: adds asince-at-floor split that runs exactly one pass and returns the rows at or above the floor.postgres_tests::since_bounded_mention_read_locks_only_recent_partitions(new): runs partition maintenance, inserts#prows at the floor, now, one second below the ceiling and one day below the floor, then reads withsince = floor. The production path returns the three recent rows through the mention join. Underplan_cache_mode = force_generic_planthe recent pass locks only the partitions holding those rows. The unsplit read locks every partition.The partition-lock helper from perf(db): stop the events recent pass at a ceiling for regular-kind reads #9157's test is now shared by both lock tests.
Diagnostic on Postgres 17.11 with the 19-partition layout partition maintenance keeps, for a
#pread withsinceat the floor (not committed):eventspartitions lockedsince, generic plan after five runs)EXPLAIN (GENERIC_PLAN)before appends all 19 partitions. After, it appends only the previous and current monthlies.Mutation-checked on Blox. Each of these fails the new Postgres test or the unit tests: removing the
since_prunessplit, ignoringolder, and dropping thee.literal.What got simpler
Nothing got simpler. The change adds one flag to the existing
Splittype and no new concept. The duplicated partition-lock reader in the #9157 test became one shared helper.🤖 Generated with Claude Code