Skip to content

fix: Unify TopN detection across hooks#4026

Merged
stuhood merged 3 commits intomainfrom
stuhood.window-agg-failure
Feb 2, 2026
Merged

fix: Unify TopN detection across hooks#4026
stuhood merged 3 commits intomainfrom
stuhood.window-agg-failure

Conversation

@stuhood
Copy link
Copy Markdown
Collaborator

@stuhood stuhood commented Jan 29, 2026

What

Window function replacement was:

  • Not a strict as TopN's order-by's constraints were.
  • Not disabling itself when the custom scan was disabled.

Both of these cases lead to errors like:

ERROR:  window_agg placeholder should not be executed - custom scan should have intercepted this.

...either because the custom scan would run, but couldn't use TopN, or because it wouldn't run at all.

Why

Our planner hook needs to duplicate a lot of logic contained in basescan planning, but it needs to do so using only a parse/Query, rather than using the planning infrastructure provided to CustomScans. This is error prone, and we should still try and fix it via #3455.

How

Extracted an orderby.rs module in the customscan package to centralize the logic which is now used by the planner hook, basescan, and aggregatescan.

Tests

Added new tests which failed before, and succeed now.

@stuhood stuhood added cherry-pick/0.23.x Request that this PR to `main` should get an automatic cherry-pick PR to `0.23.x` after it lands. cherry-pick/0.22.x Request that this PR to `main` should get an automatic cherry-pick PR to `0.22.x` after it lands. labels Jan 29, 2026
@stuhood stuhood force-pushed the stuhood.window-agg-failure branch from 21d4d50 to 4ab3d72 Compare January 29, 2026 23:33
@stuhood stuhood removed the cherry-pick/0.22.x Request that this PR to `main` should get an automatic cherry-pick PR to `0.22.x` after it lands. label Jan 29, 2026
@stuhood stuhood force-pushed the stuhood.window-agg-failure branch from 4ab3d72 to cb46043 Compare January 29, 2026 23:43
@stuhood stuhood changed the title fix: Unify TopN detection across basescan and planner hook fix: Unify TopN detection across hooks Jan 29, 2026
@stuhood stuhood marked this pull request as ready for review January 29, 2026 23:43
@stuhood stuhood added the cherry-pick/0.22.x Request that this PR to `main` should get an automatic cherry-pick PR to `0.22.x` after it lands. label Jan 29, 2026
@stuhood stuhood force-pushed the stuhood.window-agg-failure branch from cb46043 to d4f1751 Compare February 2, 2026 22:41
@stuhood stuhood merged commit 6d58de1 into main Feb 2, 2026
18 of 23 checks passed
@stuhood stuhood deleted the stuhood.window-agg-failure branch February 2, 2026 23:26
@paradedb-bot
Copy link
Copy Markdown
Contributor

Created backport PR for 0.21.x:

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin backport-4026-to-0.21.x
git worktree add --checkout .worktree/backport-4026-to-0.21.x backport-4026-to-0.21.x
cd .worktree/backport-4026-to-0.21.x
git reset --hard HEAD^
git cherry-pick -x 6d58de10280bd5e3fbbf1a70242ffec0d76cce42
git push --force-with-lease

@paradedb-bot
Copy link
Copy Markdown
Contributor

Created backport PR for 0.20.x:

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin backport-4026-to-0.20.x
git worktree add --checkout .worktree/backport-4026-to-0.20.x backport-4026-to-0.20.x
cd .worktree/backport-4026-to-0.20.x
git reset --hard HEAD^
git cherry-pick -x 6d58de10280bd5e3fbbf1a70242ffec0d76cce42
git push --force-with-lease

stuhood added a commit that referenced this pull request Feb 3, 2026
Window function replacement was:
* Not a strict as TopN's order-by's constraints were.
* Not disabling itself when the custom scan was disabled.

Both of these cases lead to errors like:
```
ERROR:  window_agg placeholder should not be executed - custom scan should have intercepted this.
```
...either because the custom scan would run, but couldn't use TopN, or
because it wouldn't run at all.

Our planner hook needs to duplicate a lot of logic contained in
`basescan` planning, but it needs to do so using only a `parse`/`Query`,
rather than using the planning infrastructure provided to `CustomScan`s.
This is error prone, and we should still try and fix it via #3455.

Extracted an `orderby.rs` module in the `customscan` package to
centralize the logic which is now used by the planner hook, `basescan`,
and `aggregatescan`.

Added new tests which failed before, and succeed now.

(cherry picked from commit 6d58de1)
philippemnoel pushed a commit that referenced this pull request Feb 3, 2026
# Description
Backport of #4026 to `0.21.x`.

Co-authored-by: Stu Hood <[email protected]>
stuhood added a commit that referenced this pull request Feb 3, 2026
Window function replacement was:
* Not a strict as TopN's order-by's constraints were.
* Not disabling itself when the custom scan was disabled.

Both of these cases lead to errors like:
```
ERROR:  window_agg placeholder should not be executed - custom scan should have intercepted this.
```
...either because the custom scan would run, but couldn't use TopN, or
because it wouldn't run at all.

Our planner hook needs to duplicate a lot of logic contained in
`basescan` planning, but it needs to do so using only a `parse`/`Query`,
rather than using the planning infrastructure provided to `CustomScan`s.
This is error prone, and we should still try and fix it via #3455.

Extracted an `orderby.rs` module in the `customscan` package to
centralize the logic which is now used by the planner hook, `basescan`,
and `aggregatescan`.

Added new tests which failed before, and succeed now.

(cherry picked from commit 6d58de1)
stuhood added a commit that referenced this pull request Feb 3, 2026
# Description
Backport of #4026 to `0.20.x`.

Co-authored-by: Stu Hood <[email protected]>
mdashti pushed a commit that referenced this pull request Feb 13, 2026
Backport of #4026 to `0.21.x`.

Co-authored-by: Stu Hood <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cherry-pick/0.22.x Request that this PR to `main` should get an automatic cherry-pick PR to `0.22.x` after it lands. cherry-pick/0.23.x Request that this PR to `main` should get an automatic cherry-pick PR to `0.23.x` after it lands.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants