Skip to content

LimitPushPastWindows only pushes a limit past the first window in a plan #26659

Closed

Description

@jayzhan211

Is your feature request related to a problem or challenge?

LimitPushPastWindows turns the SortExec below a window into a TopK when a LIMIT sits above the window. It only does this for the first such window it meets. In a UNION ALL of two windowed queries with limits, the second branch keeps a full sort:

CREATE TABLE t(a INT, x INT) AS VALUES (1, 1), (2, 2), (3, 3), (4, 4), (5, 5);

EXPLAIN SELECT * FROM (SELECT a, sum(x) OVER (ORDER BY a ROWS UNBOUNDED PRECEDING) s FROM t ORDER BY a LIMIT 2)
UNION ALL
SELECT * FROM (SELECT a, sum(x) OVER (ORDER BY a ROWS UNBOUNDED PRECEDING) s FROM t ORDER BY a LIMIT 3);
UnionExec
  ProjectionExec: ...
    GlobalLimitExec: skip=0, fetch=2
      BoundedWindowAggExec: ...
        SortExec: TopK(fetch=2), expr=[a@0 ASC NULLS LAST], preserve_partitioning=[false]
          DataSourceExec: partitions=1, partition_sizes=[1]
  ProjectionExec: ...
    GlobalLimitExec: skip=0, fetch=3
      BoundedWindowAggExec: ...
        SortExec: expr=[a@0 ASC NULLS LAST], preserve_partitioning=[false]      <-- no TopK(fetch=3)
          DataSourceExec: partitions=1, partition_sizes=[1]

The same happens for any later window in the plan, not only under a union.

Describe the solution you'd like

Both branches get a TopK. Two things in datafusion/physical-optimizer/src/limit_pushdown_past_window.rs cause this:

  1. After setting the fetch, apply_limit returns Transformed::complete(...) (:201). That sets TreeNodeRecursion::Stop, which stops the whole transform_down, not just the subtree below the sort. TreeNodeRecursion::Jump would skip only the sort's children.
  2. phase (:83) becomes Phase::Apply at the first window and is never set back. The reset closure (:86-94) clears the limit but not the phase, so later limits are never picked up.

Describe alternatives you've considered

None.

Additional context

Activity

  1. cool9850311 commented on Oct 23, 2026

    @cool9850311
    Contributor

    take

  2. added 11 commits that reference this issue on Oct 27, 2026
    6ef0918
    6e4279c
    e330b6b
    177fd71
    1b857a3
    85cdf20
    d5d9abe
    d1995b3
    97fbedf
    f501775
    db77d3f
  3. added a commit that references this issue on Nov 2, 2026
    f058e2d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions