Skip to content

Wrong results: pushing a volatile filter below GROUP BY changes the count #27074

Closed

Description

@Phoenix500526

A filter applied after GROUP BY can change the number of rows counted within a group. Reproduced with DataFusion CLI 57.0.0-dev:

SELECT c
FROM (
    SELECT a, COUNT(*) AS c
    FROM (VALUES (1), (1)) AS t(a)
    GROUP BY a
) AS counts
WHERE random() < 0.5;

Expected: Both input rows belong to the same group. The query should return either 2 or no rows, depending on whether the group passes the filter.

Actual: The query can return 1. EXPLAIN VERBOSE shows that push_down_filter moves random() < 0.5 below the aggregate, so it runs once per input row. If exactly one row passes, the count is 1.

@DKER2 originally reported this behavior for an aggregate without GROUP BY. The example above reproduces it with an ordinary grouping key, independently of the empty-input behavior tracked in #26545. Original comment

Activity

  1. Phoenix500526 commented on Nov 4, 2026

    @Phoenix500526
    ContributorAuthor

    take

  2. added 2 commits that reference this issue on Nov 5, 2026
    39b8de1
    a203cf2
  3. added a commit that references this issue on Nov 7, 2026
    173f126
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions