Skip to content

TopK aggregation duplicates NaN groups and can panic during eviction #26444

Closed

Description

@bvolpato

Describe the bug

With TopK aggregation enabled, repeated identical NaN grouping keys can produce separate groups. Evicting a NaN group can also panic with internal error: entered unreachable code in aggregates/topk/hash_table.rs.

To Reproduce

Reproduced on main at 5f2158269e24ac45e2740dcfe6e79a6f69f28330:

SET datafusion.execution.target_partitions = 1;
SET datafusion.optimizer.enable_topk_aggregation = true;

SELECT k, MAX(v) FROM (VALUES
  (CAST('NaN' AS DOUBLE), 1),
  (CAST('NaN' AS DOUBLE), 2),
  (CAST(0 AS DOUBLE), 3)
) AS t(k, v)
GROUP BY k ORDER BY MAX(v) DESC NULLS LAST LIMIT 3;

This returns an extra group:

0   3
NaN 2
NaN 1

Changing LIMIT 3 to LIMIT 1 triggers the eviction panic. Disabling datafusion.optimizer.enable_topk_aggregation gives the expected results.

Expected behavior

The query with LIMIT 3 returns 0, 3 and NaN, 2. With LIMIT 1, it returns only 0, 3, without a panic.

Additional context

#16256 fixed analogous NaN key equality in ordinary aggregation. This report concerns TopK's key lookup and eviction paths. It uses repeated NaNs with identical bits and does not require the broader all-NaN canonicalization proposed in #26091.

Willingness to contribute

I plan to contribute a fix for this bug independently. A draft PR with the fix and regression coverage will follow.

Activity

  1. added a commit that references this issue on Oct 28, 2026
    8f77b41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions