Skip to content

Parquet pruning drops rows for CAST(int AS BOOLEAN) and CAST(int AS BYTEA) predicates #26981

Closed

Description

@jayzhan211

Describe the bug

Parquet statistics pruning drops row groups that contain matching rows when the predicate casts an integer column to BOOLEAN or BYTEA.

To Reproduce

CREATE TABLE s(i INT) AS VALUES (-1), (0), (1), (255), (256);
COPY s TO '/tmp/i.parquet' STORED AS PARQUET;
CREATE EXTERNAL TABLE p STORED AS PARQUET LOCATION '/tmp/i.parquet';
Query Actual (Parquet) Expected (same as the in-memory table s)
SELECT count(*) FROM p WHERE CAST(i AS BOOLEAN) <= false 0 1
SELECT count(*) FROM p WHERE CAST(i AS BYTEA) < X'05' 0 3

Cause

Pruning casts the row group's min/max and compares them with the literal, which is only correct for casts that keep the order of the values. verify_support_type_for_prune in datafusion/pruning/src/pruning_predicate.rs allows any cast unless exactly one side is a string. Int → boolean (-1 and 1 are true, 0 is false) and int → binary (little-endian bytes) don't keep order, so the bounds -1 and 256 become true/true and X'FFFFFFFF'/X'00010000', and the row group is pruned. #26451 is the same cause for timestamp → time.

Expected behavior

Parquet returns the same rows as the in-memory table. I have a fix ready that makes pruning use is_order_preserving_cast, the rule sort elimination already uses.

Activity

  1. added a commit that references this issue on Nov 7, 2026
    4bf32cb
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions