Skip to content

Only pushdown varchar if the arrow type is not a string view - #20165

Merged
pdet merged 1 commit into
duckdb:v1.4-andiumfrom
evertlammerts:arrow_strview_pushdown
Dec 12, 2025
Merged

pdet merged 1 commit into
duckdb:v1.4-andiumfrom
evertlammerts:arrow_strview_pushdown

Conversation

@evertlammerts

Copy link
Copy Markdown
Member

Fixes duckdb/duckdb-python#187

The arrow table scan function relies on pyarrow for pushed down filters. Pyarrow does not support comparisons for view types yet, so we should not push filters down for those types.

@pdet

pdet commented Dec 12, 2025

Copy link
Copy Markdown
Member

Hmm, it does support for regular strings right? I think the current code will disable filter pushdown for varchar in general.

@pdet
pdet self-requested a review December 12, 2025 17:22
@evertlammerts

Copy link
Copy Markdown
Member Author

Hmm, it does support for regular strings right? I think the current code will disable filter pushdown for varchar in general.

It returns:

return type.GetTypeInfo<ArrowStringInfo>().GetSizeType() != ArrowVariableSizeType::VIEW;

so non-stringview comparisons are still pushed down.

@pdet

pdet commented Dec 12, 2025

Copy link
Copy Markdown
Member

Ah, you are absolutely right, LGTM!

@pdet
pdet merged commit 6e4e339 into duckdb:v1.4-andium Dec 12, 2025
101 checks passed
evertlammerts added a commit to duckdb/duckdb-python that referenced this pull request Dec 13, 2025
Ref: #187
Land only after duckdb/duckdb#20165 is merged

Adds tests for arrow string and binary view filters
github-actions Bot pushed a commit to duckdb/duckdb-r that referenced this pull request Dec 31, 2025
Only pushdown varchar if the arrow type is not a string view (duckdb/duckdb#20165)
Fix use-after-free in mode aggregate Combine function (duckdb/duckdb#20146)
Fix minio nightly tests (duckdb/duckdb#20132)
github-actions Bot added a commit to duckdb/duckdb-r that referenced this pull request Dec 31, 2025
Only pushdown varchar if the arrow type is not a string view (duckdb/duckdb#20165)
Fix use-after-free in mode aggregate Combine function (duckdb/duckdb#20146)
Fix minio nightly tests (duckdb/duckdb#20132)

Co-authored-by: krlmlr <[email protected]>
@evertlammerts
evertlammerts deleted the arrow_strview_pushdown branch February 6, 2026 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants