Repository navigation
Revert #25790 (Decimal filter selectivity) due to TPC-H Q18 regression - #27145
Merged
Merged
Conversation
…sis (apache#25790)" This reverts commit 6145a09.
Contributor
|
Ope, appears great minds think alike. I've opened a PR for this as well 😆 I'll go ahead and close mine. |
alamb
approved these changes
Nov 5, 2026
Contributor
There was a problem hiding this comment.
Thank you -- I verified this is a clean revert.
Thank you @asolimando and @devanbenz
alamb
enabled auto-merge
November 5, 2026 21:00
This was referenced Nov 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
On TPC-H Q18, DataFusion 56 takes about 233 s and 22 GB of peak memory on an M3 Max, compared with about 7 s and 4 GB on 55.1.0. On x86_64 the process runs out of memory (#27108). @devanbenz traced the regression to #25790: the new Decimal filter selectivity estimate changes the join order of the plan.
As agreed in #24461 (comment), this PR reverts #25790 before the 56 release.
#27146 tracks the work to re-land the change.
What changes are included in this PR?
This reverts commit 6145a09. The only conflict was
filter_statistics_decimal.slt, which #25790 added and #26761 later edited. This PR deletes it.What is the testing strategy for this PR?
The existing tests, which pass locally.
Are there any user-facing changes?
Filters on Decimal columns use the selectivity estimate from 55.x again.
Disclaimer: I used AI to assist in the code generation, I have manually reviewed the output and it matches my intention and understanding.