Repository navigation
Release DataFusion 53.1.0 (minor) (Apr 2026) #21079
Description
Is your feature request related to a problem or challenge?
- Part of [EPIC] DataFusion Releases #19783
- Follow on to Release DataFusion
53.0.0(Feb 2026 / Mar 2026) #19692
I suspect we'll end up needing to backport a few bugs on the branch-53 release, so I am proactively opening a ticket to propose / track such a release
Steps:
- Update version and changelog:
- Voting Thread: https://lists.apache.org/thread/krg9ghwhh58o83014ncjs4zw73mvdq0w
- Publish to crates.io
Potential Content:
NOTE: we should also make sure that we backport / release anything that also went into the branch-52 line:
Requested Bug Fixes. Format: Issue / Requester / Backport PR
Requested improvements
- Super fast extended tests and improved planning speed linux #21084 / @alamb @blaginin: [branch-53] Planning speed improve (port of #21084) #21137
- chore: Optimize schema rewriter usages #21158 @comphead / [branch-53] chore: Optimize schema rewriter usages (#21158) #21183
- fix: do not recompute hash join exec properties if not required #20900 / @alamb @askalt / [branch-53] fix: do not recompute hash join exec properties if not required (#20900) #20903
- fix: use spill writer's schema instead of the first batch schema for spill files #21293 / @alamb @gruuya / [branch-53] fix: use spill writer's schema instead of the first batch schema for … #21451
- fix: use datafusion_expr instead of datafusion crate in spark #21043 (comment) / @rluvaton / [branch-53] fix: use datafusion_expr instead of datafusion crate in spark bitmap/… #21452
- regression: ProjectionPushdown failed with project index 2 out of bounds, max field 2 #21459 / @haohuaijin / [branch-53] fix: FilterExec should drop projection when apply projection pushdown #21492
- [branch-52] Restore Sort unparser guard for correct ORDER BY placement (#20658) #21358 / @alamb / [branch-53] Restore Sort unparser guard for correct ORDER BY placement (#20658) #21523
- [BRANCH-52] fix: foreign inner ffi types #21439 / @alamb / [branch-53] fix: foreign inner ffi types (#21439) #21524
Reacted by Oleks V
Reacted by Shekhar Rajak
Metadata
Metadata
Assignees
Labels
No labels
Activity
hareshkh commentedon Mar 24, 2026
@alamb : Can we add the following issues to this release please?
[branch-53] Planning speed improve (port of #21084) (#21137)
[branch-53] Fix push_down_filter for children with non-empty fetch fi…
[branch-53] Substrait join consumer should not merge nullability of j…
alamb commentedon Mar 31, 2026
I am happy to run this release unless someone else wants to
I'll plan to get it ready for next week sometime
comphead commentedon Mar 31, 2026
I can continue with 53ish to get a full picture on release and hotfixes
alamb commentedon Apr 3, 2026
Sold! I'll assign it you you @comphead
Thank you
I will then help with some of the backports with myfreed up time
comphead commentedon Apr 7, 2026
I'm planning to make all backports today
alamb commentedon Apr 7, 2026
Thanks @comphead -- We should also make sure that anything backported into 52.5.0 is ported to branch-53. I will do that review tomorrow and make any additional backports needed
comphead commentedon Apr 8, 2026
fix: do not recompute hash join exec properties if not required #20900 was applied before, prob its missing from change log
fix: use spill writer's schema instead of the first batch schema for spill files #21293 backport [branch-53] fix: use spill writer's schema instead of the first batch schema for … #21451
fix: use datafusion_expr instead of datafusion crate in spark #21043 backport [branch-53] fix: use datafusion_expr instead of datafusion crate in spark bitmap/… #21452
Remove the statistics() api in execution plan #20319 exists, missing from change log
feat: Add null-aware anti join support #19635 is huge, needs to be merged properly
haohuaijin commentedon Apr 8, 2026
can we add this to the list？
alamb commentedon Apr 8, 2026
Added
17 remaining items
alamb commentedon Apr 9, 2026
After we merge that one I think we need a PR to update the version / changelog and we'll be good to go for 53.1.0 from my perspective
[branch-53] fix: foreign inner ffi types (#21439) (#21524)
[branch-53] Restore Sort unparser guard for correct ORDER BY placement (
comphead commentedon Apr 9, 2026
Merged, looks like we are stable now, I'll start with version/changelog releases PR tomorrow
comphead commentedon Apr 13, 2026
Changelog PR #21559
[branch-53] chore: update deps for cargo audit (#21415) (#21587)
comphead commentedon Apr 14, 2026
Voting thread https://lists.apache.org/thread/krg9ghwhh58o83014ncjs4zw73mvdq0w
mbutrovich commentedon Apr 14, 2026
Testing with Comet apache/datafusion-comet#3946
comphead commentedon Apr 15, 2026
#21625 is the hotfix blocker
comphead commentedon Apr 16, 2026
Release completed, crates.io updated.
Changelog backport to main #21686
mbutrovich commentedon Apr 17, 2026
Shouldn't we bump the version in
mainto 53.1.0?comphead commentedon Apr 17, 2026
Correct, backporting that, thanks for catching. Created #21708
chore: bump datafusion to 53.1.0 (#2350)
chore: bump datafusion to 53.1.0 (apache#2350)