You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 2904c4d
Browse filesBrowse the repository at this point in the historyBrowse files
Merge apache/main into chore/track-datafusion-git-main; bump datafusion rev
Merged latest apache/main into the branch and bumped the datafusion* git
dependencies from b79d3965 to efa84e83 (current apache/datafusion main HEAD).
That bump forced a re-port of the physical-plan and proto-codec APIs:
- statistics_with_args + StatisticsArgs::compute_child_statistics were
refactored into StatisticsContext::compute + ExecutionPlan
::statistics_from_inputs(input_stats, args) +
ExecutionPlan::child_stats_requests(partition). All Ballista operator
overrides and call sites were migrated. Custom operators
(ChaosExec, ShuffleWriterExec, SortShuffleWriterExec, BufferExec,
RuntimeStatsExec, executor CollectExec) now declare a single
ChildStats::At(partition) request and return their sole child's
precomputed stats, restoring the correctness fix the original PR made
when statistics_with_args replaced partition_statistics.
- PhysicalExtensionCodec::try_decode/try_encode gained a
PhysicalProtoConverterExtension argument; all test call sites now pass
&DefaultPhysicalProtoConverter {}.
- Partitioning::Range is a new enum variant; the shuffle-writer
partition-mapping walker treats it as a K-space (like Hash), and the
task-builder shuffle-reader rewrite falls back to Unknown for it.
- ExecutionPlan::required_input_distribution is deprecated in favour of
input_distribution_requirements(); task_builder was migrated.
- apache/datafusion#23642 is fixed upstream, so the pin test that
asserted the bug's presence was inverted into a regression test that
the partition count survives round-trip. make_empty_exec_serde_safe
itself is kept as belt-and-suspenders for legacy stage plans decoded
from persisted state.
- Plan-structure test expectations were updated for the new physical
optimizer ordering (ProjectionExec now sits above SortExec in the
aggregate/window fan-out) and the new RoundRobinBatch(2) repartition
the optimizer inserts above the SortMergeJoin.
- context_checks.rs's EXPLAIN ANALYZE snapshot merged the new
ShuffleReaderExec fetch metrics from main with the peak_mem_used drop
from the datafusion bump.
0 commit comments