Skip to content

Latest commit

 

History

History
1494 lines (1473 loc) · 164 KB

56.0.0.md

File metadata and controls

1494 lines (1473 loc) · 164 KB

Apache DataFusion 56.0.0 Changelog

This release consists of 1200 commits from 247 contributors. See credits at the end of this changelog for more information.

See the upgrade guide for information on how to upgrade from previous versions.

Breaking changes:

  • refactor(physical-plan): Simplify ExecutionPlan API with replace_children #23903 (JSOD11)
  • fix(ffi): preserve TableProvider DML overrides #23906 (Amogh-2404)
  • fix: don't report equi-key equivalence for PiecewiseMergeJoin range joins #24361 (viirya)
  • perf: Update TableProvider to take scan projection as Option<&[usize]> #24376 (Dandandan)
  • fix(proto): preserve Arrow IPC stream format #24224 (buraksenn)
  • fix: translate SQL wildcards in SIMILAR TO patterns (#22263) #23188 (oc7o)
  • docs: add HashJoinExec metrics to user guide #24055 (fyrsta7)
  • perf: Extend WindowTopN to dense_rank #23869 (SubhamSinghal)
  • Split and deprecate and_or #24276 (Nagato-Yuzuru)
  • chore: remove NowFunc::new #24555 (2010YOUY01)
  • chore: remove ScalarValue::raw_data #24548 (2010YOUY01)
  • perf: skip eval of standard window functions for unchanged partitions #24532 (neilconway)
  • chore: remove UserDefinedFunctionPlanner #24554 (2010YOUY01)
  • chore: remove ExecutionProps::start_execution #24553 (2010YOUY01)
  • chore: remove ExprSimplifier::simplify_with_cycle_count #24551 (2010YOUY01)
  • chore: remove DisplayableExecutionPlan::to_stringified #24550 (2010YOUY01)
  • feat: add projection support to SortMergeJoinExec #24517 (Dandandan)
  • chore: deprecate with_dynamic_filter_expr on Sort/Aggregate/HashJoin #24610 (adriangb)
  • chore: remove deprecated ScalarUDF / ScalarUDFImpl methods past the API health policy #24561 (MannXo)
  • clean up : remove re-export of overlay (moved since 47.0.0) #24635 (Rich-T-kid)
  • fix(proto): preserve CSV/JSON scan options on roundtrip #24233 (buraksenn)
  • feat: make StatisticsRegistry the default operator-statistics path #23651 (asolimando)
  • feat: proto serialization for PiecewiseMergeJoinExec #24378 (SubhamSinghal)
  • Make udaf_default_* display/schema name helpers non-generic (214 KB smaller binary) #24728 (alamb)
  • fix(proto): preserve AnalyzeExec metric types across serialization #24669 (haohuaijin)
  • feat: preserving grouped state reads #24714 (rkrishn7)
  • feat(dataframe): improve DataFrame::from_columns input types (#24630) #24633 (cj-zhukov)
  • Fix CASE evaluation for custom column expressions #24484 (Hasnaathussain)
  • fix: preserve Parquet sort pushdown across proto roundtrips #24930 (jayshrivastava)
  • feat: add ASOF join logical semantics #23829 (Xuanwo)
  • fix: prevent CSE from extracting window functions #24750 (cdelmonte-zg)
  • Share per-chunk JoinLeftData across right partitions in NLJ memory-limited fallback #22038 (viirya)
  • Range partition on primitives. #24598 (stuhood)
  • fix: preserve Arrow field metadata on scalar subquery expressions #24939 (M-Tesla)
  • fix(proto): destructure JSON source and sink serde hooks #24945 (buraksenn)
  • refactor(proto): destructure FileScanConfig and MemorySourceConfig serde hooks #24813 (buraksenn)
  • feat: support ASOF JOIN SQL #23830 (Xuanwo)
  • fix: validate parquet statistics config #24642 (zhangxinyao88)
  • fix: from_unixtime should respect datafusion.execution.time_zone #25161 (adriangb)
  • refactor: widen GroupColumn::vectorized_equal_to to &mut self #25238 (Rich-T-kid)
  • feat: serialize ASOF join plans #23832 (Xuanwo)
  • minor: Remove empty public module `datafusion::physical_plan::joins::chain #25286 (2010YOUY01)
  • fix: map pushed-down filter columns by position instead of by name #25259 (haohuaijin)
  • chore(deps): bump dirs from 6.0.0 to 7.0.0 #25324 (dependabot[bot])
  • fix(proto): preserve CSV sink writer options #25058 (buraksenn)
  • fix: report regex compile failures as DataFusion errors, consistently across the regexp family #25352 (adriangb)
  • perf: reuse one StatisticsContext across ensure_distribution #25098 (zhuqi-lucas)
  • fix(proto): preserve Parquet source and sink state #25057 (buraksenn)
  • feat: simplify contradicting and redundant predicates on a column #25207 (wudidapaopao)
  • fix: preserve MERGE target qualifier bindings #24429 (wirybeaver)
  • chore: remove deprecated Parquet APIs past the API health policy #25202 (WaterWhisperer)
  • docs: Document the convention for internal public APIs #25495 (2010YOUY01)
  • fix: preserve overflow behavior and exhaustively destructure expression proto hooks #25034 (peterxcli)
  • fix: correct LEAD/LAG IGNORE NULLS evaluation and limit pushdown #25472 (lyne7-sc)
  • Add an extension point to allow SQL based registration of external catalogs #25112 (pepijnve)
  • refactor: use row-group-local selections for Parquet scans #25608 (haohuaijin)
  • feat: allow scaling RangePartitioning #24766 (goutamadwant)
  • Add datafusion.execution.soft_max_bytes_per_output_file config parameter #24924 (Tpt)
  • fix: do not prune ORDER BY keys using nullable UNIQUE constraints #23918 (buraksenn)
  • feat: Push down offset/skip to TableScan #25404 (martin-g)
  • refactor: Make AggregateExec state modeling and public updates safe (part2: topk) #25696 (2010YOUY01)
  • chore: Hide test-only internal public APIs #25695 (2010YOUY01)
  • fix: Fix make_array null input handling #25924 (2010YOUY01)
  • feat: Add option to assign Parquet row groups to file ranges by midpoint #25940 (comphead)
  • fix: use session timezone for timestamp subtraction and comparison #25094 (kumarUjjawal)
  • introduce optional rle reads from parquet #24227 (Rich-T-kid)
  • refactor: deprecate the bundled statistics providers #25969 (asolimando)
  • fix: preserve literal metadata and destructure leaf and unary proto hooks #25382 (peterxcli)
  • feat: support multi-column NOT IN subqueries with null-aware joins #19857 (viirya)
  • fix: preserve dotted column relation qualifiers #25334 (AnuragRaut08)
  • fix: preserve ProjectionExec metadata during serialization #25009 (goutamadwant)
  • feat: Use NativeType in get_example_type, information schema #21737 (theirix)
  • Feat(parquet) : Introduce support for optionally writing Distinct Values to statistics #25576 (Rich-T-kid)
  • test: cover CrossJoinExec row metrics across batched inputs #25530 (rgbuilds)
  • chore: Hide internal public utility APIs #25963 (2010YOUY01)
  • perf: Add an adaptive integer membership prefilter for hash joins #25363 (lyne7-sc)
  • Deprecate CsvSource::newlines_in_values and preserve proto coverage #26179 (kosiew)
  • chore(optimizer): remove deprecated supports_rewrite #26180 (haohuaijin)
  • fix: reject date_bin strides that are not positive #26070 (viirya)
  • perf: share one StatisticsContext across all physical optimizer rules #26094 (asolimando)
  • fix(proto): preserve physical cast expression state #25056 (buraksenn)
  • fix: match functional dependencies by column position, not by name #26150 (jayzhan211)
  • fix(proto): preserve EmptyRelation schema across logical plan round trip #25944 (bigluck)
  • Add Arrow IPC format options #26155 (pepijnve)
  • feat: allow extension ExecutionPlans to decode via a per-type registry instead of PhysicalExtensionCodec #24631 (adriangb)
  • refactor: use group clustering terminology in aggregate execution #24697 (xavlee)
  • feat!: pass input schema to ExprPlanner::plan_scalar #26177 (fengys1996)
  • fix: restore floating-point rounding mode after panic #26412 (emanuele-em)
  • feat: serialize LazyMemoryExec through generator hooks #25765 (QinXi-ai)
  • fix: keep file order for scans that advertise an output ordering #26434 (jayzhan211)
  • Revert #25363 due to overlapping optimizations #26410 (2010YOUY01)
  • fix: AT TIME ZONE on a timezone-aware timestamp returns a naive timestamp #25165 (adriangb)
  • feat: allow extension PhysicalExprs to decode via a per-type registry #24628 (adriangb)
  • fix: include nulls in file ordering bounds #26280 (rank-p)
  • bench: non-ASCII cases for left and right #26413 (mightsleep)
  • feat: support col reference for percentiles #25337 (dd-annarose)
  • fix: reject DISTINCT in corr, covar and regr_* instead of ignoring it #26484 (abp002)
  • test: run microbenchmarks in extended CI #25836 (aetosdios27)
  • fix: declare Hive partitioned scans as Range #25279 (gstamatakis95)
  • feat: estimate Decimal filter selectivity with interval analysis #25790 (shrivasshankar)
  • fix: Correct ANY/ALL results for unaliased subqueries on the same table #25273 (AdamGS)
  • fix(ffi): forward distinct_handling through FFI_AggregateUDF #25876 (bharadwaj-pendyala)
  • feat: make the materialized CTE nodes a first-class LogicalPlan variant #26269 (adriangb)
  • refactor: Centralize float normalization #26595 (mkleen)
  • refactor: Use cast preimages for cast predicate rewrites #22906 (discord9)
  • fix: preserve hash join dynamic filters across recursive CTE iterations #26669 (Kristofferjoha)
  • Support partitioning of information schema content per catalog #25887 (pepijnve)
  • chore: Move AggregateExec optimizer utilities to a new file #26733 (2010YOUY01)
  • refactor: register proto decoders with T::register_proto instead of a free function #26565 (adriangb)
  • refactor: share the REPLACE and validation steps of the wildcard arms in project_with_validation #26312 (ozenyunus35)
  • fix: Unify schema metadata merge logic in unions to "last one wins" #26881 (AdamGS)
  • Rename LogicalPlan::DescribeTable to LogicalPlan::Show move statement implementations out of sql module #26057 (pepijnve)
  • refactor: replace normalize_float_scalar with ScalarValue::normalize #26727 (mkleen)
  • Check the final physical plan in the planner and name the rule that broke it #26461 (jayzhan211)
  • refactor: one vocabulary for moving LIMIT through the physical plan #26660 (jayzhan211)
  • feat: reuse range partitioning for audited date_bin aggregation #26835 (benbellick)
  • feat: deprecate AggregateUDFImpl::is_nullable in favour of return_field #26746 (CuteChuanChuan)
  • perf: reuse projection schema in OptimizeProjections instead of recomputing it #24281 (zhuqi-lucas)

Performance related:

  • perf: Concretely typed TopK ArrowHashTable storage #23609 (MassivePizza)
  • [Parquet] Skip bloom filter predicate evaluation when no bloom filters are loaded #24101 (rgehan)
  • perf(catalog): cut datafusion-catalog compile time ~6x #24325 (Dandandan)
  • perf(core): cut datafusion core compile time ~10x #24329 (Dandandan)
  • perf(session): cut datafusion-session compile time ~1.7x #24326 (Dandandan)
  • perf(build): don't build the doc-generator binaries by default #24338 (Dandandan)
  • perf(build): use line-tables-only debug info in the dev profile #24339 (Dandandan)
  • Improve planning speed: Fast path for union_schema when all children share a schema #24389 (reidkaufmann)
  • perf: Reduce record batch memory accounting overhead #24319 (ryux1)
  • IN LIST: centralize primitive filters and optimize Decimal128 #24283 (geoffreyclaude)
  • perf: avoid processing discarded rows for negative nth_value #24453 (lyne7-sc)
  • perf: reuse the projected equivalence group when only child orderings change #24445 (zhuqi-lucas)
  • perf: make cloning SessionState faster by sharing most state behind an Arc #24560 (tschwarzinger)
  • perf: skip building FilterRemapper when there are no parent filters #24428 (Braedon-Wooding-Displayr)
  • perf(parquet): skip RowFilter on statically fully-matched row groups #23696 (zhuqi-lucas)
  • perf: cache dictionary arc pointer #24418 (Rich-T-kid)
  • perf: use compact pruning for large string IN lists #24526 (sunchao)
  • perf: replace PiecewiseMergeJoin's visited bitmap with a suffix watermark (~1.8x faster Left join) #24579 (jayzhan211)
  • perf: Avoid cloning EquivalenceProperties in ordering satisfaction checks #24800 (jayzhan211)
  • perf: use compact pruning for large string NOT IN lists #24781 (kumarUjjawal)
  • perf: remove per-row String allocation from Spark soundex and quote #23880 (andygrove)
  • perf: extend string IN-list pruning benchmark coverage #24836 (goutamadwant)
  • perf(optimizer): avoid redundant aggregate planning work #24977 (neilconway)
  • perf: extend compact IN-list pruning to ordered types #25012 (kumarUjjawal)
  • perf(sort-merge): cache current-row bytes in RowValues for SortPreservingMerge #23990 (zhuqi-lucas)
  • perf: Compute array_length of first dimension from offsets #25121 (neilconway)
  • perf: reduce invariant-check code generation #25073 (kumarUjjawal)
  • perf: compact inputs for grouped aggregate filter #24869 (lyne7-sc)
  • perf: emit unmatched build rows in batch_size chunks in HashJoinExec #25028 (jayzhan211)
  • perf: defer nulls creation until needed in ArrowBytesViewMap #25014 (rluvaton)
  • Perf: Take the comparison-free fast path in the grouped first/last accumulator when input is pre-ordered #25050 (zhuqi-lucas)
  • perf: compute cardinality from list/map offsets #25079 (neilconway)
  • perf: use columnar state for ordered ARRAY_AGG and STRING_AGG #24392 (lyne7-sc)
  • perf: Optimize prefix-group processing in PartialSortExec #24979 (lyne7-sc)
  • perf: reuse cached dictionary value hashes in vectorized_append #25185 (LiaCastaneda)
  • perf: Unify, optimize map[k] and map_extract(key) #25201 (neilconway)
  • perf: allow concurrent TrackConsumersPool updates #24993 (yashrb24)
  • perf: optimize levenshtein #23543 (andygrove)
  • perf: embed projections in ASOF joins #24873 (Xuanwo)
  • perf(range): avoid eager ok_or in gen_range_timestamp #25308 (niebayes)
  • perf: Improve efficiency of array_has_any on sliced lists #25427 (neilconway)
  • perf: Reduce peak memory usage of NLJ by 2X using segmented batch layout #25371 (2010YOUY01)
  • perf: Improve efficiency of array_resize for sliced arrays #25479 (neilconway)
  • perf: Use agg DistinctHandling in join optimization #25385 (neilconway)
  • perf: Avoid needless conversion of Metadata to HashMap #25510 (neilconway)
  • perf: reuse column normalization context across expressions #25010 (goutamadwant)
  • perf: Share backing buffers in string view functions #25501 (neilconway)
  • perf: preserve left functional dependencies for ASOF joins #24799 (Xuanwo)
  • perf: Size builder capacity using visible array span, not total size #25502 (neilconway)
  • perf: push left filters through ASOF joins #24801 (Xuanwo)
  • perf: add typed bounds for date_bin and from_unixtime #25575 (Toby1009)
  • perf: GroupsAccumulatorAdapter walks only the groups a batch has rows for #25123 (adriangb)
  • perf: reuse GroupsAccumulatorAdapter scratch buffers across batches #25605 (adriangb)
  • perf: Optimize array_compact #25607 (neilconway)
  • perf: Improve output materializing speed for OrderedFinalAggregateStream #25639 (2010YOUY01)
  • perf: mirror ASOF equality-key filters to right #25574 (Xuanwo)
  • perf: interleave retained rows at emit instead of a take per group entry #25468 (SubhamSinghal)
  • perf: skip wasted per-batch work in TopK insert_batch #25657 (dwsmith1983)
  • perf: avoid unnecessary aggregate spill rewrites #25428 (sunchao)
  • perf: use one null-aware mark join for hashable IN subqueries in projections #25338 (adriangb)
  • perf(parquet): skip Bloom reads for fully matched row groups #25854 (xudong963)
  • perf: reuse cached dictionary value slots in vectorized_equal_to #25652 (CuteChuanChuan)
  • perf: PartitionedTopK hold retained rows in one shared RecordBatchStore #25730 (SubhamSinghal)
  • perf: preserve scalar UDF output types in property analysis #25668 (Toby1009)
  • perf(aggregate): specialize fully ordered group keys without hashing #25150 (QinXi-ai)
  • perf: Optimize array_agg(... IGNORE NULLS) #25723 (neilconway)
  • perf: mirror compound ASOF key predicates to right #25728 (Xuanwo)
  • perf: avoid quadratic planning for SELECTs with many aggregates #25739 (1fanwang)
  • perf(proto): avoid re-normalizing logical plans #25297 (AnuragRaut08)
  • perf: stop redoing per-chunk setup on the hash join probe path #25658 (dwsmith1983)
  • perf: evaluate AND chains as one n-ary conjunction (#25035) #25771 (Abhisheklearn12)
  • perf: normalize float hash join build keys once #25972 (dwsmith1983)
  • perf: hash only the selected values of union arrays #25793 (Abhisheklearn12)
  • perf: materialize completed groups once in OrderedSingleAggregateStream #25984 (jayzhan211)
  • perf: skip functional dependency work when inputs carry none #25655 (dwsmith1983)
  • perf: Fix perf regression in lcm #26023 (neilconway)
  • perf: Optimize left, right #26039 (neilconway)
  • perf: PartitionedTopKRank on the shared store with decide-then-gather #25968 (SubhamSinghal)
  • perf: keep the hash join unique-key fast path when the build side has NULL keys #26018 (jayzhan211)
  • perf: Use sub_view in trim, substr, split_part, and substring_index #26121 (neilconway)
  • perf: Fix performance regression in unary math UDFs #26062 (neilconway)
  • perf: Prune matched right rows in nested loop semi and anti joins #25413 (lyne7-sc)
  • feat: push window predicates on expression PARTITION BY keys #25865 (zhuqi-lucas)
  • perf: optimize aggregate functions over literal arguments #25849 (wudidapaopao)
  • perf(pwmj):settle never-matching streamed rows against the buffered extreme… #25840 (SubhamSinghal)
  • perf: speed up DictionaryArray hashing #26159 (Rich-T-kid)
  • perf: Concretely typed TopK StringHeap, and StringHashTable slot reuse #24105 (MassivePizza)
  • perf: pre-size collections and reuse per-iteration buffers #26192 (comphead)
  • perf: copy bitmaps a word at a time in pre_selection_scatter #25494 (abokhalill)
  • perf: compare nested ScalarValues by type before converting to ArrayData #26066 (alexy)
  • perf: Partitioned topk emit row number #26133 (SubhamSinghal)
  • perf: Optimize upper, lower, translate, replace #26377 (neilconway)
  • perf: Optimize array_filter #25754 (neilconway)
  • perf: avoid building a DFSchema for every pruning comparison #26405 (cetra3)
  • perf: resolve parquet leaf columns in O(1) during statistics collection #26355 (cetra3)
  • perf: append sort-merge join output pairs in bulk #26277 (andygrove)
  • perf: extend fully ordered aggregation to more scalar types #26370 (geoffreyclaude)
  • perf: make LAST_VALUE ... IGNORE NULLS linear in the partition size #26533 (comphead)
  • perf: Optimize array_sort to count validity bits while compacting #26453 (neilconway)
  • perf: add a GroupsAccumulator for array_agg(DISTINCT) #26357 (zhuqi-lucas)
  • IN LIST: retain short integer, floating-point, and fixed-size binary lists #25187 (geoffreyclaude)
  • perf: don't build a parquet stream for a file range with no row group left #26476 (Braedon-Wooding-Displayr)
  • perf: balance file groups by size for scans with an output ordering #26458 (jayzhan211)
  • perf: unroll PrimitiveArray hashing for the no-null path #26191 (Rich-T-kid)
  • perf: memoize LAST_VALUE and NTH_VALUE with IGNORE NULLS #26664 (comphead)
  • perf: create DiskManager Directories-mode working dirs lazily #26583 (raghavyadav01)
  • perf: preserve dictionary encoding through regexp_replace #26186 (Rich-T-kid)
  • perf: opt substr into make_scalar_function dict fast path #25874 (Rich-T-kid)
  • perf: deduplicate found expressions with an IndexSet #26474 (Braedon-Wooding-Displayr)
  • perf: stop re-pairing marked build rows in the scope-key null-aware join #25679 (adriangb)
  • perf: avoid quadratic metadata merging in union schemas #26790 (kumarUjjawal)
  • perf: use native distinct accumulator for COUNT(DISTINCT) on Decimal32/Decimal64 #26821 (tjnangosha)
  • perf: resolve aliased field once during metadata inference #26156 (osipovartem)
  • perf: avoid re-flattening join chains in EliminateCrossJoin #26786 (xudong963)
  • perf: decode Parquet footers for statistics in parallel during planning #26407 (cetra3)
  • perf: build each column's unbounded range once per projection #26879 (Weijun-H)
  • perf(pwmj): build output batches from buffered runs and one streamed … #26287 (SubhamSinghal)

Implemented enhancements:

  • feat: Store only the last row in the previous cursor for round robin tie-breaking purposes #23619 (ariel-miculas)
  • feat(pwmj): support LeftSemi/LeftAnti existence joins via classic scan #23870 (SubhamSinghal)
  • feat(cli): add opt-in Spark function registration #24179 (goutamadwant)
  • feat: union nested Parquet leaves across projection accesses #24130 (goutamadwant)
  • feat: add implicit native coercion constructor #24282 (Jamison929611)
  • feat: Add support for Map type in SQL #24397 (nuno-faria)
  • feat: add sphinx-sitemap dependency and configuration for sitemap generation #24332 (imtherealnaska)
  • feat: Add robots.txt to site #24333 (imtherealnaska)
  • feat: remove extra Rows from ReusableRows #23802 (ariel-miculas)
  • feat: null aware RightAnti hash join execution + planning support #23957 (saadtajwar)
  • feat(substrait): serialize correlated subqueries with OuterReference field references #23488 (clflushopt)
  • feat(parquet): clip nested wrappers during schema pruning #24315 (buraksenn)
  • feat: add ASOF join physical operator #23828 (Xuanwo)
  • feat(spark): add to_binary and try_to_binary #24266 (arsathS)
  • feat(parquet): add bytes_processed scan-completion metric #24524 (adriangb)
  • feat: decimal support for percentile_cont and median #24419 (theirix)
  • feat: derive a distinct count from primary key and unique constraints #24520 (Dandandan)
  • feat: tests for nan and nulls in percentile and median UDAF #24580 (theirix)
  • feat: widen percentile input to use larger precision #24577 (theirix)
  • feat: widen output decimal type for decimal ceil/floor #24703 (theirix)
  • feat: Support null aware hash mark-joins #21585 (AdamGS)
  • feat: extend SLT tests for hash join vs PWMJ, hash join vs SMJ #24805 (comphead)
  • feat(sql): expose CTE scope to relation planners #24755 (geoffreyclaude)
  • feat: include source field context in cast errors #24920 (haohuaijin)
  • feat: Add Spark-compatible encode function to datafusion-spark #21331 (JeelRajodiya)
  • feat(parquet): support page-level limit pruning #25000 (peterxcli)
  • feat: support binary types in octet_length #24867 (eddietejeda)
  • feat: add ASOF joins to DataFrame API #23831 (Xuanwo)
  • feat: add early emit count metric to partial aggregates #24996 (yashrb24)
  • feat: add an async API for spill file writing #24923 (Phoenix500526)
  • feat(pwmj): support RightSemi/RightAnti existence joins #24457 (SubhamSinghal)
  • feat: add strictly_order_preserving for more expr #23922 (rluvaton)
  • feat: transfer parent and dynamic filters across HashJoinExec equi-join keys (inner and semi joins) #25255 (jayzhan211)
  • feat: DISTINCT handling for Aggregate UDFs #25288 (mkleen)
  • feat: destructure proto hooks #25204 (Rich-T-kid)
  • feat: Support lazy per-file Parquet Arrow schema derivation #25343 (peterxcli)
  • feat: push dynamic filters through nested loop joins #25030 (yashrb24)
  • feat(pwmj): support left/right mark joins #25467 (SubhamSinghal)
  • feat(spark): add equal_null #24827 (advitrocks9)
  • feat: support timezone, timezone_hour and timezone_minute in date_part #25163 (adriangb)
  • feat: add GroupColumn support for Decimal32/Decimal64 #25470 (SubhamSinghal)
  • feat: Load Parquet page indexes for external row selections #25460 (haohuaijin)
  • feat: recognize more lossless casts for statistics and ordering #25500 (haohuaijin)
  • feat: support reusable prepared hash-join builds #25491 (sunchao)
  • feat: add partition-aware metrics snapshots #25583 (viirya)
  • feat: log drift between MemoryPool reservations and allocated bytes in SLT #25651 (andygrove)
  • feat: add builders for CreateExternalCatalog and DropCatalog #25785 (TTyChud)
  • feat: Support RunEndEncoded type in approx_distinct #25951 (mkleen)
  • feat: preserve bitmap-backed Parquet row selections #25731 (haohuaijin)
  • feat: add parquet_file_metadata and parquet_page_index to datafusion-cli #25640 (jay-dee7)
  • feat: add Nix flake and direnv support #26005 (VVKot)
  • feat(benchmarks): enable Criterion automatically when a benchmark namespace is set #26210 (Omega359)
  • feat: simplify contradictory ranges across AND/OR predicates #26132 (xudong963)
  • feat: align multi-key equi-join keys with Range partitioning order #25589 (ShayanGho)
  • feat: support WITH ... AS [NOT] MATERIALIZED CTEs #25580 (adriangb)
  • feat(benchmarks): add plan diagnostics to SQL benchmark runners #26212 (Omega359)
  • feat: preserve ordering through to_timestamp_seconds/millis/micros/nanos and to_unixtime #26111 (haohuaijin)
  • feat: rewrite inner joins against a single constant row into a filter #26404 (xudong963)
  • feat(benchmarks): add sql benchmark sort pushdown SQL suite #26245 (Omega359)
  • feat: allow COPY to be disabled independently #25824 (shyjsarah)
  • feat: add SessionConfig::update_extension so libraries can share one ProtoDecoderRegistry #26353 (adriangb)
  • feat: support Spark-compatible stack function #26266 (comphead)
  • feat: derive per-column pruning guarantees from tuple IN lists #25469 (yashrb24)
  • feat: decorrelate a window below a correlated equality filter #26013 (comphead)
  • feat: reuse range partitioning for date_trunc aggregation #26449 (benbellick)
  • feat: support releasing batches in RecordBatchMemoryCounter #26149 (txwyy123)
  • feat: build multi-column null-aware NOT IN on the subquery side #26213 (viirya)
  • feat: show MERGE INTO details in EXPLAIN #26038 (wirybeaver)
  • feat: improve Spark from_utc_timestamp compatibility #25979 (nat-openai)
  • feat: report statistics of materialized CTE scans #26271 (adriangb)
  • feat(perf): compute routing and lookup hashes once #26249 (devanbenz)

Fixed bugs:

  • fix: emit unmatched NULL-key right rows in RIGHT/FULL PiecewiseMergeJoin #24336 (viirya)
  • fix: preserve NULL semantics in bitwise xor simplification #24248 (lyne7-sc)
  • fix(parquet): sync rg_plan to decoder frontier — fix wrong TopK results from re-reading already-delivered row groups (#24352) #24354 (zhuqi-lucas)
  • fix: correct list field inner type in array functions #24345 (timsaucer)
  • fix(parquet): don't runtime-prune row groups while a page-index RowSelection is live (#24355) #24359 (zhuqi-lucas)
  • fix: preserve the input list's inner field in array_append/prepend/replace #24365 (timsaucer)
  • fix: drain PiecewiseMergeJoin output before state transitions #24298 (buraksenn)
  • fix: preserve NULL semantics when simplifying col ~ '.*' #24380 (viirya)
  • fix(spark): correct mod ANSI zero divisor and negative zero handling #23987 (u70b3)
  • fix: consider nulls_first when propagating ordered SortProperties #24206 (Nagato-Yuzuru)
  • fix: skip WindowTopN rewrite when fetch is 0 to avoid PartitionedTopK panic #24405 (viirya)
  • fix: ScalarSubqueryToJoin fails when CTE inside correlated scalar subquery #24363 (Ruchirtripathi)
  • fix: make UnnestExec respect datafusion.execution.batch_size #24384 (andygrove)
  • fix: Streaming aggregation produces wrong result with grouping sets #24422 (2010YOUY01)
  • fix(spark): match Spark abs overflow errors #24156 (goutamadwant)
  • fix: Support RANGE window frames over binary ORDER BY keys #24357 (fornwall)
  • fix: apply struct field filters when the file schema needs adaptation #24125 (adriangb)
  • fix: account for empty scalar subqueries in nullability #24516 (fornwall)
  • fix: expand float fuzz coverage and correct discovered inconsistencies #24433 (Theodus)
  • fix: count UnnestExec output batches after splitting #24482 (goutamadwant)
  • fix: reject duplicate field names during CSV and Parquet schema inference #24387 (sovsparrow)
  • fix: tree-style explain for memory table shows wrong row count #24511 (2010YOUY01)
  • fix: preserve approximate percentile arguments with inline ORDER BY #24411 (haohuaijin)
  • fix: evaluate grouped aggregate arguments after FILTER #24444 (lyne7-sc)
  • fix: preserve nullability for non-empty scalar subqueries #24534 (fornwall)
  • fix: Validate duration format at SET time #24545 (subhramit)
  • fix: Validate filter selectivity at SET time #24541 (subhramit)
  • fix: scan subqueries when advancing the extracted-alias generator #24574 (jayzhan211)
  • fix: allocated_size in MaybeNullBufferBuilder returns incorrect value #24605 (rluvaton)
  • fix: remap SortMergeJoin filter during projection pushdown #24415 (haohuaijin)
  • fix: don't allow zero-argument first_value/last_value/nth_value window functions #24595 (Stazer)
  • fix: decide hash join dynamic filter production at planning time #24601 (adriangb)
  • fix: honor Parquet byte-array statistics ordering #24525 (sunchao)
  • fix: preserve the error type of a failing parquet row filter predicate #24638 (adriangb)
  • fix: annotate to_timestamp array results with the execution timezone #24636 (adriangb)
  • fix: support RANGE window frames over TIME keys #24515 (fornwall)
  • fix(ffi): use the session's LogicalExtensionCodec in create_physical_plan #24690 (timsaucer)
  • fix: reduce RANK window top-K memory from O(input) to O(K + ties) per partition #24591 (jayzhan211)
  • fix: keep join filter columns when pushing a projection below SortMergeJoinExec #24593 (jayzhan211)
  • fix: ensure deferred-filtered outer joins preserve streamed output order #24573 (jayzhan211)
  • fix: drain completed sort-merge join output before awaiting input #24685 (sunchao)
  • fix: spill the NestedLoopJoin build side from the pass that consumed it #24677 (ranflarion)
  • fix(proto): check plan integer conversions across usize boundaries #24483 (buraksenn)
  • fix: hash join dynamic filter with dictionary-encoded join keys #24587 (jayzhan211)
  • fix: preserve explicit Struct casts and evolved field semantics #24680 (sunchao)
  • fix: SortPreservingMerge round-robin tie breaker reads stale poll counts, so tied keys are mostly drained from one partition #24585 (jayzhan211)
  • fix: refuse memory-limited NestedLoopJoin fallback for left-emission joins with a multi-partition probe side #24675 (ranflarion)
  • fix: preserve provided arguments during FFI object construction #24723 (timsaucer)
  • fix: emit deferred unmatched rows when memory-limited NestedLoopJoin exhausts its left side #24746 (viirya)
  • fix: frame IPC messages when reading spill files so batches pin only their own bytes #24592 (jayzhan211)
  • fix: Avoid casting unreachable nested List values #24590 (goutamadwant)
  • fix: charge accumulators' construction-time state to the memory pool in ungrouped aggregation #24692 (ranflarion)
  • fix: reject inconsistent parameterized ScalarValues #24539 (Theodus)
  • fix: Address avg return type overflow for decimals #24371 (AdamGS)
  • fix: Show unique valid field names when an invalid one is provided #24774 (nuno-faria)
  • fix(common): support empty struct in ScalarValue::compact and new_default #24582 (unikdahal)
  • fix: count aggregation_time once per batch instead of once per accumulator #24736 (ranflarion)
  • fix(substrait): preserve window function semantics #22664 (bvolpato)
  • fix: record join_time and output metrics in PiecewiseMergeJoin #24689 (buraksenn)
  • fix: avoid double-counting build_time in HashJoinExec elapsed_compute #24142 (ulrichurriola-parada-ops)
  • fix: ignore exact empty files in ordering analysis #24648 (xudong963)
  • fix: dedup commutative AND/OR operands when canonicalize is disabled #23615 (1fanwang)
  • fix: support RunEndEncoded arrays in function and window frame type coercion #24565 (thorfour)
  • fix: preserve nested projection expressions #24686 (discord9)
  • fix(substrait): preserve grouping set output order #23468 (bvolpato)
  • fix: do not duplicate volatile expressions when extracting leaf expressions #24720 (jaideeppyne)
  • fix: clear memory after emit all in count distinct #24888 (rluvaton)
  • fix: avoid invalid qualifiers in unparsed subqueries #24808 (blinding-pixels)
  • fix: exclude child compute from NestedLoopJoinExec metrics #24491 (goutamadwant)
  • fix: literal-on-left equality no longer collapses the filter to false #24763 (bharadwaj-pendyala)
  • fix(proto): check integer conversions in common options and constraints #24726 (buraksenn)
  • fix: find_in_set internal error with two LargeUtf8 scalar arguments #24802 (advitrocks9)
  • fix: RepartitionExec deadlock when its spill pool holds two open files #24891 (adriangb)
  • fix: preserve external sort workspace across spilling #24740 (sunchao)
  • fix: keep decimal precision for SLT tests #24934 (theirix)
  • fix: prevent DISTINCT ORDER BY LIMIT from returning too few rows #24932 (geoffreyclaude)
  • fix(optimizer): bound planning-time regex compilation #24917 (goutamadwant)
  • fix(spark): accept every integer width in factorial, matching Spark's implicit cast #24943 (ShayanGho)
  • fix: normalize InterleaveExec to UnionExec before enforcing distribution #24959 (jayzhan211)
  • fix: preserve projection metadata during optimization #24670 (gene-bordegaray)
  • fix(spark): derive pmod's decimal result type from the declared arguments #24409 (amitvijapur)
  • fix: stop pre-allocating and undercounting a hash table per COUNT(DISTINCT) group #24857 (adriangb)
  • fix: preserve nulls in array_has_any/all empty-needle path #24986 (yashrb24)
  • fix: format Bytes-category custom metrics with byte units #24218 (DevShiba)
  • fix: preserve null semantics in expression simplification #24258 (fzlzjerry)
  • fix: handle multibyte characters in physical plan tree renderer #24664 (rustyconover)
  • fix: defer InterleaveExec partitioning check to InvariantLevel::Executable #24963 (jayzhan211)
  • fix: accept an empty flags string in regexp_replace #24987 (jackylee-ch)
  • fix: skip count-distinct byte tests under force_hash_collisions #25020 (DevShiba)
  • fix: sort pushed below GlobalLimitExec ignores skip, returning too few rows #24958 (jayzhan211)
  • fix(benchmarks): validate predicate_eval results #25037 (adriangb)
  • fix(ffi): prevent recursive session physical planning #24492 (goutamadwant)
  • fix: preserve fetch across distribution reoptimization #24809 (xudong963)
  • fix: cancel the NestedLoopJoin coordinated fallback when a partition is dropped unfinished #25004 (viirya)
  • fix: preserve output field name when reversing aggregate window expressions #24887 (timsaucer)
  • fix: duplicate groups after spilling in legacy hash aggregation with a fallback group key (type not having dedicated impl) #24889 (rluvaton)
  • fix: aggregate dynamic filtering with unsupported expressions #24817 (lyne7-sc)
  • fix: avoid i8 overflow when computing the wider decimal precision #24851 (dylanpulver)
  • fix: check for key retriever in Parquet decryption properties conversion #24866 (adamreeve)
  • fix: partial reduce should not propegate input ordering #25072 (rluvaton)
  • fix: account for memory that we still hold on when splitting batch #24852 (rluvaton)
  • fix: report correct number of items left in the iterator for IncrementalSortIterator #25107 (rluvaton)
  • fix: correct the copy-pasted docs for covar_pop and bool_or #24990 (jackylee-ch)
  • fix: charge retained scratch indices capacity in GroupsAccumulatorAdapter #24858 (adriangb)
  • fix: accept empty flags in regexp_like and regexp_match #25046 (peterxcli)
  • fix(smj): handle floating-point keys across batch boundaries #25134 (jeffw13)
  • fix: null-aware anti join could emit rows after a sibling probe partition saw NULL #25076 (jayzhan211)
  • fix: reject window functions in WHERE and HAVING during logical planning #25139 (adriangb)
  • fix: prevent panic and incorrect results for COUNT with ORDER BY #24997 (geoffreyclaude)
  • fix: map[key] panics for nested map keys #25130 (neilconway)
  • fix: prevent date_trunc array underflow #25131 (benbellick)
  • fix(ci): pull the MinIO test image from quay.io #25216 (comphead)
  • fix: preserve EXTRACT year comparisons with unrepresentable bounds #25140 (1fanwang)
  • fix: Ensure array_append, array_prepend handle NULLs correctly #25229 (neilconway)
  • fix: Correct array_repeat NULL handling #25230 (neilconway)
  • fix: out-of-bounds read in ArrowBytesMap on a short/long hash collision #25218 (jayzhan211)
  • fix: discard parquet bounds when row group statistics are missing #25228 (haohuaijin)
  • fix: reject groups accumulator for bit_xor(DISTINCT) #24989 (jackylee-ch)
  • fix: preserve computed projections in unions_to_filter #25074 (aoto-tech)
  • fix: cap the field list in "No field named" errors #24931 (akashjainn)
  • fix: preserve target extension metadata in INSERT #24971 (osipovartem)
  • fix: reject NULL partition keys during partitioned writes #24951 (bvolpato)
  • fix: avoid unsafe set rewrites for compound IN lists #24952 (bvolpato)
  • fix: stabilize grouped correlation with centered moments #24953 (bvolpato)
  • fix: preserve missing Parquet null counts #25242 (peterxcli)
  • fix: make Parquet file schema type coercion work on nested schemas #25193 (adragomir)
  • fix: update rustls to address RUSTSEC-2026-0285 #25309 (Xuanwo)
  • fix: return an error for an invalid CSV null_regex instead of panicking #25261 (Developer1010x)
  • fix: report a negative array_resize size as a user error #25179 (1fanwang)
  • fix: report a non-nullable return field for named_struct() and struct() #25306 (adriangb)
  • fix: count each probe row once in HashJoinExec probe_hit_rate and avg_fanout #25272 (CuteChuanChuan)
  • fix: register the sort-merge join stream as a spillable memory consumer #25250 (jayzhan211)
  • fix: evaluate struct-returning UDFs once across repeated field accesses #23691 (tohuya6)
  • fix: prune row groups when file statistics collapse the predicate to a constant #24770 (jensholdgaard)
  • fix: do not use the top-k aggregate stream for a DISTINCT limit with no ordering direction #25349 (adriangb)
  • fix: ignore constant aggregate completion boundaries #25019 (yashrb24)
  • fix: Handle nulls correctly when extracting nested arrays from nullable struct/map #25122 (AdamGS)
  • fix(docs): Fix code snippet in LogicalPlan docs #25377 (martin-g)
  • fix: preserve short-circuit evaluation in Boolean CASE simplification #25143 (emecii)
  • fix: preserve UPPER and LOWER input nullability #25381 (osipovartem)
  • fix: avoid overflow and underflow in vector functions #25289 (namanjain24-sudo)
  • fix: omit ordering fields from the state of order-insensitive aggregates #25402 (hassaanch23)
  • fix: leave memory for aggregate spill replay #25383 (sunchao)
  • fix: do not push filters on volatile group keys below Aggregate #25416 (adriangb)
  • fix: NOT IN (subquery) with a constant value ignores NULLs in the subquery #25348 (adriangb)
  • fix: prevent unsafe integer interval propagation #25234 (haohuaijin)
  • fix: allow free RANGE window frames over Duration and Interval ORDER BY types #24916 (edubraqd)
  • fix(datetime): floor negative scalar timestamps in date_trunc #25430 (benbellick)
  • fix: preserve date_trunc sorting for timezone-aware timestamps #25409 (Toby1009)
  • fix: accept all timestamp precisions in generate_series/range #25173 (adriangb)
  • fix: Spark map_from_arrays, map_from_entries for sliced lists #25431 (neilconway)
  • fix: shrink NLJ load outcome to satisfy Clippy #25492 (Xuanwo)
  • fix: only propagate cast statistics through safe conversions #25227 (haohuaijin)
  • fix: reject LargeList offsets that exceed i32 #25516 (viirya)
  • fix: count Parquet decoding in the scan's elapsed_compute #25486 (akashjainn)
  • fix: account for hash-join row-index storage #25508 (sunchao)
  • fix: duplicate ordering state field names in partial aggregates #25196 (cdelmonte-zg)
  • fix: restrict reverse cast constraint propagation to safe conversions #25531 (haohuaijin)
  • fix: preserve sorts for non-monotonic temporal casts #25526 (Toby1009)
  • fix: reject mixed-sign intervals in range functions #25520 (viirya)
  • fix: preserve statistics for safe casts without extrema #25517 (viirya)
  • fix(substrait): consume chained window functions whose default names collide #25181 (ShayanGho)
  • fix: sort-merge join filter columns follow the filter's own column order #25489 (jayzhan211)
  • fix: compare ScalarValue maps using Arrow ordering #25555 (viirya)
  • fix: Prevent SubqueryAlias to be pushed past FilterExec when unparsing #25547 (nuno-faria)
  • fix: preserve computed names in derived SQL projections #25521 (1fanwang)
  • fix: Emit equality conditions for Substrait CASE base expressions #25191 (namanjain24-sudo)
  • fix: validate trailing ORDER BY keys in free RANGE frames #25518 (viirya)
  • fix: apply MemTable DELETE and UPDATE at execution, not planning #25040 (michaelsembwever)
  • fix: PartitionedTopKRank counts tie rows twice in output_rows #25535 (jayzhan211)
  • fix: preserve nulls when unnesting structs #25389 (1fanwang)
  • fix: Set Substrait output_type on aggregate functions #25090 (namanjain24-sudo)
  • fix: correlated NOT IN with a non-equality correlation returns wrong results #25339 (adriangb)
  • fix(core): reject a DELETE or an UPDATE whose WHERE clause cannot reach the provider #24657 (michaelsembwever)
  • fix: reject unsupported Substrait aggregation phases #25045 (goutamadwant)
  • fix: document empty() as returning a boolean, not 1/0 #25697 (jackylee-ch)
  • fix: reserve the concatenated build batch and computed join keys in HashJoinExec #25496 (jayzhan211)
  • fix: avoid full listings for cached pruned partitions #23402 (kumarUjjawal)
  • fix: validate parquet.compression when it is set #25626 (akashjainn)
  • fix: request a null-aware mark join wherever a NULL mark is observable #25560 (adriangb)
  • fix: drop partition columns already present in the file schema of a ListingTable #25212 (ShayanGho)
  • fix: keep a correlated filter below an aggregate with a grouping set #25529 (namanjain24-sudo)
  • fix: end a sort-merge join partition once its streamed side is exhausted, and release its reservations before the final output #25206 (ranflarion)
  • fix: preserve nested log and power results #25138 (1fanwang)
  • fix: cast lambda variables to their declared type instead of erroring #25418 (LiaCastaneda)
  • fix: apply regexp_instr subexpr to the N-th match, not the first #24991 (MaxFreedomPollard)
  • fix: Derive Substrait intersection nullability from every input #25091 (namanjain24-sudo)
  • fix: drop the redundant Limit above a Sort in the same PushDownLimit visit #25672 (adriangb)
  • fix: plan UNION of untyped aliased placeholders #25675 (adriangb)
  • fix: coerce timezone-naive minus timezone-aware timestamps at equal time units #25240 (adriangb)
  • fix: avoid invalid date_bin ordering after overflow #25692 (viirya)
  • fix: respect volatility for all function expressions #25511 (1fanwang)
  • fix: keep a correlated filter below the nullable side of an outer join #25764 (kita-renji)
  • fix: derive scan equivalences only from filters the source applies exactly #25780 (adriangb)
  • fix: preserve unbounded multiplication interval endpoints #25795 (haohuaijin)
  • fix(ffi): don't discard downcast-delegating wrappers of foreign plans #25749 (timsaucer)
  • fix: Error if both group_by and aggr is empty in AggregateExec::try_new() #25786 (2010YOUY01)
  • fix: Add a subquery when unparsing a Filter depending on a Projection #25549 (nuno-faria)
  • fix: preserve uncertainty in singleton filter statistics #25863 (kumarUjjawal)
  • fix: unparser drops ORDER BY of ordered aggregates #25803 (Vanzeren)
  • fix: apply pushed-down fetch to filter statistics #25862 (kumarUjjawal)
  • fix: Substrait serialization mishandles sliced map literals #25543 (neilconway)
  • fix: reject groups accumulator for NULL-typed bitwise aggregates #25687 (jackylee-ch)
  • fix: xxhash64 of a NULL struct ignores the struct fields #25774 (mrhard9090)
  • fix: to_char returns row 0 for every row with a scalar datetime and a format column #25700 (jackylee-ch)
  • fix: reserve initial table memory in aggregate streams #25900 (rluvaton)
  • fix: Remove serde_json/preserve_order feature from library crates #25884 (adriangb)
  • fix: Avoid special-casing count(1) in Count::value_from_stats #25911 (neilconway)
  • fix: keep the OFFSET of a correlated EXISTS subquery #25284 (fornwall)
  • fix: keep the recovery projection when leaf pushdown would drop a computed same-name column #25445 (adriangb)
  • fix: clamp cosine_distance to enforce metric non-negativity and boundedness [0.0, 2.0] #25744 (IgnatiusPang)
  • fix: preserve compound expression equivalences through projection #25922 (aaron-perpetual)
  • fix: truncate scalar timestamps in their own unit in date_trunc #25498 (namanjain24-sudo)
  • fix: account for every partition in local TopK and limit statistics #25654 (hassaanch23)
  • fix: support zero-field struct group keys in aggregate emit #25282 (fornwall)
  • fix: Validate execution.time_zone at SET time #25788 (KassaSana)
  • fix: keep decimal fractions in modulo by one #25961 (1fanwang)
  • fix: release semi/anti/mark sort-merge join reservations before the final output #25986 (jayzhan211)
  • fix: pull a correlated filter only through plan nodes known to be safe #25838 (adriangb)
  • fix: handle null lists with empty children in Spark's shuffle #25544 (neilconway)
  • fix: Set Substrait output_type on window functions and LIKE #25367 (namanjain24-sudo)
  • fix: keep aggregate limits when combining or rebuilding AggregateExec #25983 (jayzhan211)
  • fix: clear_shrink(0) on rows should reset to original size + single group by fixes in clear_shrink #25914 (rluvaton)
  • fix: allow NULL-padded rows in sort-merge join filter evaluation #26027 (CuteChuanChuan)
  • fix: bin coarse date_bin timestamps in i128 and restore ordering #25815 (viirya)
  • fix: keep filters with correlated subqueries above extension nodes #25294 (kakiuwang-ui)
  • fix: avoid query failures from inferred numeric cast bounds #25528 (haohuaijin)
  • fix: preserve sorts for wrapping integer negation #25568 (haohuaijin)
  • fix: use session statistics registry in dfbench reports #25570 (gabotechs)
  • fix: decide the precedence between PushDownFilter and PushDownLeafProjections #25455 (adriangb)
  • fix: handle zero endpoints in division interval bounds #25985 (haohuaijin)
  • fix: infer LIMIT and OFFSET parameter types #25364 (1fanwang)
  • fix: ignore repeated sort keys in an ordered aggregate's ORDER BY #25400 (hassaanch23)
  • fix: charge semi/anti sort-merge join key groups by their sliced size #25933 (dwsmith1983)
  • fix(substrait): roundtrip conditionless joins #23469 (bvolpato)
  • fix: stop treating arithmetic negation as bitwise NOT #24668 (Amogh-2404)
  • fix: report Parquet Bloom filter pruning metrics accurately #25822 (rgbuilds)
  • fix(spark): return NULL from char for a NULL scalar argument #26126 (ranflarion)
  • fix: align nested piecewise merge join sorting and comparison #26043 (mikamikasuki)
  • fix: keep the running hash for nested dictionaries in Spark xxhash64 #25813 (tohuya6)
  • fix: do not panic when displaying a Date64 scalar of i64::MIN #26139 (geographybuff)
  • fix: reject unsupported DELETE LIMIT #25005 (ryux1)
  • fix: apply spill_compression to RepartitionExec spill files #26142 (adriangb)
  • fix: Emit LIKE the way the Substrait extension defines it #25442 (namanjain24-sudo)
  • fix: preserve local direnv settings #26122 (VVKot)
  • fix: keep a DISTINCT aggregate's arity in SingleDistinctToGroupBy #25461 (namanjain24-sudo)
  • fix: Estimate string equality and LIKE filter selectivity #25835 (limadog9)
  • fix: preserve null treatment when unparsing window and aggregate functions #25475 (Shoryamishra61)
  • fix: report LowerEqual cardinality effect for CoalescePartitionsExec with fetch #26051 (asolimando)
  • fix: build the IN list static filter only from literal-only items #26083 (ranflarion)
  • fix: handle dictionary nulls in null-aware left mark joins #25693 (viirya)
  • fix(spark): derive concat_ws nullability from separator #26205 (VVKot)
  • fix: infer a Parquet column as nullable when some files lack it #26104 (zhuqi-lucas)
  • fix: Unparse random as rand for some dialects #26009 (nuno-faria)
  • fix: do not extract common subexpressions from the arguments of an aggregate with a FILTER #26184 (adriangb)
  • fix(functions): prevent invalid sort omission in log monotonicity when base <= 1 #25746 (IgnatiusPang)
  • fix(substrait): Preserve pushed-down table scan offsets and limits #26035 (viirya)
  • fix: preserve grouping set member names during type coercion #25891 (cetra3)
  • fix: update quinn-proto to resolve cargo audit failures #26253 (haohuaijin)
  • fix: resolve pass-through columns against the input when merging an extraction projection #25412 (adriangb)
  • fix: remove duplicate predicate benchmark result #26244 (Omega359)
  • fix: support RunEndEncoded arguments in MIN/MAX #26248 (grevend)
  • fix: Account for buffered rows in RowCursorStream #25372 (EmilyMatt)
  • fix: count buffers shared between sort input batches once #25820 (andygrove)
  • fix: avoid shifting zero grouping ordinals by 64 #26136 (alexandrefimov)
  • fix: skip HashJoinExec nodes carrying dynamic filters in JoinSelection #26123 (siddubakka)
  • fix: propagate CASE unparser errors #26311 (1cbyc)
  • fix: validate parquet.coerce_int96 when it is set #25628 (akashjainn)
  • fix: report OneOf signature mismatches as planning errors #26329 (SingletonYe)
  • fix: validate cast target type in interval check_support #25562 (shoemoney)
  • fix: plan aggregates over CASE conditions with multiple columns #25684 (1fanwang)
  • fix: preserve negative zero in trunc #25732 (1fanwang)
  • fix: decorrelate grouping sets that leave out the correlated column when safe #25781 (namanjain24-sudo)
  • fix: preserve input strictness in from_unixtime #26414 (haohuaijin)
  • fix: wait for the whole NULL peer group before emitting RANGE N PRECEDING window results #25860 (kita-renji)
  • fix: avoid quadratic row-buffer accounting in sort merges #26366 (geoffreyclaude)
  • fix: forward create_sliding_accumulator to the inner UDAF when aliased #26480 (cool9850311)
  • fix: preserve lambda bindings in nested-loop join projection pushdown #26399 (xudong963)
  • fix: preserve functional dependencies across unnest #25892 (limadog9)
  • fix: validate parquet encoding when configured #25784 (rxdrccq)
  • fix: Avoid pushing down sort under limits #25408 (AdamGS)
  • fix: nvl2 result type should not depend on its first argument #26498 (CuteChuanChuan)
  • fix(functions): pass correct number_rows in gcd benchmarks #26465 (Subodh-17)
  • fix: keep renamed columns intact in leaf projection pushdown #25478 (adriangb)
  • fix: align Union inputs after leaf expression pushdown #26421 (adriangb)
  • fix: validate parquet.coerce_int96_tz when it is set. #26420 (akashjainn)
  • fix: return NULL from bit_xor when a sliding frame has no non-NULL values #26485 (cool9850311)
  • fix: compact sparse string views in FilterExec coalescing #26436 (jayzhan211)
  • fix: keep the time zone when unparsing timestamp literals #25949 (Tyagiquamar)
  • fix: UNNEST now preserves item fields in expression inference and logical-plan schemas #26433 (quwin)
  • fix: preserve list element field in array_repeat #25953 (1fanwang)
  • fix: infer missing nested Parquet fields as nullable #26195 (rgbuilds)
  • fix: correlated exists/not exists subqueries hit the count bug for groupless aggregates #25391 (mohammadnaqvi04)
  • fix: keep EliminateGroupByConstant from requalifying group expressions #26356 (adriangb)
  • fix: account for row duplication and padding in join null counts #25861 (kumarUjjawal)
  • fix: respect ORDER BY in array_agg(DISTINCT) when the aggregate is reversed #26440 (raams95)
  • fix: PushDownLeafProjections yields when the extraction cannot move #26085 (adriangb)
  • fix: do not build TopK dynamic filters from nested sort key thresholds #25999 (jayzhan211)
  • fix: reject null-aware LeftAnti hash join in Partitioned mode #26586 (cool9850311)
  • fix: restore all-features Clippy for DataFrame aggregate rewrites #26590 (kumarUjjawal)
  • fix(datetime): error on out-of-range date_bin timestamp bins #26607 (benbellick)
  • fix: keep a subquery correlated when an aggregate sits above a pulled up scalar aggregate #26544 (comphead)
  • fix: array_concat panics or fails on non-default nested lists #25970 (1fanwang)
  • fix: keep decimal and temporal types in nvl and ifnull #25964 (1fanwang)
  • fix: plan and run repeated SUM(x + literal) on decimals and intervals #25966 (1fanwang)
  • fix: map remaining union input by position in propagate_empty_relation #26557 (moraistejerina)
  • fix: emit completed groups when a fully sorted partial aggregate runs out of memory #26576 (comphead)
  • fix: SortPreservingMergeExec drops input orderings when merging partitions #26654 (HarshilMital)
  • fix: drop build rows with a dictionary NULL key in null-aware anti joins #26578 (abp002)
  • fix: keep probe-side constants in join equivalence properties #26459 (jayzhan211)
  • fix: keep fractional seconds in EXTRACT(EPOCH) for Duration #26676 (mbutrovich)
  • fix(optimizer): preserve correlated columns when removing filter above SubqueryAlias #26551 (princejain756)
  • fix: keep per-partition constant sort keys in a partitioned TopK #26455 (cning112)
  • fix: do not drop GROUP BY keys using nullable UNIQUE constraints #26220 (buraksenn)
  • fix: handle nested RANGE CURRENT ROW peers #26067 (sunchao)
  • fix: evaluate sort-merge join filters that reference no columns #26010 (mrhard9090)
  • fix: report map_extract on a non-map argument as a plan error #25996 (jackylee-ch)
  • fix: validate recursive CTE column coercion #25956 (goutamadwant)
  • fix: apply CSV null_regex when reading, not only when inferring the schema #25254 (Developer1010x)
  • fix: reject correlated IN/NOT IN over a groupless aggregate #26677 (mohammadnaqvi04)
  • fix: keep RANGE default window frame for nullable UNIQUE ORDER BY keys #26721 (viirya)
  • fix: reset aggregate dynamic filters between recursive iterations #26056 (mikamikasuki)
  • fix: unwrap nested encodings inside Dictionary arguments to MIN/MAX #26371 (grevend)
  • fix: preserve TopK dynamic filters across recursive CTE iterations #26670 (Kristofferjoha)
  • fix: WindowTopN resolves the filter column through intermediate proje… #26417 (SubhamSinghal)
  • fix: date_part sub-second parts and result nullability, reusing arrow kernels #26686 (mbutrovich)
  • fix: preserve sorts before limits when projections hide sort keys #26763 (kumarUjjawal)
  • fix: emit completed groups when fully sorted final and single aggrega… #26637 (yashaggarwal85d)
  • fix: compact FIRST_VALUE scalars during merge #26743 (shehabgamin)
  • fix: saturate window frame lookahead instead of overflowing on huge FOLLOWING bounds #26662 (ump45nose)
  • fix: keep build-side input order across batches in hash join #26228 (dwsmith1983)
  • fix: use NDV for unresolved scalar subquery selectivity instead of 20% fallback #25719 (mohitgurav20)
  • fix: Preserve column identifiers in DataFrame fill methods #25857 (limadog9)
  • fix(smj): compare all key types across batch boundaries in semi/anti/mark joins #26830 (d0ra-1h3-3xpl0ra)
  • fix: look up a node's outputs by column, not by flat name, in PushDownFilter #26783 (raghavyadav01)
  • fix: correct NaN grouping and eviction in TopK aggregation #26445 (bvolpato)
  • fix: validate sort pushdown through custom execution plans #26118 (geoffreyclaude)
  • fix: keep a constant sort key when a projection reads a duplicate alias #26397 (cetra3)
  • fix: simplify IN lists containing NULL literals in EliminateFilter #26548 (raghavyadav01)
  • fix(cli): respect sql_parser.recursion_limit when parsing and validating input #25807 (KassaSana)
  • fix: release clustered aggregate table capacity after emitting completed groups on memory pressure #26726 (viirya)
  • fix: don't declare Unique group keys for partial aggregation output #26819 (viirya)
  • fix: spill final hash aggregate output when the pool refuses it #26734 (yashaggarwal85d)
  • fix: reject invalid placeholders in CREATE FUNCTION bodies at definition time #25039 (quwin)
  • fix: validate temporal display formats #26045 (mikamikasuki)
  • fix(spark): accept STRING and TIMESTAMP start dates in next_day #26127 (harsh-ande)
  • fix: bound parallel batch sorting in SortExec #26242 (geoffreyclaude)
  • fix: do not evaluate a volatile coalesce or nvl argument twice #25561 (DeviousCardi)
  • fix(spark): floor instead of truncate in unix_millis/unix_seconds for pre-epoch timestamp #26129 (harsh-ande)
  • fix: propagate field metadata through coalesce, nvl, and nvl2 #24865 (james-willis)
  • fix: release per-batch accumulator scratch after a clustered aggregate emit under memory pressure #26782 (viirya)
  • fix: keep field metadata when the Parquet scan replaces columns with literals #26856 (cblp)
  • fix: keep fractional seconds in MySQL timestamp casts #26886 (hozan23)
  • fix: treat an uncorrelated scalar subquery as a constant in ordering checks #26836 (raghavyadav01)
  • fix: do not fail when an expression's ordering cannot be derived #26884 (jayzhan211)
  • fix: preserve precision in log() with mixed-width float arguments #23676 (U0001F3A2)
  • fix: avoid pruning timestamp-to-time casts #26452 (rgbuilds)
  • fix: keep a projection's field names when decorrelating a predicate s… #26657 (SingletonYe)
  • fix: unparse Int8 casts as SIGNED for MySQL dialect #26883 (hozan23)
  • fix: Preimage rewrites are not allowed for volatile arguments #26995 (vikrantmehta123)
  • fix: Fix flaky sort fuzzer test #26983 (2010YOUY01)
  • fix: retain only valid orderings after order-preserving repartition #26747 (Kristofferjoha)
  • fix: push a limit past every window in a plan, not just the first #26773 (cool9850311)
  • fix: keep a LIMIT inside EXISTS that an operator above it can empty #26640 (raghavyadav01)
  • fix: keep field metadata in the result of min and max #26939 (cblp)
  • fix: keep extension types when skip_metadata clears Parquet field metadata #26854 (cblp)
  • fix(cli): preserve rows in non-table (csv, json, etc) output #26755 (koriyoshi2041)

Documentation updates:

  • chore(deps): bump setuptools from 83.0.0 to 84.0.0 #24301 (dependabot[bot])
  • docs: fix signum documentation to say zero returns 0 #24324 (codeAnqiang-ma)
  • docs: update proto-models README for the datafusion-common dependency #24279 (adriangb)
  • Docs: document manual catalog impls and how compilation got faster #24362 (alamb)
  • docs: Document WindowFn and add comments #24118 (alamb)
  • docs(upgrading): document RecursiveQuery schema field in upgrade guide #23991 (AlvaroBalbin)
  • docs: add an agent skill for auditing datafusion-spark expressions #23893 (andygrove)
  • docs: add Streamling to known users #24439 (sap1ens)
  • docs: Update Concepts, Readings, Events page (add Videos section, recent posts and releases) #24221 (alamb)
  • Update version to 55.0.0, add changelog #24473 (alamb)
  • chore: fix malformed example tables in aggregate function docs #24503 (advitrocks9)
  • doc: more comments to evaluate_selection function #24485 (2010YOUY01)
  • docs: Document missing SQL data types #24471 (nuno-faria)
  • doc: add implementation guidelines for datafusion-spark #24140 (Jefffrey)
  • docs(dataframe): improve dataframe! and from_columns documentation (#… #24434 (cj-zhukov)
  • Revise example usage section headings #22909 (Tchoupinax)
  • Add nested struct schema evolution support for Map types #23914 (kosiew)
  • docs: document negative n behavior for nth_value #24557 (lyne7-sc)
  • docs: Add London DataFusion meetup to events page #24521 (alamb)
  • Raise hash_join_single_partition_threshold to 4MB #24558 (Dandandan)
  • docs: list DataFusion Bloom extension #24575 (YimingQiao)
  • chore: fix the remaining malformed example tables in function docs #24596 (advitrocks9)
  • chore: migrate HawkEye to v7 #24578 (tisonkun)
  • docs: Fix rendering of get_record_batch_memory_size buffer diagram #24637 (alamb)
  • docs: Document expectations of PartialReduce aggregate mode #24640 (alamb)
  • docs: add runnable examples for input_file_name() and file_row_index() #24634 (alamb)
  • docs(catalogs): fix punctuation and spacing #24693 (Tim-Raphael)
  • docs: Remove stale "temporary fix" note from get_record_batch_memory_size #24674 (alamb)
  • docs: Upate governance page #24749 (alamb)
  • docs: Clarify the testing section in GitHub PR Template #24798 (2010YOUY01)
  • docs: fix three dead links to datafusion.apache.org #24872 (rawsun007)
  • docs: fix 'sqllite' typo in sqllogictest README #24825 (MsfPablo)
  • Optimize NULL-containing string IN list pruning #24954 (kumarUjjawal)
  • Add docs section for debugging for contributors #24946 (mkleen)
  • Refine and document AggregateExec metrics #24757 (kosiew)
  • chore: Fix duplicated word typos in comments and docs #25048 (geographybuff)
  • fix, perf: Use Arrow comparator for key comparison in map_extract #24999 (neilconway)
  • chore: Clean up out-of-date comments for aggregate refactor #25007 (2010YOUY01)
  • docs: fix the Substrait round-trip instructions in the sqllogictest README #24855 (alexandrefimov)
  • Doc: note enforce_batch_size_in_joins only applies to symmetric hash join #24811 (2010YOUY01)
  • docs: Point SessionConfig setter/getter docs to the config definitions #25102 (2010YOUY01)
  • docs: Point ParquetFormat setter/getter docs to the ParquetOptions definitions #25103 (2010YOUY01)
  • ci: require extended tests in the merge queue #25203 (kumarUjjawal)
  • chore: update version 55.1.0 (#25070) #25200 (timsaucer)
  • dev: include Markdown link checks in the local lint suite #25287 (kumarUjjawal)
  • chore: Update Rust toolchain to 1.98.1 #25295 (neilconway)
  • dev: include ASF status checks in the local lint suite #25313 (kumarUjjawal)
  • dev: include security auditing in the local lint suite #25359 (kumarUjjawal)
  • v55 upgrade guide: Remove unreleased-note #25396 (mag1c1an1)
  • docs: move misplaced upgrade notes to DataFusion 55 #25331 (haohuaijin)
  • Add aggregate-internal submetrics #25051 (kosiew)
  • docs: Add SKILL.md for designing benchmark #25285 (2010YOUY01)
  • dev: include dependency checks in the local lint suite #25399 (kumarUjjawal)
  • chore: Update sqlparser to 0.63.0 #25278 (alamb)
  • ci: skip queue-verified Rust jobs on pushes to main #25365 (kumarUjjawal)
  • dev: include examples README checks in the local lint suite #25466 (kumarUjjawal)
  • chore: Update to arrow/parquet 60.0.0 and object_store 0.14.2 #25335 (alamb)
  • ci: move the large file check into a script shared with rust_lint.sh #25440 (akashjainn)
  • dev: include generated config and function docs in the local lint suite #25525 (kumarUjjawal)
  • Fix outdated Python DataFrame API documentation link #25225 (dhruvatr)
  • docs: document DELETE and UPDATE in the SQL reference #25553 (michaelsembwever)
  • docs: document DELETE and UPDATE for SQL users and table provider authors #24567 (michaelsembwever)
  • dev: include the HTML documentation build in the local lint suite #25586 (kumarUjjawal)
  • docs: Remove extra commas in count results #25552 (nuno-faria)
  • docs: Fix formatting of allow_single_file_parallelism #25551 (nuno-faria)
  • docs: clarify what PushedDown::Yes and PushedDown::No mean #25670 (adriangb)
  • docs: note the rustdoc trick for hiding use statements in examples #25718 (asolimando)
  • refactor: track NestedLoopJoin fallback matches in one left bitmap and drop the cancel protocol #25542 (jayzhan211)
  • docs: document the TIMESTAMP WITH TIME ZONE type mapping and fix a stale comment #25171 (adriangb)
  • docs: Add November 2026 San Francisco DataFusion meetup to events page #25703 (alamb)
  • docs: point proto CONTRIBUTING at proto-models/gen #25666 (ZainnQureshii)
  • docs: Make the preference for sqllogictest over Rust unit tests more visible #25756 (alamb)
  • docs: Add narwhals-datafusion to integrations list #25387 (s5dsn-eqee)
  • chore(docs): Fix a typo (us -> is) #25898 (martin-g)
  • chore: Upgrade to Rust 1.99, MSRV to 1.95 #25935 (neilconway)
  • docs: add community showcase volumes 4, 5 and 6 #25939 (alamb)
  • doc: add comments for LocalLimitExec and GlobalLimitExec #25993 (2010YOUY01)
  • ci: Enforce PR limit to 3 for non-committers #25973 (2010YOUY01)
  • docs: mention output_rows_skew on partitioned operators in explain-usage #26048 (LiaCastaneda)
  • Update findepi's affiliation in governance doc #26053 (findepi)
  • chore(deps): bump pydata-sphinx-theme from 0.20.0 to 0.21.0 in the all-uv-deps group across 1 directory #24843 (dependabot[bot])
  • docs: Update release README examples to 55.2.0 and add SVN commands #26119 (comphead)
  • doc: Improve docs for configuration 'max_passes' #26102 (2010YOUY01)
  • Update contributor guidelines regarding AI spam, reviews, code #24510 (Jefffrey)
  • chore(docs): Remove entries for two known limitations which were fixed recently #26113 (martin-g)
  • docs: preserve released documentation during deployment #25880 (yiaany)
  • docs: add datafusion-arrowmetal to the integrations list #26004 (singhpratech)
  • docs: show array_distinct keeping first-seen order #25930 (rawsun007)
  • docs: add DataFusion security guidance #26064 (efegokdemir)
  • docs: Add distributed DataFusion blog post to readings page #26206 (alamb)
  • docs: document ^ as the bitwise xor operator #26224 (rawsun007)
  • Document and enforce compatibility between an aggregate's accumulators #25710 (masonh22)
  • chore(deps): bump pydata-sphinx-theme from 0.21.0 to 0.23.0 in the all-uv-deps group #26320 (dependabot[bot])
  • docs: fix missing word in BytesLikeIndexMap field comment #26472 (aarushkandukoori)
  • doc: Document the physical optimizer contract #26351 (2010YOUY01)
  • docs(minor): Explain the role of EnsureRequirements in the physical optimizer pipeline #26479 (alamb)
  • docs: update committer list in governance page #26502 (alamb)
  • ci: run the Rust workflow for fork pull requests on the fork #25591 (kumarUjjawal)
  • chore: Move more CI jobs to run on ubuntu-slim #26571 (neilconway)
  • docs: add examples for walking and rewriting logical plans #25954 (mattsu2020)
  • chore: check for circular dependencies with cargo publish #26582 (neilconway)
  • chore(deps): bump setuptools from 84.0.0 to 84.2.0 in the all-uv-deps group #26611 (dependabot[bot])
  • docs: fix SQL examples that the default dialect cannot parse #26373 (rawsun007)
  • Update Parseable description in user guide #26714 (yashdotrv)
  • ci: draft inactive PRs awaiting author action #26715 (kumarUjjawal)
  • docs(dataframe): add dataframe aggregation example (#26597) #26600 (cj-zhukov)
  • chore: Update Rust toolchain to 1.99.1 #26778 (neilconway)
  • docs: add Mycorr to known users #26770 (maxschub)
  • chore(deps): bump setuptools from 84.2.0 to 84.2.1 in the all-uv-deps group #26797 (dependabot[bot])
  • docs: add version picker and manual release snapshots #24792 (yiaany)
  • chore: Use short Apache license header #26761 (2010YOUY01)
  • ci: cancel obsolete merge queue runs #26760 (kumarUjjawal)
  • Make full microbenchmark execution local-only #26766 (kosiew)
  • Point readers at tpchgen-cli for TPCH SLT tests #26724 (AdamGS)
  • docs: Add point queries blog to Good Reads and Tuning Guide #26751 (alamb)
  • ci: add slt-only test coverage #26219 (2010YOUY01)
  • docs: publish older version docs from versions.json with a workflow and build script #26828 (alamb)
  • chore: Update to arrow/parquet 60.1.0 #26867 (alamb)
  • docs: add 55.2.0 and 54.1.0 to the version picker #26829 (alamb)
  • Set version on main to 56.0.0-dev, document release versioning scheme #26845 (alamb)
  • ci: Skip macOS tests in forks #27017 (sdf-jkl)
  • [branch-56] fix: read Hive's null partition placeholder as NULL (backport #26310) #27171 (xanderbailey)
  • [branch-56] feat: support ALP parquet encoding in writer options (backport #27036) #27093 (alamb)
  • [branch-56] fix: reject a multi-key null-aware join without a NOT IN value key count (backport #27111) #27176 (alamb)

Other:

  • bench: add standard window function cases to bounded_window #24151 (neilconway)
  • Add branch-55 protection #24293 (timsaucer)
  • chore: Update depcheck toolchain and dependencies #24289 (neilconway)
  • chore: Expand dependabot coverage #24290 (neilconway)
  • Enable more lints #24066 (emilk)
  • refactor(hash-aggr): migrate ordered single aggregation #24259 (2010YOUY01)
  • Minor: avoid a few clones in sum accumulator setup #24309 (alamb)
  • chore(deps): bump the all-other-cargo-deps group across 1 directory with 3 updates #24320 (dependabot[bot])
  • perf(catalog-listing): cut datafusion-catalog-listing compile time ~4.4x #24330 (Dandandan)
  • bench: add parquet_row_filter_skip suite for the fully-matched RowFilter skip #24328 (zhuqi-lucas)
  • chore(deps): bump Swatinem/rust-cache from 2.8.1 to 2.9.2 in /.github/actions/setup-macos-aarch64-builder #24307 (dependabot[bot])
  • chore(deps): bump taiki-e/install-action from 2.85.10 to 2.85.11 #24306 (dependabot[bot])
  • chore(deps-dev): bump webpack-cli from 5.1.4 to 7.2.2 in /datafusion/wasmtest/datafusion-wasm-app #24302 (dependabot[bot])
  • chore(deps): bump pygithub from 2.8.1 to 2.9.1 #24299 (dependabot[bot])
  • chore(deps): bump rich from 14.3.2 to 15.0.0 #24305 (dependabot[bot])
  • chore(deps-dev): bump webpack from 5.105.0 to 5.109.2 in /datafusion/wasmtest/datafusion-wasm-app in the all-npm-deps group across 1 directory #24300 (dependabot[bot])
  • chore(deps): bump typing-extensions from 4.15.0 to 4.16.0 #24304 (dependabot[bot])
  • chore(deps): bump requests from 2.33.0 to 2.34.2 #24303 (dependabot[bot])
  • Reduce datafusion-spark dependency footprint #24351 (dentiny)
  • Fix ArrowBytesViewMap retained capacity accounting #24257 (kosiew)
  • Perf: remove extra allocations for hex operations #24057 (Rich-T-kid)
  • Document decimal AVG wrapping arithmetic #23685 (sakshichitnis27)
  • bench: pwmj left semi/anti join #24160 (SubhamSinghal)
  • Feat: add dictionaries as a supported group column type #23187 (Rich-T-kid)
  • Add per-aggregate argument evaluation metrics for grouped hash aggregation #24024 (kosiew)
  • Add focused benchmark for Parquet pruning setup cache #24323 (kosiew)
  • refactor: CrossJoinStream (simplifying, less state, async generator pattern) #24291 (saadtajwar)
  • chore(deps): bump astral-sh/setup-uv from 9.0.0 to 10.0.1 #24452 (dependabot[bot])
  • chore(deps): bump taiki-e/install-action from 2.85.11 to 2.86.1 #24451 (dependabot[bot])
  • chore(deps): bump tomlkit from 0.14.0 to 0.15.1 #24448 (dependabot[bot])
  • chore(deps): bump the codeql-actions group across 1 directory with 2 updates #24449 (dependabot[bot])
  • Enable more clippy lints #24322 (emilk)
  • Add dfbench statistics command #23975 (gabotechs)
  • Fix cargo audit by updating h2 to 0.4.16 #24467 (alamb)
  • docs: Update dev/release/README.md for new crate datafusion-proto-models #24472 (alamb)
  • minor(test): strengthen bitwise sort-merge join spill coverage #24192 (buraksenn)
  • chore: group uv dependency updates #24408 (kumarUjjawal)
  • test(physical-plan): add colocated field-level plan serde tests #24222 (adriangb)
  • bench: pwmj right semi/anti existence joins #24458 (SubhamSinghal)
  • Apply same sparse routing for range partition dynamic filter as hash partition and restructure build_filter #24417 (peterxcli)
  • minor: Add alias of some SQL types (string, binary, uint64) #24398 (nuno-faria)
  • chore: benchmarks compare.py documentation fixes #24505 (adamreeve)
  • introduce new dictionary benchmarks #24416 (Rich-T-kid)
  • refactor: Unify Spark format_string integer numeric assembly #24480 (goutamadwant)
  • refactor(proto): destructure plan and proto structs in join serde hooks #24164 (adriangb)
  • Fix self join leaf expressions #24242 (cetra3)
  • Add per-aggregate accumulator phase metrics #24423 (kosiew)
  • minor: document EquivalenceProperties::project #24063 (zhuqi-lucas)
  • chore(deps): bump the all-other-cargo-deps group across 1 directory with 2 updates #24450 (dependabot[bot])
  • refactor(parquet): strip empty row groups so PreparedAccessPlan stays well-formed #24509 (zhuqi-lucas)
  • ci: Check datafusion-spark feature combinations to catch publish errors #24475 (alamb)
  • test: add range-partitioned sorted time-bin aggregation coverage #24500 (NGA-TRAN)
  • bench(functions): Improve build time by reducing 63 bench targets into 9 #24342 (Dandandan)
  • Enable more clippy lints #24466 (emilk)
  • refactor(proto): destructure plan and proto structs in core plan serde hooks #24167 (adriangb)
  • chore: remove DynEq and DynHash re-exports #24552 (2010YOUY01)
  • chore: remove get_statistics_with_limit #24549 (2010YOUY01)
  • test: end-to-end regression tests for view array spill compaction (replaces #21325) #24547 (adriangb)
  • refactor: Simplify and optimize Parquet projection planning for nested casts #24522 (jayzhan211)
  • refactor(parquet): extract InitialDecoderState from the opener decoder-setup block #24572 (zhuqi-lucas)
  • feat(CI): Introduce xtask to keep local test and CI commands in sync #24083 (2010YOUY01)
  • chore: add missing since versions to #[deprecated] attributes #24544 (alamb)
  • Align scalar UDF literal args with coerced return fields #23232 (yinli-systems)
  • Fix physical distinctness expression nullability #24244 (blinding-pixels)
  • Sql benchmark sync updated queries #24583 (Omega359)
  • chore: replace MaybeNullBuilder with trait over arrow NullBuilder #24606 (rluvaton)
  • chore(deps): bump the all-other-cargo-deps group with 2 updates #24653 (dependabot[bot])
  • chore(deps): bump cargo from 0.98.0 to 0.99.0 in /dev/depcheck #24649 (dependabot[bot])
  • chore(deps): bump pygithub from 2.9.1 to 2.10.0 in the all-uv-deps group #24650 (dependabot[bot])
  • chore(deps): bump the codeql-actions group across 1 directory with 2 updates #24651 (dependabot[bot])
  • tests: Add IN list coverage for FixedSizeBinary #24660 (alamb)
  • Fix topk emitted batches metric #24481 (cetra3)
  • IN LIST: reuse primitive filters for FixedSizeBinary #24102 (geoffreyclaude)
  • Add aggregate-specific metrics to legacy grouped hash and TopK #24523 (kosiew)
  • chore(deps): bump taiki-e/install-action from 2.86.1 to 2.86.5 #24652 (dependabot[bot])
  • Fix panic on RightMark hash joins when propagating ordering #24718 (Tpt)
  • test: check sql_example tables line up #24645 (advitrocks9)
  • IN LIST: Handle dictionary inputs once #24662 (geoffreyclaude)
  • bench(functions-aggregate): benchmark coalescing peer first_value into a struct (#23682) #24559 (zhuqi-lucas)
  • main branch test fix: adjust dynamic filter assertion for hash collisions handling #24729 (jayzhan211)
  • Account for transformer-retained batches in symmetric hash join #24364 (kosiew)
  • fix(physical-expr): stop false infeasibility report from Eq+FALSE in cp_solver #24464 (Smallfu666)
  • Add per-aggregate metrics to non-grouped AggregateStream #24706 (kosiew)
  • Support config matrix for SLT tests #24493 (comphead)
  • Fix OneSideHashJoiner memory size accounting #24744 (kosiew)
  • refactor(proto): destructure Arrow and Avro source serde hooks #24696 (buraksenn)
  • Adapt existing partitioning for unsatisfied inputs in co-partitioned joins #24600 (stuhood)
  • refactor(proto): destructure CASE and IN-list expression serde hooks #24700 (buraksenn)
  • refactor(asof_join): enhance match value handling and comparator caching #24519 (jayzhan211)
  • IN LIST: use runtime dispatch for FixedSizeBinary filter (reduce binary size) #24687 (alamb)
  • Fix sum by name in custom metrics #24673 (gabotechs)
  • bench: add grouped aggregate filter benchmarks #24748 (lyne7-sc)
  • bench: add window aggregate filter benchmarks #24589 (lyne7-sc)
  • SQL benchmark configuration parity updates #24778 (Omega359)
  • bench: add simplify_function_over_case benchmark #24489 (radmirnovii)
  • Enable more clippy lints #24566 (emilk)
  • refactor: thread PhysicalOptimizerContext through join_selection stats helpers #24716 (asolimando)
  • Enable more clippy lints #24837 (emilk)
  • SQL benchmark configuration parity updates - #24778 2 #24803 (Omega359)
  • bench: multi_group_by cases to exercise List #24824 (mzabaluev)
  • array_sort / array_has: monomorphize per native type instead of per Arrow type (reduce binary size) #24823 (jayzhan211)
  • Align metadata propagation through Physical and Logical casts #23169 (paleolimbot)
  • chore(deps-dev): bump browserslist from 4.28.1 to 4.28.8 in /datafusion/wasmtest/datafusion-wasm-app #24862 (dependabot[bot])
  • chore(deps): bump the all-other-cargo-deps group with 7 updates #24840 (dependabot[bot])
  • chore(deps): bump taiki-e/install-action from 2.86.5 to 2.87.1 #24845 (dependabot[bot])
  • chore(deps): bump the codeql-actions group across 1 directory with 2 updates #24844 (dependabot[bot])
  • chore(deps): bump blake2 from 0.10.6 to 0.11.0 #24841 (dependabot[bot])
  • chore(deps-dev): bump the all-npm-deps group in /datafusion/wasmtest/datafusion-wasm-app with 2 updates #24839 (dependabot[bot])
  • refactor(hash-aggr): Let partial-reduce mode aggregation early emit when OOM #24785 (2010YOUY01)
  • minor: reenable failing mark join test #24863 (comphead)
  • chore: convert FinalHashAggregateStream to async generators and cleanup #24874 (rluvaton)
  • chore: update chacha20 dependency #24880 (timsaucer)
  • Prevent array_position panic for minimum start positions #24838 (1fanwang)
  • chore(deps-dev): bump fast-uri from 3.1.5 to 3.1.7 in /datafusion/wasmtest/datafusion-wasm-app #24882 (dependabot[bot])
  • bench(h2o): window Top-N sweep over a declared-sorted input #24732 (SubhamSinghal)
  • test: skip count distinct spill memory test under forced hash collisions #24918 (rluvaton)
  • refactor(hash-aggr): route grouping sets to the migrated hash streams #24870 (2010YOUY01)
  • Unify temporal field parsing with DatePart #24906 (sdf-jkl)
  • refactor(proto): destructure lambda expression serde hooks #24775 (buraksenn)
  • Add support for dictionary for approx_distinct #24646 (mkleen)
  • Support planning scalar functions in ExprPlanner #24846 (MichaelScofield)
  • Read primitive values directly in branchless IN LIST filters #24747 (geoffreyclaude)
  • refactor: simplify null-aware and join-type branching in hash join #24957 (jayzhan211)
  • chore: convert PartialHashAggregateStream to async generators and cleanup #24017 (rluvaton)
  • feat(physical-plan): GroupColumn support for List / LargeList #23648 (mzabaluev)
  • refactor(hash-aggr): Small fixes and finish hash aggregation refactor #24961 (2010YOUY01)
  • bench: add PWMJ vs NestedLoopJoin criterion coverage for LeftMark/Rig… #24955 (SubhamSinghal)
  • Preserve field metadata when coercing INT96 timestamps #24790 (ryzhyk)
  • change: fully deprecated the acceptance of value field in VirtualTables #24346 (eliot1480)
  • test: cover ArrayStaticFilter and dictionary flattening IN list #25001 (peterxcli)
  • substrait: add an extension point for per-field Arrow metadata on user-defined types #25015 (moraistejerina)
  • bench: add a ClickBench extended query for a grouped COUNT(DISTINCT) over a string #25026 (adriangb)
  • bench: discover ClickBench query files in sql_planner instead of hardcoding ranges #25027 (adriangb)
  • bench: scalar functions over dictionary-encoded columns #24586 (radmirnovii)
  • Fix Postgres decimal formatter clippy #24975 (osipovartem)
  • Add special case in validate_struct_compatibility for empty structs #25023 (pepijnve)
  • Cover the configMatrix runner path with an end-to-end sqllogictest fixture #24878 (bharadwaj-pendyala)
  • chore(deps): bump the arrow-parquet group with 9 updates #25060 (dependabot[bot])
  • chore(deps): bump runs-on/action from 2.3.0 to 2.3.1 #25061 (dependabot[bot])
  • chore(deps-dev): bump webpack from 5.110.1 to 5.110.3 in /datafusion/wasmtest/datafusion-wasm-app in the all-npm-deps group #25059 (dependabot[bot])
  • chore(deps): bump taiki-e/install-action from 2.87.1 to 2.87.6 #25062 (dependabot[bot])
  • chore(deps): bump zstd from 0.13.3 to 0.14.0 #25064 (dependabot[bot])
  • chore(deps): bump the all-other-cargo-deps group across 1 directory with 6 updates #25063 (dependabot[bot])
  • chore: collapse duplicated branches that differ by one value #24814 (2010YOUY01)
  • chore: mark partial reduce stream as can handle back pressure (spillable) #25075 (rluvaton)
  • test: add regression coverage for terminal hash aggregate drain #24853 (hhhizzz)
  • Add write support to Avro format #24925 (pepijnve)
  • bench(predicate_eval): add expensive-first and wide-column shapes #25032 (adriangb)
  • Fix issue #24701 Better error when the query contains reserved names #24715 (Ruchirtripathi)
  • fix(physical-plan): report Final emission for LeftSemi hash and neste… #24976 (TinyMurky)
  • Allow a non-distinct count alongside the single distinct aggregate in SingleDistinctToGroupBy #24859 (adriangb)
  • minor: make MinIO tests more stable #25092 (comphead)
  • test: make nested_key_spill_keeps_groups_unique independent of scheduling #25156 (rluvaton)
  • chore: check license headers in SQL logic tests #25182 (Xuanwo)
  • Fix AvgGroupsAccumulator::size() self-overcount #25117 (kosiew)
  • Feat: allow fine tuning output record batch size for repartionionExec #25176 (Rich-T-kid)
  • Prepare SQL runner for native benchmark migration #24806 (Omega359)
  • ci: cache dependencies for feature checks #25231 (kumarUjjawal)
  • ci: share extended test commands through xtask #25256 (kumarUjjawal)
  • Fix Numeric signature coercion to properly handle null types #24988 (Jefffrey)
  • IN LIST: optimize Utf8View and BinaryView filters #24088 (geoffreyclaude)
  • Reduce binary size of ScalarUDFImpl default impls #24966 (simonvandel)
  • fix(physical-plan): honor distinct soft limits in SingleHashAggregateStream #25158 (TinyMurky)
  • Reduce repetitive string formatting in push_projection_dedupl #25236 (wyatt-herkamp)
  • Support ORDER BY ALL for projected expressions #25243 (osipovartem)
  • refactor: clean up aggregation planning #25104 (2010YOUY01)
  • bench: add ASOF join benchmarks #23833 (Xuanwo)
  • Support predicate subqueries in projections #24972 (osipovartem)
  • Push down topk through join #21621 (SubhamSinghal)
  • bench: cover dictionary group keys with a shared values array #25198 (LiaCastaneda)
  • ci: share workspace check artifacts with feature checks #25249 (kumarUjjawal)
  • test: bound merge fan-in in ordered aggregate spill tests #25252 (kumarUjjawal)
  • chore(deps): bump rstest from 0.26.1 to 0.27.0 #25323 (dependabot[bot])
  • chore(deps): bump the all-other-cargo-deps group with 3 updates #25322 (dependabot[bot])
  • chore(deps): bump the codeql-actions group across 1 directory with 2 updates #25319 (dependabot[bot])
  • chore(deps): bump astral-sh/setup-uv from 10.0.1 to 10.1.0 #25320 (dependabot[bot])
  • chore(deps): bump taiki-e/install-action from 2.87.6 to 2.87.12 #25321 (dependabot[bot])
  • Support INSERT OVERWRITE for MemTable #24969 (osipovartem)
  • docs(physical-optimizer): warn that wrapper rules must forward schema_check #25357 (zhuqi-lucas)
  • refactor: exhaustively destructure dynamic filter and scalar subquery proto hooks #25178 (peterxcli)
  • chore(docs): Fix the rendering of an expression in docstring #25379 (martin-g)
  • fix(physical-plan): CoalescePartitionsExec panic on wasm32-unknown-unknown #24890 (kentkwu)
  • chore(docs): Fix the documentation for TableProvider::scan()'s limit argument #25397 (martin-g)
  • bench: SQL benchmark suite for null-aware (NOT IN) joins #25386 (adriangb)
  • Avoid normalizing hidden values in sliced list set operations #25300 (yinli-systems)
  • ci: skip queue-verified Dev and Dependencies jobs on pushes to main #25443 (comphead)
  • bench: add projection_subquery SQL benchmark suite #25346 (adriangb)
  • Release consumed sort input batches after partial output #25406 (Weijun-H)
  • chore(deps): bump soupsieve from 2.8.4 to 2.9 #25444 (dependabot[bot])
  • bench: keep the expect_plan check out of the measured region #25441 (adriangb)
  • chore: Add helpers for computing OffsetBuffer span, length #25481 (neilconway)
  • Match Parquet map key/value by position in nested schema coercion #25342 (adriangb)
  • minor: remove redundant state from the NestedLoopJoin fallback coordinator #25533 (jayzhan211)
  • minor: move create_group_accumulator out of the legacy aggregate stream #25539 (jayzhan211)
  • minor: derive group_column_supported_type from make_group_column #25541 (jayzhan211)
  • test: bound merge fan-in in the final hash aggregate spill test #25424 (kumarUjjawal)
  • perf: Avoid copying when materializing output in OrderedPartialAggregateStream #25312 (2010YOUY01)
  • refactor(hash-aggr): share one spill context between the spilling aggregate streams #25538 (jayzhan211)
  • refactor: factor SortMergeJoinExec::execute into a reusable sort_merge_join_stream #25490 (jayzhan211)
  • FilterExec : expose push_batch_with_filter on LimitedBatchCoalescer #25503 (Rich-T-kid)
  • test: characterize correlated NOT IN null-aware behaviour #25558 (adriangb)
  • chore(deps): bump the all-other-cargo-deps group with 4 updates #25598 (dependabot[bot])
  • chore(deps): bump codecov/codecov-action from 7.0.0 to 7.1.1 #25597 (dependabot[bot])
  • Un-deprecate ScalarSubqueryExpr::index() #25609 (masonh22)
  • chore(deps): bump the codeql-actions group across 1 directory with 2 updates #25594 (dependabot[bot])
  • chore(deps): bump taiki-e/install-action from 2.87.12 to 2.87.16 #25595 (dependabot[bot])
  • bench: add spill_views SQL benchmarks for spilling StringView/BinaryView columns #25625 (adriangb)
  • ci: remove audit ignore of quick-xml advisories #25600 (Phoenix500526)
  • chore(deps): bump jlumbroso/free-disk-space from 1.3.1 to 2.0.0 #25596 (dependabot[bot])
  • chore(deps-dev): bump webpack from 5.110.3 to 5.111.1 in /datafusion/wasmtest/datafusion-wasm-app in the all-npm-deps group #25593 (dependabot[bot])
  • refactor: Make AggregateExec state modeling and public updates safe (part 1: distinct limit) #25564 (2010YOUY01)
  • bench: make the spill_views memory limits overridable from the environment #25634 (adriangb)
  • bench: report per-query memory pool peaks from the Criterion SQL harness #25644 (adriangb)
  • Fix EliminateLimit not eliminating limits under LogicalPlan::Extension nodes. #25577 (masonh22)
  • test: characterization suite for timestamps with time zone #25175 (adriangb)
  • fix datafusion-cli CI: switch from minio to rustfs #25706 (a10y)
  • refactor: remove redundant join-flattening precheck #25685 (Toby1009)
  • docs(physical-plan): add doctest example for StatisticsContext::compute #25689 (asolimando)
  • Avoid formatting errors for expected missing physical columns #25664 (strawgate)
  • chore: Use preinstalled clang for wasm CI job #25649 (neilconway)
  • Make is_complete() public #25738 (gabotechs)
  • Fix panic from avg(x ORDER BY y) (and more) #25711 (masonh22)
  • Fix panic in PercentileContGroupsAccumulator::convert_to_state() #25755 (masonh22)
  • test: PostgreSQL differential coverage for timestamps with time zone #25164 (adriangb)
  • perf(functions-aggregate): coalesce small ordered ARRAY_AGG batches #25497 (TinyMurky)
  • refactor: Simplify optimizer traversal with TreeNode::exists() #25769 (2010YOUY01)
  • refactor: use apply for physical plan invariant checks #25770 (2010YOUY01)
  • refactor: simplify filter pushdown volatility check #25768 (2010YOUY01)
  • test: cover unsorted contiguous groups in one partition #24737 (xavlee)
  • Refresh statistics parser settings between query files #24738 (EltonChang1)
  • ci: Add target release version labels to merged PRs #25776 (nuno-faria)
  • ci: Ensure only the main datafusion crate is used in the examples #25777 (nuno-faria)
  • ci: Checkout repo first in the Release Version Labeler workflow #25845 (nuno-faria)
  • datafusion-physical-plan: handles nulls correctly in aggregate dynamic filters #25579 (jayshrivastava)
  • chore(deps): bump thiserror from 2.0.20 to 2.0.21 in the all-other-cargo-deps group #25869 (dependabot[bot])
  • chore(deps): bump astral-sh/setup-uv from 10.1.0 to 10.2.0 #25868 (dependabot[bot])
  • chore(deps): bump taiki-e/install-action from 2.87.16 to 2.87.21 #25867 (dependabot[bot])
  • chore(deps): bump the codeql-actions group across 1 directory with 2 updates #25866 (dependabot[bot])
  • chore(deps): bump webpack-dev-middleware from 8.0.4 to 8.3.0 in /datafusion/wasmtest/datafusion-wasm-app #25879 (dependabot[bot])
  • benchmarks: draw simulated object store latency at random, not round-robin #25809 (adriangb)
  • bench: add AND conjunction benchmarks to binary_op #25870 (Abhisheklearn12)
  • chore: remove unreachable code and cleanup data type matching + support interval in min/max groups acc #25872 (rluvaton)
  • Expand scalar async UDF results to the batch row count #25828 (hareshkh)
  • chore(deps): bump pyjwt from 2.13.0 to 2.15.0 #25918 (dependabot[bot])
  • chore(deps): bump urllib3 from 2.7.0 to 2.8.0 #25919 (dependabot[bot])
  • chore: gate Parquet opener encryption on parquet_encryption #25907 (neilconway)
  • Return provider extensions directly from the statistics walk #25826 (kosiew)
  • test: close five mutation-testing gaps in leaf expression pushdown guards #25452 (adriangb)
  • Make RowValues::compare's current-offset assumption self-checking #25643 (nathanb9)
  • test: consolidate partition metrics tests and clarify snapshot docs #25848 (viirya)
  • fix(physical-plan): replace approximate map_size accounting with exact HashTable::allocation_size() in GroupValuesRows #25750 (mohitgurav20)
  • Make StatisticsContext cache entries node-lifetime safe #25929 (kosiew)
  • Prune hash join probe side via a build-side key range bitmap #25602 (gruuya)
  • test: enable DurationSecond fuzzing #24772 (Theodus)
  • chore(SingleHashAggregateStream): refactor to async generator implementation #24016 (buraksenn)
  • test: strengthen scalar UDF type recovery regression coverage #25967 (Toby1009)
  • bench: add array_compact benchmarks #25566 (neilconway)
  • bench: Fix panic in power benchmark #26028 (neilconway)
  • bench: Improve benchmarks for left, right #26026 (neilconway)
  • Add skewness metric to nodes that execute in partitioned mode #25977 (LiaCastaneda)
  • Add SECURITY.md for DataFusion #25917 (alamb)
  • Plan struct colon access as get_field #25373 (osipovartem)
  • chore(deps): bump taiki-e/install-action from 2.87.21 to 2.87.23 #26075 (dependabot[bot])
  • chore(deps): bump runs-on/action from 2.3.1 to 2.5.0 #26076 (dependabot[bot])
  • Add regression tests for nullable NOT IN expressions #25994 (kosiew)
  • chore(PartialReduceHashAggregateStream): convert to async generators and cleanup #24015 (rluvaton)
  • chore(deps): bump compression from 1.8.1 to 1.8.2 in /datafusion/wasmtest/datafusion-wasm-app #26079 (dependabot[bot])
  • chore(deps): bump proxy-addr from 2.0.7 to 2.0.8 in /datafusion/wasmtest/datafusion-wasm-app #26078 (dependabot[bot])
  • chore(deps): bump the all-other-cargo-deps group with 6 updates #26077 (dependabot[bot])
  • test: differential fuzz for leaf expression pushdown and parquet filter pushdown #25453 (adriangb)
  • Fix GroupValues retained memory accounting #25188 (kosiew)
  • docs(physical-plan): add doctest examples for StatisticsRegistry, ExtendedStatistics, and compute_extended #25699 (asolimando)
  • test: consolidate ASOF SQL coverage #24964 (Xuanwo)
  • GC & Deduplicate String View on Spill #23565 (cetra3)
  • chore(deps): bump shell-quote from 1.10.0 to 1.12.0 in /datafusion/wasmtest/datafusion-wasm-app #26117 (dependabot[bot])
  • refactor: Simplify LimitedDistinctAggregation optimizer rule, push limit in more cases #26069 (2010YOUY01)
  • bench: introduce create_hash benchmarks #25952 (Rich-T-kid)
  • chore: Add "Willingness to contribute" dropdown to issue templates #26120 (comphead)
  • Make TopKDynamicFilters public #25429 (masonh22)
  • perf(sort-merge): skip null checks in ArrayValues when neither cursor has nulls #26215 (zhuqi-lucas)
  • bench: expand dictionary group-by coverage #25303 (yinli-systems)
  • refactor: simplify redundant iterator chains #26251 (comphead)
  • Fix retained-capacity accounting for grouped byte MIN/MAX #26154 (kosiew)
  • bench: model remote object storage faithfully for --simulate-latency #24093 (adriangb)
  • Preserve list element metadata when rebuilding scalar values #26092 (osipovartem)
  • refactor: replace MapOffset tuple with ProbeOffset enum #26203 (CuteChuanChuan)
  • bench: classic pwmj sql benchmark #25839 (SubhamSinghal)
  • Add tests fpr nullability of AND/OR expressions #26164 (siddubakka)
  • Fix GroupOrdering memory descriptor double-counting #26074 (kosiew)
  • DiskManager: include the description in file names #26110 (Tpt)
  • chore: bump rustls to fix audit #26352 (Jefffrey)
  • chore(deps): bump actions/upload-artifact from 7.0.1 to 7.0.2 #26324 (dependabot[bot])
  • chore(deps): bump actions/download-artifact from 8.0.1 to 8.0.2 #26325 (dependabot[bot])
  • chore(deps): bump astral-sh/setup-uv from 10.2.0 to 10.3.0 #26323 (dependabot[bot])
  • chore(deps): bump actions/setup-node from 7.0.0 to 7.1.0 #26326 (dependabot[bot])
  • chore(deps): bump taiki-e/install-action from 2.87.23 to 2.87.27 #26322 (dependabot[bot])
  • chore(deps): bump the codeql-actions group across 1 directory with 2 updates #26321 (dependabot[bot])
  • test: use WITH ORDER tables for date_bin/date_trunc ordering plan tests #26332 (viirya)
  • Keep ClickBench SQL in sync across runners #25773 (kosiew)
  • Account for TopK common-prefix converter memory #26286 (kosiew)
  • refactor: extract aggregate physical planning #26403 (1cbyc)
  • fix(physical-plan): replace approximate map_size accounting with exac… #25825 (mohitgurav20)
  • chore(ci): Register the substrait check job with runs-on, route spark checks through xtask #25195 (namanjain24-sudo)
  • minor: fold tpcds_planning test into core_integration binary #26422 (alamb)
  • Support Preimage for date_trunc #22114 (sdf-jkl)
  • chore(deps): bump the all-other-cargo-deps group across 1 directory with 9 updates #26495 (dependabot[bot])
  • refactor: deduplicate concat string and binary builders #26318 (andygrove)
  • test: add NULL values to Fuzz window #26386 (Nagato-Yuzuru)
  • Show nullability of an explicit CastExpr target in its display #26359 (zhuqi-lucas)
  • refactor: retain hash join build batches after allocation failure #26182 (peterxcli)
  • Resolve named windows referenced inside OVER (...) #26400 (zhuqi-lucas)
  • bench: add ASOF join benchmark with a large unfiltered right input #26483 (Kristofferjoha)
  • chore(deps): bump runs-on/action from 2.3.1 to 2.5.0 #26526 (dependabot[bot])
  • chore(deps): bump tomlkit from 0.15.1 to 0.15.2 in the all-uv-deps group #26525 (dependabot[bot])
  • chore(deps): bump rustyline from 18.0.1 to 19.0.0 #26524 (dependabot[bot])
  • chore(deps): bump taiki-e/install-action from 2.87.27 to 2.87.29 #26523 (dependabot[bot])
  • chore(deps): bump the codeql-actions group across 1 directory with 2 updates #26522 (dependabot[bot])
  • chore(deps): bump the all-other-cargo-deps group with 3 updates #26521 (dependabot[bot])
  • chore(deps-dev): bump webpack from 5.111.1 to 5.112.0 in /datafusion/wasmtest/datafusion-wasm-app in the all-npm-deps group #26520 (dependabot[bot])
  • minor: combine datafusion-ffi test binaries into ffi_integration #26423 (alamb)
  • chore(deps): bump certifi from 2026.1.4 to 2026.10.19 #26546 (dependabot[bot])
  • chore(deps): bump launch-editor from 2.14.1 to 2.14.2 in /datafusion/wasmtest/datafusion-wasm-app #26547 (dependabot[bot])
  • Resolve named windows in HAVING #26515 (zhuqi-lucas)
  • refactor: share a single pad kernel between lpad and rpad #26367 (shazilkhan)
  • minor: combine set_comparison test into core_integration binary #26419 (alamb)
  • chore: Use value_field to fetch list element field #26567 (neilconway)
  • chore: run the circular dependency check on the fork for fork PRs #26574 (neilconway)
  • chore: Skip datafusion-sqllogictest in semver check #26577 (neilconway)
  • chore: move datafusion-ffi integration tests into their own crate #26564 (jprochazk)
  • chore(deps): bump rustls-webpki and webpki-root-certs to fix security… #26550 (yashaggarwal85d)
  • DataFrame API: allow aggregate functions in select() 2 (#17874) #25482 (cj-zhukov)
  • chore: skip slow test under force_hash_collisions #26594 (neilconway)
  • chore(deps): bump rstest from 0.27.0 to 0.28.0 #26615 (dependabot[bot])
  • chore(deps): bump runs-on/action from 2.3.1 to 2.5.0 #26612 (dependabot[bot])
  • chore(deps): bump the all-other-cargo-deps group with 2 updates #26614 (dependabot[bot])
  • chore(deps-dev): bump webpack-dev-server from 6.0.0 to 6.1.0 in /datafusion/wasmtest/datafusion-wasm-app in the all-npm-deps group #26610 (dependabot[bot])
  • chore(deps): bump actions/setup-node from 7.0.0 to 7.1.0 #26613 (dependabot[bot])
  • test: add planner tests for ordering, constant and equality facts #26570 (jayzhan211)
  • chore: use shallow checkout for semver check #26604 (neilconway)
  • chore: speedup semver check using shared target directory #26598 (neilconway)
  • ci: stop re-running the workspace test suite in the extended tests job #26649 (jayzhan211)
  • ci: halve extended test fuzz time by computing the join fuzz reference once #26644 (jayzhan211)
  • Fix new security advisory: RUSTSEC-2026-0385 #26666 (AdamGS)
  • refactor: keep the hash join NOT IN rules in one module #26647 (jayzhan211)
  • test: cover the memory accounting paths of the grouped aggregate streams that no test reaches #26562 (jayzhan211)
  • fix(physical-optimizer): decline the WindowTopN rewrite when the FilterExec carries a fetch #23599 (zhuqi-lucas)
  • Fix literal projection distinct count statistics #26385 (masonh22)
  • ci: check all features with Clippy #26591 (kumarUjjawal)
  • chore(csv): use shared aggr_test_schema in CSV scan tests #26681 (ninglinLiu)
  • refactor(spark): deduplicate bucket computation in width_bucket_interval_mdn_exact #26489 (Shahzeb-DataEngineer)
  • refactor: Implement optimizer hook for combining partial/final aggregate in AggregateExec #25962 (2010YOUY01)
  • chore(deps-dev): bump fast-uri from 3.1.7 to 3.1.9 in /datafusion/wasmtest/datafusion-wasm-app #26709 (dependabot[bot])
  • refactor: use ProbeCompletion in NestedLoopJoinExec #26717 (CuteChuanChuan)
  • ci: disable incremental compilation correctly #26735 (kumarUjjawal)
  • ci: run extended tests from one test target instead of name filters #26694 (jayzhan211)
  • test: Fix flaky window slt by adding rowsort #26764 (2010YOUY01)
  • chore(deps-dev): bump serialize-javascript from 7.0.5 to 7.1.4 in /datafusion/wasmtest/datafusion-wasm-app #26708 (dependabot[bot])
  • test: keep fuzz decimal values within the declared precision #26774 (jayzhan211)
  • Fix 'dependant' typo in explain_analyze test comments #26804 (aarushkandukoori)
  • chore(deps): bump parking_lot from 0.12.5 to 0.13.0 #26802 (dependabot[bot])
  • chore(deps): bump Swatinem/rust-cache from 2.9.2 to 2.9.3 in /.github/actions/setup-macos-aarch64-builder #26800 (dependabot[bot])
  • chore(deps): bump Swatinem/rust-cache from 2.9.2 to 2.9.3 #26799 (dependabot[bot])
  • chore(deps): bump taiki-e/install-action from 2.87.29 to 2.87.31 #26798 (dependabot[bot])
  • chore(deps): bump astral-sh/setup-uv from 10.3.0 to 10.3.2 #26796 (dependabot[bot])
  • chore(deps): bump the codeql-actions group across 1 directory with 2 updates #26795 (dependabot[bot])
  • chore(deps-dev): bump webpack from 5.112.0 to 5.112.2 in /datafusion/wasmtest/datafusion-wasm-app in the all-npm-deps group #26794 (dependabot[bot])
  • chore(deps): bump the all-other-cargo-deps group across 1 directory with 5 updates #26801 (dependabot[bot])
  • datafusion-physical-expr: add DynamicFilterPhysicalExpr::update_from(Self) method #26534 (jayshrivastava)
  • Stop requiring microbenchmark tests (amd64) check #26811 (kosiew)
  • Ordering tests on dictionary types #26834 (matthewmturner)
  • test: generate NULLs in equivalence fuzz tests #26086 (Nagato-Yuzuru)
  • bench: add stacked wide projection cases to sql_planner #26841 (zhuqi-lucas)
  • chore(deps): bump qs from 6.15.3 to 6.17.0 in /datafusion/wasmtest/datafusion-wasm-app #26833 (dependabot[bot])
  • test: pin parquet created_by in tests so expected output is independent of the version string #26892 (alamb)
  • Align statistics cache sharing test comments with reset-free contract #25965 (kosiew)
  • dev: Allow new release version to be automatically updated #26007 (nuno-faria)
  • Fix retained memory accounting for ExprIntervalGraph #26487 (kosiew)
  • test: reduce spill-presence fixture sizes #26840 (bvolpato)
  • Feat/map sql extension types #21714 (cyberbeam524)
  • minor: Speedup one external sort test #26928 (2010YOUY01)
  • test: cover grouping set equivalences and constants #26898 (0xdeafbeef)
  • minor: speedup one slow aggregate unit test #26925 (2010YOUY01)
  • minor: speedup one SPM unit test #26929 (2010YOUY01)
  • refactor: pin satisfied sort keys in a walk instead of adding them as constants #26853 (jayzhan211)
  • refactor: consolidate benchmark options and environment settings #26768 (mattsu2020)
  • Move filter for HAVING and QUALIFY when volatile aliases are used above projection #26850 (matthewmturner)
  • refactor: Centralize CI command definitions in xtask #26765 (2010YOUY01)
  • refactor: build the parquet stats future once before choosing whether to spawn it #26952 (alamb)
  • test: count evaluations of volatile calls in sqllogictests #25458 (adriangb)
  • bench: bound sort-axis smoke inputs #26837 (bvolpato)
  • test: Add regression tests for some unwrap_cast related bugs that don't reproduce #27018 (AdamGS)
  • typo: UnionExtractFun to UnionExtractFunc #27009 (AdamGS)
  • [branch-56] Revert #25790 (Decimal filter selectivity) due to TPC-H Q18 regression (#27145) #27147 (asolimando)
  • [branch-56] perf: prune oversized dictionaries before IPC spilling (backport #27015) #27107 (alamb)
  • [branch-56] feat: show extension type names in DESCRIBE and information_schema.columns (backport #26893) #27057 (alamb)
  • [branch-56] fix: give each SortExec reset_state copy its own TopK dynamic filter (backport #27142) #27174 (raghavyadav01)
  • [branch-56] fix: infer UNION output types before analysis (backport #27139) #27178 (alamb)
  • [branch-56] fix: preserve constant equivalence properties through projection (backport #27123) #27179 (alamb)
  • [branch-56] fix: count the largest capacity seen for a shared buffer address (backport #27097) #27175 (alamb)

Credits

Thank you to everyone who contributed to this release. Here is a breakdown of commits (PRs merged) per contributor.

    95	dependabot[bot]
    74	Adrian Garcia Badaracco
    54	Yongting You
    52	Jay Zhan
    48	Neil Conway
    45	Andrew Lamb
    32	Kumar Ujjawal
    26	Liang-Chi Hsieh
    25	kosiew
    23	Qi Zhu
    22	Huaijin
    19	Burak Şen
    19	Subham Singhal
    18	Raz Luvaton
    17	Oleks V
    17	Stefan Wang
    16	RIchard Baah
    15	Goutam Adwant
    15	Xuanwo
    13	Geoffrey Claude
    13	Nuno Faria
    13	linfeng
    11	Daniël Heres
    11	namanjain24-sudo
    10	Adam Gutglick
    10	Chao Sun
    10	Peter Lee
     9	Alessandro Solimando
     9	Artem Osipov
     9	Bruno Volpato
     9	Tim Saucer
     8	Andy Grove
     8	Bruce Ritchie
     8	Peter L
     7	Mason
     7	Yu-Chuan Hung
     7	jackylee
     7	ranflarion
     7	xudong.w
     6	Fredrik Fornwall
     6	Malingshu
     6	Martin Grigorov
     6	Michael Kleen
     6	Pepijn Van Eeckhoudt
     6	RAGHVENDRA KUMAR YADAV
     6	Yash Bhisikar
     6	akashjainn
     6	dustin
     6	theirix
     5	Advit Arora
     5	Ben Bellick
     5	Emil Ernerfeldt
     5	Lía Adriana
     4	Cheng-Yi Shih
     4	Gabriel
     4	Jeffrey Vo
     4	Kristoffer Vestergård Johansson
     4	Nagato Yuzuru
     4	Rishabh Gupta
     4	Roshan Ramani
     4	Sergey Zhukov
     4	Shayan Gh
     4	mck
     3	Abhishek
     3	Bharadwaj Pendyala
     3	Braedon Wooding
     3	Jayant Shrivastava
     3	Kosta Tarasov
     3	Mohit Gurav
     3	Muhammad Hassaan
     3	Theo Butler
     3	Thomas Tanon
     3	Tiny_Murky
     3	Vova Kot
     3	Yash Aggarwal
     3	Yin Li
     3	Yury Syrovetsky
     3	limadog9
     3	mika
     2	Aarush Kandukoori
     2	Adam Reeve
     2	Alejandro Borrego
     2	Aleksandr Efimov
     2	Alex Huang
     2	Amogh Ramesh
     2	Anurag Tryambak Raut
     2	Ariel Miculas-Trif
     2	Christian Del Monte
     2	David Greven
     2	DevShiba
     2	Ethan Tran
     2	Ignatius Pang
     2	Ilya Krasnov
     2	Isaac Emmanuel
     2	Jorge Morais
     2	Kassahun Sanayew
     2	Kita Renji
     2	Matt Butrovich
     2	Matthew Turner
     2	Michal Piatkowski
     2	Mikhail Zabaluev
     2	Mohammad Naqvi
     2	Narendra
     2	Patrick Ribbsaeter
     2	Phoenix
     2	QinXi
     2	Radmir Mukhambetov
     2	Ruchir Tripathi
     2	Ryu
     2	S Prajwall Narayana
     2	Saad Tajwar
     2	Singleton
     2	Stu Hood
     2	Subhramit Basu
     2	Varun
     2	Xiaozhe Yu
     2	Xuanyi Li
     2	blinding-pixels
     2	discord9
     2	geographybuff
     2	hozan23
     2	lazzypanda7
     2	mattsu
     2	siddu
     2	xavlee
     1	0x70
     1	@clflushopt
     1	Alexy Khrabrov
     1	Alvaro Balbin
     1	Amit Vijapur
     1	Andrei Dragomir
     1	Andrew Duffy
     1	Anna-Rose Lescure
     1	Bill Easton
     1	Chao Ning
     1	DeviousCardi
     1	Dewey Dunnington
     1	Dhruva Kumar
     1	Dmitri B.
     1	Dylan Pulver
     1	Eddie A Tejeda
     1	Eduardo Alcantara
     1	Efe
     1	Elton Chang
     1	Emanuele Micheletti
     1	Emily Matheys
     1	Gene Bordegaray
     1	George Stamatakis
     1	Han-Yin Chang
     1	Haresh Khanna
     1	Harshil Mital
     1	Hasnaat hussain
     1	Huang Qiwei
     1	Jaideep Pyne
     1	James Willis
     1	Jamison-J
     1	Jan Procházka
     1	Jasdeep Singh
     1	Jeff Wang
     1	Jens Holdgaard Pedersen
     1	Jeremy Schoemaker
     1	Justin O'Dwyer
     1	Justus Flerlage
     1	Kent Wu
     1	LFC
     1	Leonid Ryzhyk
     1	Luca Bigon
     1	Marcelo Tesla
     1	Marko Grujic
     1	Max Freedom Pollard
     1	Maximilian Georg Schuberth
     1	Mohd Quamar Tyagi
     1	Morax
     1	MrHard
     1	MsfPablo
     1	Nathan
     1	Nga Tran
     1	NingLin Liu
     1	Parafee41
     1	Parman Mohammadalizadeh
     1	Patrick Rank
     1	Piotr Findeisen
     1	Prince Jain
     1	QQQ
     1	Reid Kaufmann
     1	Renan GEHAN
     1	Rohan Krishnaswamy
     1	Rusty Conover
     1	Rusty Raven
     1	Shahzeb Ali
     1	Shehab Amin
     1	Sherwin Tan
     1	Shoryamishra61
     1	Shrivas Shankar
     1	Simon Vandel Sillesen
     1	Subodh
     1	TTyChud
     1	Tchoupinax
     1	Thor
     1	Tim Raphael
     1	Tobias Schwarzinger
     1	Unik Dahal
     1	Vikrant Mehta
     1	Vladimir
     1	WaterWhisperer
     1	WeblWabl
     1	Wyatt Herkamp
     1	Xander
     1	Xinyao Zhang
     1	Yaroslav Tkachenko
     1	Yifan Chen
     1	Yiming Qiao
     1	Yousef Mahmoud
     1	YukiAoto
     1	Yunus Ozen
     1	Yuwei Zhao
     1	Zain Qureshi
     1	Zeel Rajodiya
     1	Zeren Wang
     1	aaron-perpetual
     1	aetos
     1	anchor
     1	arsathS
     1	batu
     1	cyberbeam524
     1	dentiny
     1	dora
     1	eliot1480
     1	fys
     1	jdymitarai
     1	kid
     1	mag1c1an1
     1	mightsleep
     1	mrhard9090
     1	nangosha
     1	nat-openai
     1	niebayes
     1	raams95
     1	sakshichitnis27
     1	shazilkhan
     1	shyjsarah
     1	singhpratech
     1	tison
     1	ulrichurriola-parada-ops
     1	ump45nose
     1	yashdotrv
     1	yoongbok lee

Thank you also to everyone who contributed in other ways such as filing issues, reviewing PRs, and providing feedback on this release.