You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Release arrow-rs / parquet Major version 59.0.0 (May 2026) #9110
changed the title [-]Release arrow-rs / parquet Major version 59.0.0 (April 2026)[/-][+]Release arrow-rs / parquet Major version `59.0.0` (April 2026)[/+]on Jan 7, 2026
changed the title [-]Release arrow-rs / parquet Major version `59.0.0` (April 2026)[/-][+]Release arrow-rs / parquet Major version `59.0.0` (May 2026)[/+]on Feb 23, 2026
I'd like to put forward #9683 as a requested item for this release.
It adds an MSD radix sort kernel (arrow_row::radix::radix_sort_to_indices) that sorts directly on row-encoded keys, taking advantage of the row format already being big-endian and memcmp-comparable. It consistently beats lexsort_to_indices for multi-column sorts (up to ~2.5x at 32K rows), and the docs are upfront about where it loses.
Worth landing because it's:
Correct — 17 tests covering sort-option combinations, nulls, NaN/Inf, threshold boundaries, a fuzz test, and cross-validation against comparison sort on the same Rows.
Well documented — clear guidance on when to use it vs lexsort_to_indices, backed by benchmarks.
Foundational — operates on Rows, returns indices, composes with take. A building block for downstream sorting work, not a full sort-path replacement.
There's already downstream interest in building on it. I have a series of (now-closed) experimental DataFusion PRs exploring integration into ExternalSorter:
These showed the kernel's utility but also that getting a real win needs a careful redesign of the sort stream rather than a drop-in swap. Landing the kernel in a release would give DataFusion (and other downstream consumers) something stable to build that work against.
I tried to make an RC (details below) but the verification script fails:
Emaikl
Hi,
I would like to propose a release of Apache Arrow Rust Implementation, version 59.0.0.
This release candidate is based on commit: 9f96a8f5c3f47c6303c7d17ac7f00ad4ad0df513 [1].
The SHA256 of the release candidate is: a22e39069a3bbcc02a9ba9837b36e6b87686c3bad18fe0a762aa4851f35499d8
The proposed release tarball and signatures are hosted at [2].
The changelog is located at [3].
The release tracking issue is: [4]
Please download, verify checksums and signatures, run the unit tests,
and vote on the release. There is a script [4] that automates some of
the verification.
The vote will be open for at least 72 hours.
[ ] +1 Release this as Apache Arrow Rust 59.0.0
[ ] +0
[ ] -1 Do not release this as Apache Arrow Rust 59.0.0 because...
[1]: https://github.com/apache/arrow-rs/tree/9f96a8f5c3f47c6303c7d17ac7f00ad4ad0df513
[2]: https://dist.apache.org/repos/dist/dev/arrow/apache-arrow-rs-59.0.0-rc1
[3]: https://github.com/apache/arrow-rs/blob/9f96a8f5c3f47c6303c7d17ac7f00ad4ad0df513/CHANGELOG.md
[4]: https://github.com/apache/arrow-rs/blob/master/dev/release/verify-release-candidate.sh
[5]: https://github.com/apache/arrow-rs/issues/9110
But the verification script fails like this:
failures:
---- bad_data::test_invalid_files stdout ----
thread 'bad_data::test_invalid_files' (96123527) panicked at parquet/tests/arrow_reader/bad_data.rs:68:9:
Found new file in bad_data, please add test: ARROW-GH-47662.parquet
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
failures:
bad_data::test_invalid_files
test result: FAILED. 85 passed; 1 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.79s
error: test failed, to rerun pass `-p parquet --test arrow_reader`
I think we need to update the test. I'll see if I can figure out what to do in the morning (but I am basically out of time now)
Hmmm, I had to bump the parquet-testing commit to pull in the test file for IEEE754, and with that file came another failure file. But that PR wasn't committed...does the build process update submodules?
Anyway, I have a fix for that in #9619 I can split out and submit today.
I looked at the verify script, and it clones directly from parquet-testing, rather than going through git submodule. That's why it pulled the most recent HEAD.
Activity
[-]Release arrow-rs / parquet Major version 59.0.0 (April 2026)[/-][+]Release arrow-rs / parquet Major version `59.0.0` (April 2026)[/+][-]Release arrow-rs / parquet Major version `59.0.0` (April 2026)[/-][+]Release arrow-rs / parquet Major version `59.0.0` (May 2026)[/+]alamb commentedon Feb 23, 2026
Given that the 58.0.0 release didn't go out until almost the end of February, moving this date back by a month
58.0.0(January 2026) #8466Remove deprecated parquet::format module and thrift dependency (#9962)
xanderbailey commentedon May 17, 2026
Would love to see #9203 in the 59 release if that's possible. We'd really like this for the iceberg-rust encryption work
alamb commentedon May 20, 2026
Merged!
etseidl commentedon May 20, 2026
It would be nice to get #9995 in during this breaking change window.
theirix commentedon May 31, 2026
I'd appreciate it if these arrow-buffers PRs #9418 and #9453 could be included in the release - already reviewed, and I've applied suggestions
alamb commentedon Jun 1, 2026
Will do
alamb commentedon Jun 1, 2026
I plan to make an RC for 59 this week
AdamGS commentedon Jun 1, 2026
If any maintainer has some spare cycles to review a performance-only change - #9625
mbutrovich commentedon Jun 1, 2026
I'd like to put forward #9683 as a requested item for this release.
It adds an MSD radix sort kernel (
arrow_row::radix::radix_sort_to_indices) that sorts directly on row-encoded keys, taking advantage of the row format already being big-endian and memcmp-comparable. It consistently beatslexsort_to_indicesfor multi-column sorts (up to ~2.5x at 32K rows), and the docs are upfront about where it loses.Worth landing because it's:
Rows.lexsort_to_indices, backed by benchmarks.Rows, returns indices, composes withtake. A building block for downstream sorting work, not a full sort-path replacement.There's already downstream interest in building on it. I have a series of (now-closed) experimental DataFusion PRs exploring integration into
ExternalSorter:ExternalSorterExternalSorterrefactorThese showed the kernel's utility but also that getting a real win needs a careful redesign of the sort stream rather than a drop-in swap. Landing the kernel in a release would give DataFusion (and other downstream consumers) something stable to build that work against.
CC @Dandandan
brancz commentedon Jun 2, 2026
It's already approved, but would really appreciate it if this could make it into 59: #10039
alamb commentedon Jun 2, 2026
Merged!
Remove deprecated parquet::format module and thrift dependency (apach…
3 remaining items
alamb commentedon Jun 4, 2026
Ok, I think we have everything we planned for the release
Now I just need
59.0.0release #10063And then I'll cut the RC
Prepare for `59.0.0` release (#10063)
alamb commentedon Jun 4, 2026
I tried to make an RC (details below) but the verification script fails:
Emaikl
But the verification script fails like this:
I think we need to update the test. I'll see if I can figure out what to do in the morning (but I am basically out of time now)
etseidl commentedon Jun 4, 2026
Hmmm, I had to bump the parquet-testing commit to pull in the test file for IEEE754, and with that file came another failure file. But that PR wasn't committed...does the build process update submodules?
Anyway, I have a fix for that in #9619 I can split out and submit today.
etseidl commentedon Jun 4, 2026
Submitted #10077, hopefully that will fix this.
I looked at the verify script, and it clones directly from parquet-testing, rather than going through git submodule. That's why it pulled the most recent HEAD.
Filed #10078 to track this.
Add test for `parquet-testing/bad_data/ARROW-GH-47662.parquet` (#10077)
alamb commentedon Jun 5, 2026
RC2 is out for voting
https://lists.apache.org/thread/xlozjylbqfo7tgh2lcvb6d3dvj5bwwxd
Thank you @etseidl
alamb commentedon Jun 9, 2026
The release was approved and is now available on crates.io
The next release is tracked here
59.1.0(June 2026) #9878Thank you all for the help
chore: Update to arrow/parquet 59.0.0 (apache#22744)
chore: Update to arrow/parquet 59.0.0 (apache#22744)
chore: Update to arrow/parquet 59.0.0 (apache#22744)
Cherry-pick apache#22744 and apache#23312 (#157)
Add test for `parquet-testing/bad_data/ARROW-GH-47662.parquet` (apach…
[58_maintenance] Add test for `parquet-testing/bad_data/ARROW-GH-4766…