Repository navigation
Upgrade to Arrow 60 and use native Variant conversion kernels - #92
Merged
Merged
Conversation
This was referenced Oct 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Upgrade Arrow, Parquet, and Variant crates to released 60.0.0 and pin DataFusion to compatible commit
484183dc34ea565190d9cd4a66db6d24789f95d0. This removes the previous Arrow fork patches and raises the Rust requirement to 1.95.Typed getters now use Arrow’s native Variant conversion kernels. Remove the local unshredding and output-storage adapter, and adapt to the new Arrow and DataFusion APIs.
variant_get_jsonretains JSON serialization; typed string extraction follows Arrow’s conversion behavior.Known string-conversion and Variant-target discrepancies are skipped with TODOs while preserving desired expectations, rather than adding compatibility overrides or asserting incorrect results. Path coverage remains active using string fixtures.
Validation:
cargo test: 77 passed, 4 ignored; SQL suite passed.cargo clippy --all-targets -- -D warningspassed.cargo fmt --checkand diff whitespace checks passed.variant_getandvariant_to_jsonbenchmark smoke checks passed (not performance measurements).