Repository navigation
[Variant] Move VariantType from parquet-variant-compute to parquet-variant - #12005
Merged
Merged
Conversation
Schema-only consumers can now use the `arrow.parquet.variant` extension type without depending on the compute kernels or JSON conversion. `parquet-variant-compute` re-exports it, so the existing path keeps working. Closes apache#11952. Co-Authored-By: Claude Opus 5.5 <[email protected]>
github-actions
Bot
added
parquet-variant
sdf-jkl
marked this pull request as ready for review
October 30, 2026 20:18
Member
Author
|
@alamb @Rich-T-kid @Jefffrey PTAL |
alamb
approved these changes
Oct 30, 2026
| mod variant_to_arrow; | ||
|
|
||
| pub use variant_array::{ShreddingState, VariantArray, VariantArrayIter, VariantType}; | ||
| pub use parquet_variant::VariantType; |
Contributor
There was a problem hiding this comment.
backwards compatbility pub use 👍
Member
Author
|
Thanks @alamb 🎉 |
Member
Author
|
@alamb we moved it to |
Contributor
|
It does appear to be a canonical extension type: https://arrow.apache.org/docs/format/CanonicalExtensions.html#parquet-variant So I agree with idea of moving it It would be good to do it before we release another version too |
mbutrovich
pushed a commit
to mbutrovich/arrow-rs
that referenced
this pull request
Nov 3, 2026
…ache#12033) # Which issue does this PR close? Follow-up to apache#12005; relates to apache#11952. # Rationale for this change Schema-only consumers still need `parquet-variant` to use `VariantType`, and `Field::try_canonical_extension_type` does not recognize `arrow.parquet.variant`. Hosting the descriptor alongside the other canonical extensions lets these consumers use `arrow-schema` directly. # What changes are included in this PR? Move `VariantType` into `arrow-schema`'s canonical extensions and register `CanonicalExtensionType::Variant`. Re-export the same type from `parquet-variant`, preserving the existing compute and Parquet import paths. Metadata handling and storage validation are unchanged; apache#11938 remains separate. # Are these changes tested? The moved unit test retains the existing storage-type checks and now verifies canonical extension lookup. Existing Variant compute doctests and Parquet Variant tests exercise the retained import paths. # Are there any significant user-facing changes? Existing imports remain compatible. Direct `arrow-schema` users need its `canonical_extension_types` feature. `parquet-variant` now enables that feature to retain its unconditional re-export, which also enables `serde_core` and `serde_json` dependencies in `arrow-schema`. Implementation and test changes prepared with Codex.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
VariantTypefromparquet-variant-computetoparquet-variant#11952.Rationale for this change
Using the
arrow.parquet.variantextension type currently requires depending onparquet-variant-compute, which pulls in the fullarrowcrate andparquet-variant-json. Schema-only consumers such asdatafusion-commonanddatafusion-expr(apache/datafusion#26822) only need the type itself, which depends on nothing beyondarrow-schema.What changes are included in this PR?
VariantTypemoves unchanged fromparquet-variant-compute/src/variant_array.rsto a newparquet-variant/src/extension.rsand is exported asparquet_variant::VariantType.parquet-variant-computere-exports it, soparquet_variant_compute::VariantTypeandparquet::variant::VariantTypekeep working and refer to the same type.Are these changes tested?
Yes. Existing uses through the old paths (the
VariantArraydoctests,variant_get, and theparquetVariant schema tests) are unchanged and still pass. A new unit test inparquet-variantcovers the type on its own: a struct field accepts the extension and a non-struct field is rejected with the existing error message.Are there any significant user-facing changes?
VariantTypeis now also available fromparquet-variant. The existing import path is kept, so this is not a breaking change.🤖 Generated with Claude Code