Skip to content

[Variant] Validate storage fields in VariantType #11938

Open

Description

@sdf-jkl

Describe the bug

VariantType, which #12033 moved into arrow-schema, accepts any DataType::Struct as storage. supports_data_type only checks matches!(data_type, DataType::Struct(_)), so try_new, validate and Field::try_canonical_extension_type all accept structs with no metadata or value field.

This came up in apache/datafusion#26822 (for apache/datafusion#21144). Like the other canonical wrappers there, DFVariant should delegate storage validation to the Arrow extension type. Today a real check means building a VariantArray, which pulls in parquet-variant-compute.

To Reproduce

On main at 9c94861, arrow-schema with the canonical_extension_types feature:

use arrow_schema::extension::{ExtensionType, VariantType};
use arrow_schema::{DataType, Field};

let invalid = DataType::Struct(vec![Field::new("unrelated", DataType::Int32, false)].into());

// All succeed, although this is not a Variant storage type.
assert!(VariantType::try_new(&invalid, "").is_ok());
assert!(VariantType::validate(&invalid, "").is_ok());
let field = Field::new("v", invalid, true).with_extension_type(VariantType);
assert!(field.try_canonical_extension_type().is_ok());

Expected behavior

Check the top-level storage fields with the rules VariantArray::try_new already applies (binary checks, value/typed_value presence):

  • metadata is present and is Binary, LargeBinary, BinaryView, or a dictionary or run-end-encoded array of one of those
  • value, if present, is Binary, LargeBinary or BinaryView
  • at least one of value and typed_value is present

The Parquet reader attaches VariantType to every VARIANT column (extension.rs), so the check must not reject anything VariantArray::try_new accepts. arrow-go's NewVariantType checks at the same depth.

Out of scope: nullability, typed_value types and nested shredded object groups. Those stay with VariantArray::try_new.

Additional context

@mbrobbel raised this in the #8392 review, and @alamb agreed, pointing to #8387. #8387 closed after adding Binary/LargeBinary support, without changing this check.

Willingness to contribute

I plan to contribute a fix for this bug independently

Activity

  1. added a commit that references this issue on Oct 28, 2026
  2. self-assigned this
    on Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions