You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Variant] Integration tests for reading parquet w/ Variants #8084
We are implementing Variant, and part of doing so is ensuring that the Rust implementation is compatible with other implementations
Describe the solution you'd like
I would like a harness that
For each test listed in cases.json
Reads the expected variant.bin file and creates a Variant from it
Reads the data from the Parquet file (should be a StructArray)
Construct a VariantArray
Read the value from the VariantArray (using both VariantArray::value and the variant_get kernel)
Verify that the value from the VariantArray matches the Variant read from the .bin file
Many of the cases will not to pass initially -- that is fine, we should just mark them as #[ignore] or something and we will enable them as we work through the Variant implementation
Activity
[VARIANT] Initial integration tests for variant reads (#8104)
alamb commentedon Aug 13, 2025
Update here @carpecodeum has added the initial basic tests in
There are a few more follow on items:
VariantArray(this requires some finagling with feature flags / dependencies that I am working on)#teststyle tests for all the cases (@carpecodeum / @mprammer is this something you able to help with?)carpecodeum commentedon Aug 14, 2025
Yep we can help with that, sorry missed this comment
alamb commentedon Aug 14, 2025
Thanks @carpecodeum -- I filed this ticket to track adding a feature flag / making it easier to use Variant with
variantfeature toparquetcrate #8132I think it will be tricky (b/c I don't really know what to suggest / don't have a clear goal yet)
Perhaps you can work on 2 and 3 while I try and sort out #8132 ?
alamb commentedon Aug 14, 2025
Here is a PR to make the variant code available in the parquet crate
variant_experimentalfeature toparquetcrate #8133Add `variant_experimental` feature to `parquet` crate (#8133)
alamb commentedon Sep 9, 2025
Now that we have merged #8133 and the variant files have been merged into parquet-testing
We can update the tests to run with the new features
alamb commentedon Sep 11, 2025
I am working on updating the test harness with the finalized testing files
Update `variant_integration` test to use final approved `parquet-test…