Repository navigation
Description
The main readme landing page https://github.com/datafusion-contrib/datafusion-variant (which also becomes the crates.io page): https://crates.io/crates/datafusion-variant I think could be improved to make it easier for users to see how to use it and how they can help
Challenges with the current page:

- The example usecase (convert json to variant and then back to json) is not a common usecase nor really shows off what you can do with this crate
- The list of functions has no links nor examples of how to use them https://github.com/datafusion-contrib/datafusion-variant#functionality
Good Examples
I think this is a good example of the right level of detail for a landing page: https://github.com/XiangpengHao/pq-vector#quick-start
Suggested Structure:
-
Quickstart that shows how to integrate the features into Datafusion (ideally with only a few lines of rust -- perhaps something like https://docs.rs/datafusion-spark/latest/datafusion_spark/#example-using-all-function-packages and then run some example (perhaps reading the data from a parquet file with variant)
-
Give an overview of the variant type and how it can be used. I think we could follow the model of the duckdb guide https://duckdb.org/docs/2.0/sql/data_types/variant for example
-
Add a cook book for common operations (of course some of these will not work yet)
Scenarios I suggest:
- Copy / Convert JSON data into a variant column (e.g. `COPY .... into 'file_with_variant.parquet')
- Select fields from variant columns
- Convert variant back into JSON
More advanced scenarios
5. Copy / convert JSON data into a shredded variant columns
6. Read some combination of files that has both shredded and non shredded files
Activity
alamb commentedon Oct 26, 2026
FYI @sdf-jkl -- if you think this is a reasonable idea I would be happy to help draft or review such a page
sdf-jkl commentedon Oct 26, 2026
Thanks @alamb. Either way of helping would be very welcome 🙏