Repository navigation
API to more easily update schema adapter factory on a ListingTable #27314
Description
Is your feature request related to a problem or challenge?
In datafusion-variant datafusion-contrib/datafusion-variant#149 I am trying to replace the schema adapter of an existing ListingTable.
At the moment, I have to recreate an equivalent ListingTableConfig like this:
// Create a ListingTable config from the existing ListingTable `table`
let partition_cols: Vec<&String> = table
.options()
.table_partition_cols
.iter()
.map(|(name, _)| name)
.collect();
let file_fields: Vec<FieldRef> = table
.schema()
.fields()
.iter()
.filter(|f| !partition_cols.contains(&f.name()))
.cloned()
.collect();
let file_schema = Arc::new(Schema::new_with_metadata(
file_fields,
table.schema().metadata().clone(),
));
let config = ListingTableConfig::new_with_multi_paths(table.table_paths().clone())
.with_listing_options(table.options().clone())
.with_schema(file_schema)
.with_expr_adapter_factory(Arc::new(VariantExprAdapterFactory::default()));
let table = ListingTable::try_new(config)?
.with_cache(state.runtime_env().cache_manager.get_file_statistic_cache())
.with_definition(cmd.definition.clone())
.with_constraints(cmd.constraints.clone())
.with_column_defaults(cmd.column_defaults.clone());
Ok(Arc::new(table))This is both verbose and brittle if new fields are added to ListingTable config they won't automatically be added here
Describe the solution you'd like
I would like some easier way to go from an existing ListingTable to another ListingTable with a different expr adapter
Describe alternatives you've considered
For example, make we could add a ListingTable::into_config() to convert into a ListingTableConfig:
// Create a ListingTable config from the existing ListingTable `table`
let config = table.into_config()
.with_expr_adapter_factory(Arc::new(VariantExprAdapterFactory::default()));
let table = ListingTable::try_new(config)?
.with_cache(state.runtime_env().cache_manager.get_file_statistic_cache())
.with_definition(cmd.definition.clone())
.with_constraints(cmd.constraints.clone())
.with_column_defaults(cmd.column_defaults.clone());
Ok(Arc::new(table))Or perhaps we could just add a method to get/set the ExprAdapterFactory the best:
// Create a ListingTable config from the existing ListingTable `table`
let existing_adapter = table.expr_adapter_factor();
let new_adapter = Arc::new(VariantExprAdapterFactory::new(existing_adapter));
let table = table
.clone()
.with_expr_adapter_factory(new_adapter);
Ok(Arc::new(table))Additional context
No response
Willingness to contribute
None
Activity
Sruhvx-jpg commentedon Nov 11, 2026
Hey @alamb, I have submitted a PR addressing this issue. I hope you review it prompt as time allows you too.
Thanks
feat: Add expr_adapter_factory getter and with_expr_adapter_factory s…
feat: Add file_schema, cache, and expr_adapter_factory accessors to L…
[branch-56] feat: Add expr_adapter_factory getter and with_expr_adapt…