Is your feature request related to a problem or challenge?
Currently SchemaAdapter::map_projections returns an error if the provided file_schema contains columns of a different type to the expected table_schema.
In order to accomodate schema evolution, it would be advantageous to be able to instead cast the array data from the file schema to the table schema where possible.
Describe the solution you'd like
impl SchemaAdapter {
fn map_schema(&self, file_schema: &Schema) -> Result<SchemaMapping> {
...
}
}
struct SchemaMapping {
...
}
impl SchemaMapping {
fn map_batch(batch: RecordBatch) -> Result<RecordBatch> {
...
}
}
Where map_schema would use can_cast_types to determine if conversion is possible, and map_batch would use cast to perform this conversion
Describe alternatives you've considered
We could not do this
Additional context
No response