Repository navigation
Description
Is your feature request related to a problem or challenge?
A plan rewrite that inspects a Parquet scan cannot tell which reader factory the scan uses. ParquetSource::parquet_file_reader_factory returns &Arc<dyn ParquetFileReaderFactory>, and the trait is Debug + Send + Sync + 'static with no way to reach the concrete type.
Our case: Algeon replaces DataFusion Parquet scans with its own GPU reader when that reader can reproduce what the scan reads. A scan with DefaultParquetFileReaderFactory or CachedParquetFileReaderFactory qualifies. A caller-provided factory does not, because it may change bytes, metadata, caching or credentials. Today we tell them apart by checking that format!("{factory:?}") starts with DefaultParquetFileReaderFactory { store: or CachedParquetFileReaderFactory { store: . That breaks if the derived Debug output changes, and a caller type with the same name and first field passes.
Describe the solution you'd like
Give ParquetFileReaderFactory an Any supertrait and the same inherent methods dyn FileSource has:
pub trait ParquetFileReaderFactory: Any + Debug + Send + Sync { ... }
impl dyn ParquetFileReaderFactory {
pub fn is<T: ParquetFileReaderFactory>(&self) -> bool {
(self as &dyn Any).is::<T>()
}
pub fn downcast_ref<T: ParquetFileReaderFactory>(&self) -> Option<&T> {
(self as &dyn Any).downcast_ref()
}
}The trait already requires 'static, so every existing implementation is Any and keeps compiling. Any implies 'static, so the explicit bound can go, as in the other traits with an Any supertrait. This is the pattern #21572 describes and #21576 applied to FileSource, DataSource, FileFormat and DataSink. Those traits had an as_any to replace; this one never had any downcast path, so it was not part of that series.
ParquetFileSchemaProvider in the same file has the same shape, but ParquetSource has no public accessor for it, so nothing outside the crate could downcast it. I would leave it alone.
Describe alternatives you've considered
- Keep matching on the
Debugoutput, as described above. - Add a method such as
fn is_default(&self) -> boolto the trait. That covers our check but not other uses, and theAnypattern is already the convention in these crates.
Additional context
None.
Willingness to contribute
None
Activity
feat: allow downcasting `ParquetFileReaderFactory` (apache#26635)