You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 18ef6cf
Browse filesBrowse the repository at this point in the historyBrowse files
prototype: DataSource/FileSource proto hook + CsvSource migration (apache#22419 stack C)
Adds a proto feature + try_to_proto hook to the DataSource and FileSource
traits (datafusion-datasource), so DataSourceExec rides the ExecutionPlan hook
and delegates polymorphically: DataSourceExec -> DataSource -> FileSource. No
downcast in the encode path for migrated sources.
Ports the shared FileScanConfig <-> proto conversion into datafusion-datasource
(FileScanConfig::{to_proto_conf, from_proto_conf, parse_table_schema_from_proto});
orderings/partition exprs go through the ctx, schema/stats/scalars via
proto-common. Needs NO PhysicalExtensionCodec.
CsvSource migrated as proof: its central encode branch is deleted and the
CsvScan decode arm repointed to CsvSource::try_from_proto. New roundtrip_csv_scan
test proves the hook chain end-to-end. All 191 proto_integration tests pass.
Foundation confirms all 4 remaining formats (json/arrow/avro/parquet) are clean
(parquet encryption is a non-issue: crypto isn't in the proto today).
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01SfBPC2yYCwHUn4V8pQCVj3
0 commit comments