Skip to content

Commit a31d4ad

Browse files
adriangbclaude
andcommitted
refactor(proto): restore common-type conversions as From/TryFrom in proto-models
`UnnestOptions`, `TableReference`, `StringifiedPlan`, `JoinType`, `JoinConstraint`, `NullEquality`, `CsvOptions`, `JsonOptions` and the parquet options types all live in `datafusion-common`, which sits below `datafusion-proto-models` in the crate graph and so cannot host the impls. They move onto the local proto type in `datafusion-proto-models` instead — the arrangement `datafusion-proto-common` already uses for `ScalarValue` and `Statistics` — and go back to being plain `From` / `TryFrom`, the shape they had in 54.1.0. This is also the only crate that can express `TryFrom<&[PartitionedFile]> for protobuf::FileGroup`. `datafusion-datasource` cannot: `&T` is `#[fundamental]` but `[T]` is not, so `&[PartitionedFile]` counts as foreign there. Here the *self* type is local, which is all the orphan rule needs, and staying generic over the element (`&T: TryInto<protobuf::PartitionedFile>`) means this crate never has to name `PartitionedFile`, which sits above it. Callers get the 54.1.0 spelling back verbatim. New `datafusion_proto_models::{from_proto, to_proto}` modules; the crate gains a direct `datafusion-common` dependency (already present transitively via `datafusion-proto-common`). Part of apache#24019. Co-Authored-By: Claude Opus 5 <[email protected]>
1 parent 7aa1b94 commit a31d4ad

11 files changed

Lines changed: 913 additions & 710 deletions

File tree

‎Cargo.lock‎

Lines changed: 1 addition & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎datafusion/datasource/src/proto.rs‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -264,6 +264,25 @@ mod tests {
264264
);
265265
}
266266

267+
#[test]
268+
fn file_group_from_slice_matches_file_group() -> Result<()> {
269+
// `protobuf::FileGroup: TryFrom<&[T]>` lives in `datafusion-proto-models`,
270+
// generic over the element so that crate never names `PartitionedFile`.
271+
// This is the caller-visible half: the bound resolves via
272+
// `TryFrom<&PartitionedFile> for protobuf::PartitionedFile` above.
273+
let files = vec![
274+
PartitionedFile::new("a.parquet", 1),
275+
PartitionedFile::new("b.parquet", 2),
276+
];
277+
278+
let from_slice = protobuf::FileGroup::try_from(&files[..])?;
279+
let from_group = protobuf::FileGroup::try_from(&FileGroup::new(files))?;
280+
281+
assert_eq!(from_slice, from_group);
282+
assert_eq!(from_slice.files.len(), 2);
283+
Ok(())
284+
}
285+
267286
#[test]
268287
fn file_group_roundtrip() -> Result<()> {
269288
let group = FileGroup::new(vec![

‎datafusion/proto-models/Cargo.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ default = []
4545
json = ["serde", "pbjson", "datafusion-proto-common/json"]
4646

4747
[dependencies]
48+
datafusion-common = { workspace = true }
4849
datafusion-proto-common = { workspace = true }
4950
pbjson = { workspace = true, optional = true }
5051
prost = { workspace = true }

0 commit comments

Comments
 (0)