Skip to content

Commit 198269d

Browse files
adriangbclaude
authored andcommitted
chore(proto): deprecate AsyncFuncExec::async_exprs, which only existed for proto serialization (apache#24168)
## Which issue does this PR close? - Follow-up cleanup for the proto hook migration EPIC apache#23494. Does not close it. ## Rationale for this change Five public accessor methods on physical plan nodes exist for one reason only: an older protobuf serialization PR needed to reach a private struct field from outside the plan's own module. Each was introduced by the serialization PR that needed it, and none was ever part of an API anyone asked for. Now that every one of these plans serializes itself through its own `try_to_proto` hook — which lives in the same module and can read the fields directly — the accessors have no callers inside DataFusion. But "no caller inside DataFusion" is not the same as "no caller". Of the five, **only one is deprecated** — the other four all turned out to have a real, non-serialization consumer downstream: | Method | Introduced by | Callers in DataFusion | Known downstream caller | This PR | | --- | --- | --- | --- | --- | | `AsyncFuncExec::async_exprs` | "[Proto]: Serialization support for `AsyncFuncExec`" (apache#19118) | none | none found | **deprecated** | | `AnalyzeExec::verbose` | "Implement protobuf serialization for AnalyzeExec" (apache#7574) | none | datafusion-distributed, openobserve | kept as-is | | `AnalyzeExec::show_statistics` | "Implement protobuf serialization for AnalyzeExec" (apache#7574) | none | openobserve | kept as-is | | `UnnestExec::list_column_indices` | "Support encoding and decoding UnnestExec" (apache#12344) | none | goldsky streamling | kept as-is | | `UnnestExec::struct_column_indices` | "Support encoding and decoding UnnestExec" (apache#12344) | none | goldsky streamling | kept as-is | An earlier revision of this PR deprecated four of the five. @kumarUjjawal's [review](apache#24168 (review)) pointed at two downstream projects I had not checked, which between them use three of those four. Those three deprecations have been reverted; see [below](#downstream-usage-check). Every one of the four kept accessors is the same shape: downstream code downcasts a planned node and needs to read its private fields in order to rebuild it as its own node. That is a legitimate use, and the fact that an accessor was *originally added* for proto doesn't make its current use wrong. Deprecating them would push a warning onto downstream projects for an API they have a real need for, with nothing to point them at instead. ## What changes are included in this PR? Adds `#[deprecated(since = "55.0.0", note = "...")]` to `AsyncFuncExec::async_exprs`. Nothing is removed, no behavior changes, and the other four accessors are untouched. The `note` is honest that there is no replacement: `AsyncFuncExec` serializes itself through `AsyncFuncExec::try_to_proto`, which reads the field directly, so there is nothing to point users at. It follows the existing phrasing used by the deprecated shims in `datafusion/proto/src/physical_plan/mod.rs` ("unused by DataFusion; ...") combined with the repo's established no-replacement idiom ("please open an issue if you have a use case for it"). `AsyncFuncExec::async_exprs` already had zero callers before apache#24166; its `try_to_proto` hook was written against the field from the start. ## Are these changes tested? There is no new behavior to test — the real verification is that the compiler agrees the method is unused. Since `deprecated` is a warning and CI builds with `-D warnings`, a clean lint over the whole workspace *is* the proof that no internal caller remains. Run locally on this branch: - `cargo fmt --all` - `cargo clippy --all-targets --workspace --features avro,integration-tests,extended_tests -- -D warnings` (CI's exact invocation) — clean across every crate, including `datafusion-cli`, `benchmarks`, `datafusion-examples` and `substrait` - `cargo test -p datafusion-proto --test proto_integration` — 219 passed, 0 failed - `cargo test -p datafusion-physical-plan` — passed, 0 failed ## Downstream usage check Before deprecating, I checked the three main downstream consumers at their current `main` (2026-08-09), by cloning each repo and grepping for all five accessor names plus every mention of `AnalyzeExec` / `UnnestExec` / `AsyncFuncExec`. GitHub code search returned 503s and silent empty results at the time, which is exactly why the survey missed two projects — @kumarUjjawal caught both in review. | Repo | Uses any of the five? | Which | | --- | --- | --- | | [datafusion-distributed](https://github.com/datafusion-contrib/datafusion-distributed) (`45bd823`) | Yes | `AnalyzeExec::verbose` | | [openobserve](https://github.com/openobserve/openobserve) (`575e8ea`) | Yes | `AnalyzeExec::verbose`, `AnalyzeExec::show_statistics` | | [goldsky streamling](https://github.com/goldsky-io/streamling) (`8d85af9`) | Yes | `UnnestExec::list_column_indices`, `UnnestExec::struct_column_indices` | | [datafusion-comet](https://github.com/apache/datafusion-comet) (`c706360`) | No | — | | [datafusion-ballista](https://github.com/apache/datafusion-ballista) (`06f8f1d`) | No | — | **datafusion-distributed — `verbose`.** `src/explain_analyze.rs:34` builds a `DistributedAnalyzeExec` from `analyze_exec.verbose()`, driven by a planner that downcasts a planned `AnalyzeExec` (`src/distributed_planner/distributed_query_planner.rs:96`). It does not read `show_statistics` — `DistributedAnalyzeExec` doesn't carry that flag. **openobserve — `verbose` *and* `show_statistics`.** [`src/search/src/datafusion/optimizer/physical_optimizer/distribute_analyze.rs:31-37`](https://github.com/openobserve/openobserve/blob/575e8ea4d5fd4b0e630aa013a1a31b1da56708e9/src/search/src/datafusion/optimizer/physical_optimizer/distribute_analyze.rs#L31-L37) does the same rewrite as datafusion-distributed, but reads both flags: ```rust if let Some(analyze) = plan.downcast_ref::<AnalyzeExec>() { let distribute_analyze = Arc::new(DistributeAnalyzeExec::new( analyze.verbose(), analyze.show_statistics(), analyze.input().clone(), )) as Arc<dyn ExecutionPlan>; ``` **goldsky streamling — both unnest accessors.** [`crates/streamling-core/src/operators/unnest.rs:85-99`](https://github.com/goldsky-io/streamling/blob/8d85af926b034085aece04233722c508ac80f2d0/crates/streamling-core/src/operators/unnest.rs#L85-L99), in `StreamingUnnestExec::from_original`, rebuilds a DataFusion `UnnestExec` as its own streaming operator and reads both index lists to do it: ```rust let list_column_indices = original_unnest .list_column_indices() .iter() .map(|idx| ListUnnest { index_in_input_schema: idx.index_in_input_schema, depth: idx.depth }) .collect(); let struct_column_indices = original_unnest.struct_column_indices().to_vec(); ``` **datafusion-comet — no usage.** It constructs `UnnestExec::new(...)` in `native/core/src/execution/planner.rs:2081` but never reads the index lists back out. Zero hits for any of the five names. **datafusion-ballista — no usage.** Two near-misses, both false positives: `ballista/core/src/planner.rs:148` reads `analyze.verbose`, but that is the public field on the **logical** `LogicalPlan::Analyze` node, not the physical accessor; `ballista/scheduler/src/state/distributed_explain.rs:155` calls `UnnestExec::new(...)`, construction only. **`AsyncFuncExec::async_exprs`.** Code search for `async_exprs` and for the literal `async_exprs()` across public Rust code returns hits only in DataFusion itself and in forks/vendored copies of it (`ClickHouse/rust_vendor`, `apache/datafusion-sandbox`, `Epsio-Labs/hiring-datafusion`, `smartdu/datafusion`). The three non-fork repos that mention `AsyncFuncExec` — `apache/sedona-db`, `influxdata/datafusion-udf-wasm`, `goldmedal/datafusion-llm-function` — have no calls to the accessor. This still covers only what public code search and these five projects show. If you know of a consumer of `AsyncFuncExec::async_exprs`, say so and I'll drop the last deprecation too, on the same reasoning applied to the other four. ## Are there any user-facing changes? Yes, and the `api change` label applies. Downstream users who call `AsyncFuncExec::async_exprs` will now see a deprecation warning. Nothing breaks in this release — the method still works exactly as before. Removal follows the normal deprecation window described in the [API health policy](https://github.com/apache/datafusion/blob/main/docs/source/contributor-guide/api-health.md) (six major versions or six months, whichever is longer), consistent with the plan in EPIC apache#23494. `AnalyzeExec::verbose`, `AnalyzeExec::show_statistics`, `UnnestExec::list_column_indices` and `UnnestExec::struct_column_indices` are unchanged, so datafusion-distributed, openobserve and goldsky streamling see no new warning. There is intentionally no replacement API for `async_exprs`. If you have a use case for reading that field from outside the plan, please open an issue — that is a real API request worth designing deliberately, rather than something to leave standing by accident. --------- Co-authored-by: Claude Opus 5 <[email protected]>
1 parent dc69242 commit 198269d

1 file changed

Lines changed: 4 additions & 0 deletions

File tree

‎datafusion/physical-plan/src/async_func.rs‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,6 +106,10 @@ impl AsyncFuncExec {
106106
))
107107
}
108108

109+
#[deprecated(
110+
since = "55.0.0",
111+
note = "unused by DataFusion; `AsyncFuncExec` serializes itself via `AsyncFuncExec::try_to_proto`, which reads the field directly. There is no replacement; please open an issue if you have a use case for it."
112+
)]
109113
pub fn async_exprs(&self) -> &[Arc<AsyncFuncExpr>] {
110114
&self.async_exprs
111115
}

0 commit comments

Comments
 (0)