Repository navigation
Commit 4d95615
refactor: Migrate ScalarSubqueryExpr to self-serialization proto pattern (apache#23130)
Move ScalarSubqueryExpr proto encode/decode into the expression itself
via the try_to_proto / try_from_proto hooks, removing the centralized
downcast branch in to_proto.rs and the inline construction in
from_proto.rs. Because the ScalarSubqueryResults container is
runtime-only shared state and not part of the wire format, from_proto.rs
still fetches it from the decode context and threads it into
try_from_proto.
Follows the pattern established in apache#21929.
## Which issue does this PR close?
- Closes #[22433](apache#22433).
## Rationale for this change
datafusion-proto serializes ScalarSubqueryExpr by downcasting in
to_proto.rs and rebuilding it inline in from_proto.rs, which forces its
internal fields to be exposed as public accessors just for the proto
crate.
PR #[21929](apache#21929) introduced
a self-serialization pattern (try_to_proto / try_from_proto) where each
expression owns its own wire format and the central dispatch just
delegates. Other expressions like InListExpr and
DynamicFilterPhysicalExpr already use it. This change moves
ScalarSubqueryExpr to the same pattern, removing its bespoke proto
branches and the proto-only public accessors.
One wrinkle: ScalarSubqueryExpr's ScalarSubqueryResults is runtime-only
shared state, not part of the wire format, so try_from_proto takes it as
an extra argument that from_proto.rs threads in from the decode context.
## What changes are included in this PR?
- [x] `try_to_proto` in `impl PhysicalExpr for ScalarSubqueryExpr` (not
inherent), `#[cfg(feature = "proto")]`
- [x] `ScalarSubqueryExpr::try_from_proto` wired into `from_proto.rs`
- [x] central `to_proto.rs` + `from_proto.rs` arms deleted in this PR
- [x] direct hook test incl. bad-input case; test module at end of file
- [x] roundtrip tests pass; fmt + clippy `-D warnings` clean; PR
template filled in
## Are these changes tested?
- **Unit tests** (`scalar_subquery.rs`, `#[cfg(all(test, feature =
"proto"))]` module at end of file) exercise the hooks directly:
- `round_trips_through_proto` — encode via `try_to_proto`, decode via
`try_from_proto`, asserting `data_type`/`nullable`/`index` and
shared-results identity survive.
- `rejects_non_scalar_subquery_node` — wrong `ExprType` variant errors
cleanly.
- `rejects_missing_data_type` — missing required field errors cleanly.
- **Round-trip integration tests** (`datafusion-proto`,
`--all-features`) cover the full plan path:
`roundtrip_scalar_subquery_exec`,
`roundtrip_nested_scalar_subquery_exec_scopes_results`, and
`roundtrip_scalar_subquery_exec_with_default_converter_executes`.
All pass, and `cargo fmt --all` + `cargo clippy --all-features
--all-targets -- -D warnings` are clean.
## Are there any user-facing changes?
No
---------
Co-authored-by: Matthew Patton <[email protected]>1 parent 89dd04e commit 4d95615
3 files changed
Lines changed: 201 additions & 31 deletions
File tree
- datafusion
- physical-expr/src
- proto/src/physical_plan
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
62 | 70 | | |
63 | 71 | | |
64 | 72 | | |
65 | 73 | | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
66 | 78 | | |
67 | 79 | | |
68 | 80 | | |
69 | 81 | | |
70 | 82 | | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
71 | 87 | | |
72 | 88 | | |
73 | 89 | | |
74 | | - | |
75 | | - | |
76 | | - | |
77 | | - | |
78 | 90 | | |
79 | 91 | | |
80 | 92 | | |
| |||
139 | 151 | | |
140 | 152 | | |
141 | 153 | | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
142 | 217 | | |
143 | 218 | | |
144 | 219 | | |
| |||
238 | 313 | | |
239 | 314 | | |
240 | 315 | | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
| 383 | + | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
| 427 | + | |
| 428 | + | |
| 429 | + | |
| 430 | + | |
| 431 | + | |
| 432 | + | |
| 433 | + | |
| 434 | + | |
| 435 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
45 | | - | |
46 | 45 | | |
47 | 46 | | |
48 | 47 | | |
| |||
366 | 365 | | |
367 | 366 | | |
368 | 367 | | |
369 | | - | |
370 | | - | |
371 | | - | |
372 | | - | |
373 | | - | |
374 | | - | |
375 | | - | |
376 | | - | |
| 368 | + | |
377 | 369 | | |
378 | 370 | | |
379 | 371 | | |
380 | 372 | | |
381 | 373 | | |
382 | 374 | | |
383 | | - | |
384 | | - | |
385 | | - | |
386 | | - | |
387 | | - | |
388 | | - | |
| 375 | + | |
389 | 376 | | |
390 | 377 | | |
391 | 378 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
34 | | - | |
35 | 34 | | |
36 | 35 | | |
37 | 36 | | |
| |||
331 | 330 | | |
332 | 331 | | |
333 | 332 | | |
334 | | - | |
335 | | - | |
336 | | - | |
337 | | - | |
338 | | - | |
339 | | - | |
340 | | - | |
341 | | - | |
342 | | - | |
343 | | - | |
344 | | - | |
345 | 333 | | |
346 | 334 | | |
347 | 335 | | |
| |||
0 commit comments