Repository navigation
Commit 7401e1b
authored
fix(ffi): prevent recursive session physical planning (apache#24492)
## Which issue does this PR close?
- Closes apache#24065.
## Rationale for this change
`ForeignSession::create_physical_plan` forwarded through the owning
session. If that session had installed the calling library's query
planner, the callback re-entered the same planner and could recurse
until the stack was exhausted. The callback also returned an
`FFI_ExecutionPlan`, which cannot reconstruct built-in nodes with the
receiving library's Rust type identities for downcasting.
The supported delegation boundary is `FFI_QueryPlanner`: the session
owner exports its original planner before installing a foreign planner,
and the foreign planner retains that handle. This boundary serializes
physical plans and reconstructs them with local type identities.
## What changes are included in this PR?
- Retain the legacy `create_physical_plan` callback at its original
position and signature in `FFI_SessionRef` for DataFusion 55 ABI
compatibility, while making it return an actionable `NotImplemented`
error without invoking the installed planner.
- Make `ForeignSession::create_physical_plan` return the same error
without crossing the FFI boundary or invoking the installed planner.
- Document the captured-`FFI_QueryPlanner` migration in the module
documentation and DataFusion 56 upgrade guide.
- Add unit and cross-library regression coverage for the rejected
direct-delegation path and the successful captured-planner path.
apache#24690 added the session's logical extension codec to the callback and
fixed planning for logical plans that require custom codec support. It
does not address the two remaining problems covered here: forwarding can
re-enter the installed foreign planner, and the returned
`FFI_ExecutionPlan` does not reconstruct built-in nodes with the
receiving library's Rust type identities. This PR preserves the callback
slot for ABI compatibility but makes direct session planning
unsupported, so its callback-specific codec wiring is no longer
exercised; other FFI codec work remains outside this PR.
## Are these changes tested?
Yes.
- The focused unit regression verifies that both
`ForeignSession::create_physical_plan` and the retained legacy callback
return `NotImplemented` with zero planner re-entry.
- The three-library `dlopen` regression verifies that direct
foreign-session delegation is rejected while the captured
`FFI_QueryPlanner` route succeeds and restores local, downcastable
physical-plan nodes.
- `cargo test -p datafusion-ffi --features integration-tests`
- `cargo clippy -p datafusion-ffi --all-targets --all-features -- -D
warnings`
- `cargo clippy --all-targets --all-features -- -D warnings`
- `RUST_BACKTRACE=1 cargo test --profile ci --exclude
datafusion-examples --exclude datafusion-benchmarks --exclude
datafusion-cli --workspace --lib --tests --bins --features
avro,json,backtrace,extended_tests,recursive_protection,parquet_encryption`
- `RUSTDOCFLAGS="-D warnings" cargo doc -p datafusion-ffi --all-features
--no-deps`
- `cargo fmt --all -- --check`
- `./ci/scripts/doc_prettier_check.sh`
## Are there any user-facing changes?
Yes. `ForeignSession::create_physical_plan` and the retained legacy
callback now return `NotImplemented`; callers must retain and invoke the
session owner's exported `FFI_QueryPlanner`. The `FFI_SessionRef`
callback remains at its original ABI position and signature, so this PR
does not change the struct layout or require consumers to rebuild solely
because of an ABI layout change.1 parent 92746a9 commit 7401e1b
4 files changed
Lines changed: 114 additions & 88 deletions
File tree
- datafusion/ffi/src
- session
- tests
- docs/source/library-user-guide/upgrading
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
47 | | - | |
48 | | - | |
49 | | - | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
50 | 51 | | |
51 | 52 | | |
52 | 53 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
24 | | - | |
25 | | - | |
26 | | - | |
27 | | - | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
28 | 28 | | |
29 | 29 | | |
30 | 30 | | |
| |||
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
43 | | - | |
| 43 | + | |
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
| |||
117 | 117 | | |
118 | 118 | | |
119 | 119 | | |
| 120 | + | |
| 121 | + | |
120 | 122 | | |
121 | 123 | | |
122 | 124 | | |
| |||
239 | 241 | | |
240 | 242 | | |
241 | 243 | | |
242 | | - | |
243 | | - | |
| 244 | + | |
| 245 | + | |
244 | 246 | | |
245 | | - | |
246 | | - | |
247 | | - | |
248 | | - | |
249 | | - | |
250 | | - | |
251 | | - | |
252 | | - | |
253 | | - | |
254 | | - | |
255 | | - | |
256 | | - | |
257 | | - | |
258 | | - | |
259 | | - | |
260 | | - | |
261 | | - | |
262 | | - | |
263 | | - | |
264 | | - | |
265 | | - | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
266 | 251 | | |
| 252 | + | |
267 | 253 | | |
268 | 254 | | |
269 | 255 | | |
| |||
529 | 515 | | |
530 | 516 | | |
531 | 517 | | |
532 | | - | |
533 | 518 | | |
534 | | - | |
535 | | - | |
536 | | - | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
537 | 523 | | |
538 | 524 | | |
539 | 525 | | |
| |||
748 | 734 | | |
749 | 735 | | |
750 | 736 | | |
751 | | - | |
| 737 | + | |
752 | 738 | | |
753 | | - | |
754 | | - | |
755 | | - | |
756 | | - | |
757 | | - | |
758 | | - | |
759 | | - | |
760 | | - | |
761 | | - | |
762 | | - | |
763 | | - | |
764 | | - | |
765 | | - | |
766 | | - | |
767 | | - | |
768 | | - | |
| 739 | + | |
| 740 | + | |
| 741 | + | |
769 | 742 | | |
770 | 743 | | |
771 | 744 | | |
| |||
859 | 832 | | |
860 | 833 | | |
861 | 834 | | |
862 | | - | |
863 | 835 | | |
864 | | - | |
| 836 | + | |
865 | 837 | | |
866 | | - | |
| 838 | + | |
867 | 839 | | |
868 | 840 | | |
869 | 841 | | |
870 | 842 | | |
871 | 843 | | |
872 | | - | |
873 | 844 | | |
874 | 845 | | |
875 | 846 | | |
| 847 | + | |
| 848 | + | |
| 849 | + | |
| 850 | + | |
| 851 | + | |
| 852 | + | |
| 853 | + | |
| 854 | + | |
| 855 | + | |
| 856 | + | |
| 857 | + | |
| 858 | + | |
| 859 | + | |
| 860 | + | |
| 861 | + | |
| 862 | + | |
| 863 | + | |
| 864 | + | |
| 865 | + | |
876 | 866 | | |
877 | 867 | | |
878 | 868 | | |
| |||
983 | 973 | | |
984 | 974 | | |
985 | 975 | | |
986 | | - | |
987 | | - | |
988 | | - | |
989 | | - | |
990 | | - | |
991 | | - | |
992 | 976 | | |
993 | 977 | | |
994 | 978 | | |
| |||
1015 | 999 | | |
1016 | 1000 | | |
1017 | 1001 | | |
1018 | | - | |
1019 | | - | |
1020 | | - | |
1021 | 1002 | | |
1022 | | - | |
1023 | | - | |
1024 | | - | |
1025 | | - | |
1026 | | - | |
1027 | | - | |
1028 | | - | |
1029 | | - | |
| 1003 | + | |
| 1004 | + | |
1030 | 1005 | | |
| 1006 | + | |
| 1007 | + | |
| 1008 | + | |
| 1009 | + | |
1031 | 1010 | | |
1032 | | - | |
| 1011 | + | |
1033 | 1012 | | |
1034 | 1013 | | |
1035 | 1014 | | |
1036 | | - | |
1037 | | - | |
1038 | 1015 | | |
1039 | | - | |
1040 | | - | |
1041 | | - | |
1042 | | - | |
1043 | | - | |
1044 | | - | |
1045 | | - | |
| 1016 | + | |
| 1017 | + | |
| 1018 | + | |
| 1019 | + | |
| 1020 | + | |
| 1021 | + | |
| 1022 | + | |
| 1023 | + | |
| 1024 | + | |
| 1025 | + | |
| 1026 | + | |
| 1027 | + | |
| 1028 | + | |
| 1029 | + | |
1046 | 1030 | | |
1047 | | - | |
| 1031 | + | |
| 1032 | + | |
| 1033 | + | |
| 1034 | + | |
| 1035 | + | |
| 1036 | + | |
| 1037 | + | |
| 1038 | + | |
| 1039 | + | |
| 1040 | + | |
| 1041 | + | |
| 1042 | + | |
| 1043 | + | |
| 1044 | + | |
1048 | 1045 | | |
1049 | 1046 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
24 | | - | |
| 24 | + | |
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
| |||
115 | 115 | | |
116 | 116 | | |
117 | 117 | | |
118 | | - | |
119 | | - | |
120 | | - | |
| 118 | + | |
121 | 119 | | |
122 | 120 | | |
123 | 121 | | |
| |||
126 | 124 | | |
127 | 125 | | |
128 | 126 | | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
129 | 140 | | |
130 | 141 | | |
131 | 142 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
28 | 45 | | |
29 | 46 | | |
30 | 47 | | |
| |||
0 commit comments