Repository navigation
Commit ac279eb
authored
fix(ffi): forward distinct_handling through FFI_AggregateUDF (apache#25876)
## Which issue does this PR close?
- Closes apache#25374.
## Rationale for this change
`AggregateUDFImpl::distinct_handling` (apache#25288) doesn't survive the FFI
boundary. `FFI_AggregateUDF` has no slot for it, so
`ForeignAggregateUDF` falls back to the trait default and every UDAF
loaded from another library reports `Sensitive`. A foreign `min` never
gets its `DISTINCT` dropped, and a foreign `stddev`, which doesn't read
`is_distinct` at all, tells the planner it deduplicates its own input.
## What changes are included in this PR?
Same shape as `order_sensitivity`, all in
`datafusion/ffi/src/udaf/mod.rs`:
- `FFI_DistinctHandling`, a `#[repr(C)]` enum with `From` impls both
ways, next to `FFI_AggregateOrderSensitivity`.
- A `distinct_handling` fn pointer on `FFI_AggregateUDF`, added at the
end of the struct after `supports_null_handling_clause`.
- `ForeignAggregateUDF::distinct_handling` calls through it.
`DistinctHandling` is `#[non_exhaustive]`, so the native-to-FFI
conversion needs a wildcard arm. I map anything unknown to `Sensitive`,
the trait default, which is what the issue suggested. That and appending
the field at the end are the two assumptions worth a second look. The
arm covers a variant added to `datafusion-expr` before the FFI enum
catches up. Mismatched library versions stay unsupported, same as today.
## What is the testing strategy for this PR?
- `udaf::tests::test_distinct_handling` wraps `min`, `sum` and `stddev`
with the mock foreign marker and checks all three variants come back. On
main `min` reports `Sensitive` and the test fails.
- `tests/ffi_udaf.rs::test_distinct_handling` does the same through the
separately built test library, for `stddev` (`Unsupported`) and `sum`
(`Sensitive`). On main's `udaf/mod.rs` it fails with `left: Sensitive,
right: Unsupported`.
- `test_round_trip_all_distinct_handlings` round-trips each variant,
like the existing order sensitivity test.
`cargo test -p datafusion-ffi --features integration-tests` passes, as
does `cargo clippy -p datafusion-ffi --all-targets --features
integration-tests --no-deps -- -D warnings` and `cargo fmt --all --
--check`.
## Are there any user-facing changes?
Foreign UDAFs now report the `distinct_handling` they declare, so an
`Insensitive` one can have `DISTINCT` eliminated. Adding a field changes
the `FFI_AggregateUDF` layout, so this probably wants the `api change`
label.
This was written with AI assistance. I've read the change end to end and
reproduced both failures above myself.1 parent 9eeb66f commit ac279eb
4 files changed
Lines changed: 159 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
23 | | - | |
| 23 | + | |
24 | 24 | | |
25 | 25 | | |
26 | | - | |
| 26 | + | |
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| |||
79 | 79 | | |
80 | 80 | | |
81 | 81 | | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
82 | 147 | | |
83 | 148 | | |
84 | 149 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
36 | 36 | | |
37 | 37 | | |
38 | 38 | | |
39 | | - | |
40 | | - | |
41 | | - | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
| |||
142 | 142 | | |
143 | 143 | | |
144 | 144 | | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
145 | 148 | | |
146 | 149 | | |
147 | 150 | | |
| |||
429 | 432 | | |
430 | 433 | | |
431 | 434 | | |
| 435 | + | |
432 | 436 | | |
433 | 437 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
29 | 29 | | |
30 | 30 | | |
31 | 31 | | |
| 32 | + | |
32 | 33 | | |
33 | 34 | | |
34 | 35 | | |
| |||
210 | 211 | | |
211 | 212 | | |
212 | 213 | | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
213 | 220 | | |
214 | 221 | | |
215 | 222 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
27 | | - | |
| 27 | + | |
| 28 | + | |
28 | 29 | | |
29 | 30 | | |
30 | 31 | | |
| |||
149 | 150 | | |
150 | 151 | | |
151 | 152 | | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
152 | 157 | | |
153 | 158 | | |
154 | 159 | | |
| |||
337 | 342 | | |
338 | 343 | | |
339 | 344 | | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
340 | 351 | | |
341 | 352 | | |
342 | 353 | | |
| |||
412 | 423 | | |
413 | 424 | | |
414 | 425 | | |
| 426 | + | |
415 | 427 | | |
416 | 428 | | |
417 | 429 | | |
| |||
610 | 622 | | |
611 | 623 | | |
612 | 624 | | |
| 625 | + | |
| 626 | + | |
| 627 | + | |
| 628 | + | |
613 | 629 | | |
614 | 630 | | |
615 | 631 | | |
| |||
655 | 671 | | |
656 | 672 | | |
657 | 673 | | |
| 674 | + | |
| 675 | + | |
| 676 | + | |
| 677 | + | |
| 678 | + | |
| 679 | + | |
| 680 | + | |
| 681 | + | |
| 682 | + | |
| 683 | + | |
| 684 | + | |
| 685 | + | |
| 686 | + | |
| 687 | + | |
| 688 | + | |
| 689 | + | |
| 690 | + | |
| 691 | + | |
| 692 | + | |
| 693 | + | |
| 694 | + | |
| 695 | + | |
| 696 | + | |
| 697 | + | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
658 | 705 | | |
659 | 706 | | |
660 | 707 | | |
| |||
798 | 845 | | |
799 | 846 | | |
800 | 847 | | |
| 848 | + | |
| 849 | + | |
| 850 | + | |
| 851 | + | |
| 852 | + | |
| 853 | + | |
| 854 | + | |
| 855 | + | |
| 856 | + | |
| 857 | + | |
| 858 | + | |
| 859 | + | |
| 860 | + | |
| 861 | + | |
| 862 | + | |
| 863 | + | |
| 864 | + | |
| 865 | + | |
801 | 866 | | |
802 | 867 | | |
803 | 868 | | |
| |||
870 | 935 | | |
871 | 936 | | |
872 | 937 | | |
| 938 | + | |
| 939 | + | |
| 940 | + | |
| 941 | + | |
| 942 | + | |
| 943 | + | |
| 944 | + | |
| 945 | + | |
| 946 | + | |
| 947 | + | |
| 948 | + | |
| 949 | + | |
873 | 950 | | |
874 | 951 | | |
875 | 952 | | |
| |||
0 commit comments