You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 8e5c789
Browse filesBrowse the repository at this point in the historyBrowse files
FFI: plumb with_updated_config for FFI_ScalarUDF (apache#22797)
## Which issue does this PR close?
Part of apache#22330.
## Rationale for this change
`ForeignScalarUDF` inherits the default `with_updated_config`, so
producer overrides are lost across the FFI boundary.
## What changes are included in this PR?
- Forward `with_updated_config` through `FFI_ScalarUDF`.
- Reuse the existing placement and timezone UDFs for unit and
dynamic-library coverage.
- Log transport failures before the infallible trait method returns
`None`.
- Update the FFI skill to cover error handling, owned returns, and
fixture reuse.
## Are these changes tested?
- `cargo test -p datafusion-ffi --features integration-tests`
- `cargo clippy --all-targets --all-features -- -D warnings`
- `./ci/scripts/doc_prettier_check.sh --write --allow-dirty`
## Are there any user-facing changes?
The FFI ABI changes. Foreign libraries must rebuild against the new
DataFusion version.
---------
Signed-off-by: Amogh Ramesh <[email protected]>
Co-authored-by: Tim Saucer <[email protected]>
Copy file name to clipboardExpand all lines: .ai/skills/datafusion-ffi/SKILL.md
+7-3Lines changed: 7 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
---
2
2
name: datafusion-ffi
3
-
description: Patterns and review checklist for the `datafusion-ffi` crate. Use whenever the user adds, edits, or reviews code under `datafusion/ffi/` — new `FFI_X` wrappers, `Foreign<X>` impls, codec changes, or expanding an existing wrapper to cover more of a trait's surface. Also use when reviewing PRs that touch this crate.
3
+
description: Patterns and review checklist for the `datafusion-ffi` crate. Use whenever the user adds, edits, or reviews code under `datafusion/ffi/` — new `FFI_X` wrappers, `ForeignX` implementations, codec changes, or expanding an existing wrapper to cover more of a trait's surface. Also use when reviewing PRs that touch this crate.
4
4
---
5
5
6
6
# DataFusion FFI Skill
@@ -210,9 +210,11 @@ cargo test -p datafusion-ffi --features integration-tests
210
210
To add coverage for a new wrapper:
211
211
212
212
1.**Add a constructor** in `src/tests/<area>.rs` (or a new file there). Return a populated `FFI_X` from a known-good native type.
213
-
2.**Wire it into `ForeignLibraryModule`** in `src/tests/mod.rs`: add a field of type `extern "C" fn(...) -> FFI_X` and populate it in `datafusion_ffi_get_module`. This struct is the cross-library contract — adding a field is itself an ABI change for the test module; integration tests will rebuild the cdylib automatically.
213
+
2.**Wire it into `ForeignLibraryModule`** in `src/tests/mod.rs`: add a field of type `extern "C" fn(...) -> FFI_X` and populate it in `datafusion_ffi_get_module`. This struct is the cross-library contract. Adding a field is itself an ABI change for the test module; integration tests will rebuild the cdylib automatically.
214
214
3.**Add the test** in `tests/ffi_<area>.rs` under `#[cfg(feature = "integration-tests")] mod tests { … }`. Call `datafusion_ffi::tests::utils::get_module()` to load the cdylib, invoke your constructor through the returned `ForeignLibraryModule`, convert into `Arc<dyn X>`, and exercise every method.
215
215
216
+
When adding a method to an existing wrapper, reuse an existing fixture and constructor when a small trait-method override can cover it. This applies both to omitted default methods and methods newly added to the trait. Add a dedicated test type or `ForeignLibraryModule` field only when the existing fixtures cannot cover the method.
217
+
216
218
What integration tests catch that unit tests cannot:
217
219
218
220
-**Real ABI layout bugs.** Two builds means the consumer's view of `FFI_X` is reconstructed from declaration, not aliased to the producer's memory. Mismatched alignment, padding, niche optimization, or accidentally non-`#[repr(C)]` types surface here.
@@ -235,7 +237,7 @@ What integration tests catch that unit tests cannot:
- A `stabby::string::String` (`SString`) returned by value, with no other args or returns.
237
239
238
-
Concrete skippable example: `fn name(&self) -> SString` reading a field already validated by another method. Concrete *non*-skippable examples: anything returning `SVec<T>`, `FFI_Option<T>`, `FFI_Result<T>`, `WrappedSchema`, `WrappedArray`, an `FfiFuture`, an `FFI_*` sub-struct, or any `*mut`/`*const` pointer — those exercise alignment / padding / niche-opt across the ABI boundary and need the two-build coverage. When unsure, write the integration test; the cost is one constructor + ~20 lines.
240
+
Concrete skippable example: `fn name(&self) -> SString` reading a field already validated by another method. Concrete *non*-skippable examples: anything returning `SVec<T>`, `FFI_Option<T>`, `FFI_Result<T>`, `WrappedSchema`, `WrappedArray`, an `FfiFuture`, an `FFI_*` sub-struct, or any `*mut`/`*const` pointer. These exercise alignment / padding / niche-opt across the ABI boundary and need the two-build coverage. When unsure, write the integration test; the cost is one constructor + ~20 lines.
239
241
240
242
If you skip the integration test for a layout change, you have effectively shipped untested ABI.
241
243
@@ -313,6 +315,8 @@ If a method's body is non-trivial, the consumer-side default is non-trivial too.
313
315
-**Logical `Expr` / `LogicalPlan`**: serialize via `datafusion-proto` using the embedded `FFI_LogicalExtensionCodec`. Same for physical plans → `FFI_PhysicalExtensionCodec`.
314
316
-**Enums** (`Volatility`, `TableType`, `InsertOp`, `TableProviderFilterPushDown`): `#[repr(u8)]`, with `From<Native> for FFI_X` and `From<&FFI_X> for Native`. Always write a round-trip unit test that exercises every variant.
315
317
-**Errors**: every `FFI_X` method that can fail returns `FFI_Result<T>`. Use the `sresult!`, `sresult_return!`, `df_result!` macros from `src/util.rs` — do not roll your own.
318
+
-**Infallible trait methods**: if an FFI call can fail but the native trait cannot return the error, log the transport error before returning the trait's fallback (`None`, `false`, or a default). Never discard it with `.ok()`, `.unwrap_or_default()`, or equivalent.
319
+
-**Owned FFI returns**: consume an owned `FFI_X` with `From<FFI_X>` instead of converting through `&FFI_X` and cloning across the boundary. Keep the borrowed conversion's local marker fast path.
0 commit comments