Skip to content

fix(ffi): preserve aggregate null-handling support - #23908

Merged
kosiew merged 2 commits into
apache:mainfrom
Amogh-2404:fix/issue-22331-ffi-udaf-null-handling
Aug 4, 2026
Merged

fix(ffi): preserve aggregate null-handling support#23908
kosiew merged 2 commits into
apache:mainfrom
Amogh-2404:fix/issue-22331-ffi-udaf-null-handling

Conversation

@Amogh-2404

@Amogh-2404 Amogh-2404 commented Jul 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Part of #22331.

Rationale for this change

ForeignAggregateUDF inherits the default supports_null_handling_clause, so producer overrides are lost across the FFI boundary.

What changes are included in this PR?

  • Forward supports_null_handling_clause through FFI_AggregateUDF.
  • Use first_value as the positive case and sum as the negative case in forced-foreign and dynamic-library tests.

Are these changes tested?

  • cargo test -p datafusion-ffi --features integration-tests
  • cargo clippy --all-targets --all-features -- -D warnings

Are there any user-facing changes?

The FFI ABI changes. Foreign libraries must rebuild against the new DataFusion version.

@github-actions github-actions Bot added the ffi Changes to the ffi crate label Jul 26, 2026
@github-actions

github-actions Bot commented Jul 26, 2026 •
edited
Loading

Copy link
Copy Markdown

Thank you for opening this pull request!

Reviewer note: cargo-semver-checks reported the current version number is not SemVer-compatible with the changes in this pull request (compared against the base branch).

Details 
     Cloning apache/main
    Building datafusion-ffi v54.1.0 (current)
       Built [  62.648s] (current)
     Parsing datafusion-ffi v54.1.0 (current)
      Parsed [   0.062s] (current)
    Building datafusion-ffi v54.1.0 (baseline)
       Built [  61.666s] (baseline)
     Parsing datafusion-ffi v54.1.0 (baseline)
      Parsed [   0.064s] (baseline)
    Checking datafusion-ffi v54.1.0 -> v54.1.0 (no change; assume patch)
     Checked [   0.231s] 223 checks: 222 pass, 1 fail, 0 warn, 30 skip

--- failure constructible_struct_adds_field: externally-constructible struct adds field ---

Description:
A pub struct constructible with a struct literal has a new pub field. Existing struct literals must be updated to include the new field.
        ref: https://doc.rust-lang.org/reference/expressions/struct-expr.html
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.49.0/src/lints/constructible_struct_adds_field.ron

Failed in:
  field FFI_AggregateUDF.supports_null_handling_clause in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/udaf/mod.rs:150
  field ForeignLibraryModule.create_first_value_udaf in /home/runner/work/datafusion/datafusion/datafusion/ffi/src/tests/mod.rs:123

     Summary semver requires new major version: 1 major and 0 minor checks failed
    Finished [ 126.224s] datafusion-ffi

@github-actions github-actions Bot added the auto detected api change Auto detected API change label Jul 26, 2026
@codecov-commenter

codecov-commenter commented Jul 26, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.88889% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.65%. Comparing base (88365dd) to head (bb1e57e).
⚠️ Report is 95 commits behind head on main.

Files with missing lines Patch % Lines
datafusion/ffi/src/udaf/mod.rs 88.88% 1 Missing and 1 partial ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main   #23908      +/-   ##
==========================================
- Coverage   80.65%   80.65%   -0.01%     
==========================================
  Files        1093     1093              
  Lines      371619   371637      +18     
  Branches   371619   371637      +18     
==========================================
- Hits       299730   299727       -3     
- Misses      53993    54009      +16     
- Partials    17896    17901       +5

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

kosiew
kosiew approved these changes Aug 3, 2026
View reviewed changes

@kosiew kosiew left a comment •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Amogh-2404
Thanks for working on this!
This looks good to me. I just have one small suggestion that could improve coverage at the planner boundary.

Comment thread datafusion/ffi/tests/ffi_udaf.rs
}

#[test]
fn test_supports_null_handling_clause() -> Result<()> {

@kosiew kosiew Aug 3, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice addition. Since this test already verifies the cross-library callback and covers the FFI regression, it might also be worth registering the foreign UDAFs in a SessionContext and checking the SQL planner behavior. For example, you could assert that first_value(...) IGNORE NULLS plans successfully, while sum(...) IGNORE NULLS is rejected. That would exercise the user-facing planner boundary around supports_null_handling_clause, not just the direct trait dispatch.

@kosiew
kosiew requested a review from timsaucer August 3, 2026 09:39
timsaucer
timsaucer approved these changes Aug 3, 2026
View reviewed changes

@timsaucer timsaucer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice addition. Thank you for the PR!

@kosiew
kosiew added this pull request to the merge queue Aug 4, 2026
Merged via the queue into apache:main with commit db0c31b Aug 4, 2026
38 checks passed
kosiew pushed a commit to kosiew/datafusion that referenced this pull request Aug 12, 2026
@Amogh-2404 @kosiew
fix(ffi): preserve aggregate null-handling support (apache#23908)
e9c0cb9 
## Which issue does this PR close?

Part of apache#22331.

## Rationale for this change

`ForeignAggregateUDF` inherits the default
`supports_null_handling_clause`, so producer overrides are lost across
the FFI boundary.

## What changes are included in this PR?

- Forward `supports_null_handling_clause` through `FFI_AggregateUDF`.
- Use `first_value` as the positive case and `sum` as the negative case
in forced-foreign and dynamic-library tests.

## Are these changes tested?

- `cargo test -p datafusion-ffi --features integration-tests`
- `cargo clippy --all-targets --all-features -- -D warnings`

## Are there any user-facing changes?

The FFI ABI changes. Foreign libraries must rebuild against the new
DataFusion version.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@timsaucer timsaucer timsaucer approved these changes
@kosiew kosiew kosiew approved these changes

Assignees

No one assigned

Labels

auto detected api change Auto detected API change ffi Changes to the ffi crate v55.0.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Amogh-2404 @codecov-commenter @timsaucer @kosiew @nuno-faria