Skip to content

[branch-55] Expand scalar async UDF results to the batch row count (#25828) - #27373

Merged
alamb merged 1 commit into
apache:branch-55from
kamilk-pal:backport-25828-branch-55
Nov 12, 2026
Merged

alamb merged 1 commit into
apache:branch-55from
kamilk-pal:backport-25828-branch-55

Conversation

@kamilk-pal

Copy link
Copy Markdown

## Which issue does this PR close?

- Closes apache#25875

## Rationale for this change

When an async UDF returns a `ColumnarValue::Scalar` (e.g. all arguments
are literals), queries on more than one row fail with `all columns in a
record batch must have the same length`.

## What changes are included in this PR?

`AsyncFuncExpr::invoke_with_args` expands each result to its chunk's row
count via `into_array_of_size`, instead of turning a scalar into a 1-row
array.

## What is the testing strategy for this PR?

Added `test_async_udf_with_scalar_result` in
`user_defined_async_scalar_functions.rs`. It fails without the fix.

## Are there any user-facing changes?

No.

(cherry picked from commit 1aed7dc)
@github-actions github-actions Bot added physical-expr Changes to the physical-expr crates core Core DataFusion crate labels Nov 12, 2026

@hareshkh hareshkh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honest backport of the original PR

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed- thank you @hareshkh

@alamb

alamb commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Will merge once CI is clean

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 20.00000% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.24%. Comparing base (4f4d73f) to head (b84f09d).

Files with missing lines Patch % Lines
...afusion/physical-expr/src/async_scalar_function.rs 20.00% 0 Missing and 4 partials ⚠️
@@              Coverage Diff              @@
##           branch-55   #27373      +/-   ##
=============================================
- Coverage      81.24%   81.24%   -0.01%     
=============================================
  Files           1110     1110              
  Lines         389341   389336       -5     
  Branches      389341   389336       -5     
=============================================
- Hits          316324   316317       -7     
  Misses         54446    54446              
- Partials       18571    18573       +2     
Files with missing lines Coverage Δ
...afusion/physical-expr/src/async_scalar_function.rs 63.90% <20.00%> (-1.31%) ⬇️

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@alamb
alamb merged commit 44a258e into apache:branch-55 Nov 12, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate physical-expr Changes to the physical-expr crates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants