Standardize UDF coercion error messages by Jefffrey · Pull Request #20070 · apache/datafusion

Jefffrey · 2026-01-30T02:18:58Z

Which issue does this PR close?

Relates to Improve candidate functions suggestions #15817 and Improve error messages when coercion fails #19004

Rationale for this change

Current error message for calling functions with incorrect arguments isn't very user friendly. Some examples:

> select log('');
Error during planning: Internal error: Function 'log' failed to match any signature, errors: Error during planning: Function 'log' requires TypeSignatureClass::Decimal, but received String (DataType: Utf8).,Error during planning: Function 'log' requires TypeSignatureClass::Float, but received String (DataType: Utf8).,Error during planning: Function 'log' expects 2 arguments but received 1,Error during planning: Function 'log' expects 2 arguments but received 1.
This issue was likely caused by a bug in DataFusion's code. Please help us to resolve this by filing a bug report in our issue tracker: https://github.com/apache/datafusion/issues No function matches the given name and argument types 'log(Utf8)'. You might need to add explicit type casts.
        Candidate functions:
        log(Coercion(TypeSignatureClass::Decimal))
        log(Coercion(TypeSignatureClass::Float, implicit_coercion=ImplicitCoercion([Numeric], default_type=Float64))
        log(Coercion(TypeSignatureClass::Float, implicit_coercion=ImplicitCoercion([Numeric], default_type=Float64), Coercion(TypeSignatureClass::Decimal))
        log(Coercion(TypeSignatureClass::Float, implicit_coercion=ImplicitCoercion([Numeric], default_type=Float64), Coercion(TypeSignatureClass::Float, implicit_coercion=ImplicitCoercion([Numeric], default_type=Float64))
> select abs('');
Error during planning: Function 'abs' expects NativeType::Numeric but received NativeType::String No function matches the given name and argument types 'abs(Utf8)'. You might need to add explicit type casts.
        Candidate functions:
        abs(Numeric(1))
> select make_array([]);
+--------------------------+
| make_array(make_array()) |
+--------------------------+
| [[]]                     |
+--------------------------+
1 row(s) fetched.
Elapsed 0.051 seconds.
> select greatest();
Error during planning: Execution error: Function 'greatest' user-defined coercion failed with "Error during planning: greatest was called without any arguments. It requires at least 1." No function matches the given name and argument types 'greatest()'. You might need to add explicit type casts.
        Candidate functions:
        greatest(UserDefined)

There's quite some word spam, especially for the log case; it stems from using a OneOf signature, and we currently concatenate each error from each signature inside the OneOf resulting in a large error message that isn't very helpful
Some error messages are direct about whats wrong, such as abs, which is nice
User defined signatures still display Candidate functions which isn't very helpful (greatest(UserDefined) doesn't really help the user understand the available signatures since user defined is opaque)

Contrast this with DuckDB:

D select log([]);
Binder Error:
No function matches the given name and argument types 'log("NULL"[])'. You might need to add explicit type casts.
        Candidate functions:
        log(DOUBLE) -> DOUBLE
        log(DOUBLE, DOUBLE) -> DOUBLE


LINE 1: select log([]);

Or Postgres:

postgres=# select sha256(1, 1);
ERROR:  function sha256(integer, integer) does not exist
LINE 1: select sha256(1, 1);
               ^
HINT:  No function matches the given name and argument types. You might need to add explicit type casts.

It looks like they prefer to omit stating specific errors, and only give general advice; for DuckDB they list the available call types, for Postgres they omit this and leave it a simple error with only details on how it was called.

This PR looks into removing some of the word spam from error messages and tries to make them more consistent with each other.

What changes are included in this PR?

When a function call fails because no signatures match, the displayed error either:

For user-defined signatures, it will simply pass on the error from the coerce_types call of the UDF (we omit candidate functions message)
For other signatures, we strip away the specific error message and display only the candidate functions

For example, user-defined:

> select greatest();
Error during planning: User-defined coercion of function call 'greatest()' failed with:
greatest was called without any arguments. It requires at least 1.

For all other signatures:

> select log([]);
Error during planning: Failed to coerce function call 'log(List(Null))'. You might need to add explicit type casts.
        Candidate functions:
        log(Coercion(TypeSignatureClass::Decimal))
        log(Coercion(TypeSignatureClass::Float, implicit_coercion=ImplicitCoercion([Numeric], default_type=Float64))
        log(Coercion(TypeSignatureClass::Float, implicit_coercion=ImplicitCoercion([Numeric], default_type=Float64), Coercion(TypeSignatureClass::Decimal))
        log(Coercion(TypeSignatureClass::Float, implicit_coercion=ImplicitCoercion([Numeric], default_type=Float64), Coercion(TypeSignatureClass::Float, implicit_coercion=ImplicitCoercion([Numeric], default_type=Float64))
> select abs('');
Error during planning: Failed to coerce function call 'abs(Utf8)'. You might need to add explicit type casts.
        Candidate functions:
        abs(Numeric(1))

Although it means we don't show a specific error anymore (see how abs no longer specifies it was because it received string instead of a numeric), I think this is an easier way to manage signature errors to be consistent for functions that have a single signature and those that are OneOf types. It also aligns with DuckDB.

Are these changes tested?

Yes.

Are there any user-facing changes?

Yes, error messages for failed function calls changes.

Jefffrey · 2026-01-30T05:02:10Z

If we decide to go this route, there is still some followup work:

Improve display of coercible API (e.g. the candidate functions of log isn't very user friendly)
Look into cleaning up error messages from type_coercion/functions.rs given we strip away most of them (all except user defined) for simplicity
Look into making array signatures consistent

For example this is how array function errors look like:

> select array_extract(1,2,3);
Error during planning: Failed to coerce function call 'array_element(Int64, Int64, Int64)'. You might need to add explicit type casts.
        Candidate functions:
        array_element(array, index)

Ideally we'd show specific types instead of just index; though this is more about ensuring all the signatures have a consistent way of displaying themselves

alamb

Thanks @Jefffrey

Some of the error messages in slt look less good to me -- though maybe the issue is that they don't include the entire error

alamb · 2026-02-03T21:46:12Z

 select union_extract(union_column, 'bool') from union_table;

-query error DataFusion error: Error during planning: 'union_extract' does not support zero arguments
+query error Failed to coerce function call 'union_extract\(\)'


the old message about not supporting zero arguments seems like it was nicer

alamb · 2026-02-03T21:47:06Z

 1

-query error DataFusion error: Error during planning: Function 'strpos' requires TypeSignatureClass::Native\(LogicalType\(Native\(String\), String\)\), but received Int64 \(DataType: Int64\)
+query error Failed to coerce function call 'strpos\(Int64, Int64\)'


it feels like the previous messages were better even if they were more verbose, as they at least try to explain to a user what was expected. I do admit that the TypeSignatureClass::Native<... is not very user friendly, however

Jefffrey · 2026-02-04T14:25:14Z

Yeah I was initially thinking of trying to cleanup the existing errors, specifically cases where they are really ugly (coercion API, also if the signature is a OneOf). But after looking at DuckDB I decided to try this way to make it more generic.

I can work towards keeping the existing specific error messages, but my main concern is how to deal with OneOf signature. For argument count errors (none of the OneOf signatures count of arguments match) I can fix the current logic to be something like udf expects 1 to 4 arguments, got 5.

e.g. log can take 1 or 2 arguments, so if we supply 3 we can form an error log expects 1 to 2 arguments, got 3

However for cases where the input arguments match more than one of the OneOf signatures in count, I wonder how to neatly handle the error? For example, log:

datafusion/datafusion/functions/src/math/log.rs

Lines 83 to 100 in 796c7d1

    
           signature: Signature::one_of( 
        
               // Ensure decimals have precedence over floats since we have 
        
               // a native decimal implementation for log 
        
               vec![ 
        
                   // log(value) 
        
                   TypeSignature::Coercible(vec![Coercion::new_exact( 
        
                       TypeSignatureClass::Decimal, 
        
                   )]), 
        
                   TypeSignature::Coercible(vec![as_float.clone()]), 
        
                   // log(base, value) 
        
                   TypeSignature::Coercible(vec![ 
        
                       as_float.clone(), 
        
                       Coercion::new_exact(TypeSignatureClass::Decimal), 
        
                   ]), 
        
                   TypeSignature::Coercible(vec![as_float.clone(), as_float.clone()]), 
        
               ], 
        
               Volatility::Immutable, 
        
           ),

For the 2 argument case, it expects [float, float] or [float, decimal]

If we supply [float, string], how should we form the error? Currently we just append errors from each of the OneOf signatures which leads to a really ugly error:

> select log(1, '');
Error during planning: Internal error: Function 'log' failed to match any signature, errors: Error during planning: Function 'log' expects 1 arguments but received 2,Error during planning: Function 'log' expects 1 arguments but received 2,Error during planning: Function 'log' requires TypeSignatureClass::Decimal, but received String (DataType: Utf8).,Error during planning: Function 'log' requires TypeSignatureClass::Float, but received String (DataType: Utf8)..
This issue was likely caused by a bug in DataFusion's code. Please help us to resolve this by filing a bug report in our issue tracker: https://github.com/apache/datafusion/issues No function matches the given name and argument types 'log(Int64, Utf8)'. You might need to add explicit type casts.
        Candidate functions:
        log(Coercion(TypeSignatureClass::Decimal))
        log(Coercion(TypeSignatureClass::Float, implicit_coercion=ImplicitCoercion([Numeric], default_type=Float64))
        log(Coercion(TypeSignatureClass::Float, implicit_coercion=ImplicitCoercion([Numeric], default_type=Float64), Coercion(TypeSignatureClass::Decimal))
        log(Coercion(TypeSignatureClass::Float, implicit_coercion=ImplicitCoercion([Numeric], default_type=Float64), Coercion(TypeSignatureClass::Float, implicit_coercion=ImplicitCoercion([Numeric], default_type=Float64))

Can try to fix the display + internal error reference, but how to handle the following:

Function 'log' requires TypeSignatureClass::Decimal, but received String (DataType: Utf8).,Error during planning: Function 'log' requires TypeSignatureClass::Float, but received String (DataType: Utf8).

Maybe only for cases like this we erase the error message and only show candidate functions, like this PR does?

## Which issue does this PR close? Part of apache#20501. This is a deliberately narrow slice of that issue, and it is the slice a maintainer previously asked for: apache#21032 attempted this fix as a large change (it also reworked `sum`/`avg` diagnostics and the public `first_value`/`last_value`/`nth_value` arity contracts), was asked to be split up, and then went stale. This PR is **one behaviour change and nothing else**. ## Rationale for this change `get_valid_types_with_udf` reports "no `OneOf` branch matched" with `internal_err!`. Every alternative signature failing means the *caller* passed the wrong number or types of arguments — a user mistake — yet the user is told: > This issue was likely caused by a bug in DataFusion's code. Please help us to resolve this by filing a bug report in our issue tracker: ... So an ordinary SQL typo invites a bug report, and the failure surfaces as `Internal error: ...` even though nothing internal went wrong. The maintainer comment on apache#20070 named exactly this ("Can try to fix the display + internal error reference"). ### Before / after `SELECT substr(1, 2)` Before: ```text Error during planning: Internal error: Function 'substr' failed to match any signature, errors: Error during planning: Function 'substr' requires String, but received Int64 (DataType: Int64).,Error during planning: Function 'substr' expects 3 arguments but received 2. This issue was likely caused by a bug in DataFusion's code. Please help us to resolve this by filing a bug report in our issue tracker: https://github.com/apache/datafusion/issues ``` After: ```text Error during planning: Function 'substr' failed to match any signature, errors: Error during planning: Function 'substr' requires String, but received Int64 (DataType: Int64).,Error during planning: Function 'substr' expects 3 arguments but received 2. ``` The message body, the candidate signatures, and the trailing "No function matches the given name and argument types ... You might need to add explicit type casts." line are all unchanged. Only the `Internal error` kind and the bug-report boilerplate are gone. ## What changes are included in this PR? - `datafusion/expr/src/type_coercion/functions.rs`: return `plan_err!` instead of `internal_err!` when every `OneOf` branch rejects the arguments. One line of behaviour change. - A unit test covering that failure. It mirrors `substr`'s real signature, so one branch fails on the argument type and the other on the arity. - Ten `sqllogictest` expectations that asserted the old user-visible text, in `array/array_range.slt`, `spark/math/round.slt`, `errors.slt` and `math.slt`. ## Are these changes tested? Yes. - The new unit test fails on `main` and passes with the change; `cargo test -p datafusion-expr --lib` is 282 passed, 0 failed. - `cargo fmt --all --check` is clean. - I audited every other expectation of this message. The remaining ones — `functions.slt`, `aggregate.slt`, `string_literal.slt`, `named_arguments.slt`, and one assertion in `datafusion/optimizer/src/analyzer/type_coercion.rs` — match on a substring that does not include the removed prefix, so they are unaffected. No `sqllogictest` file asserted the "This issue was likely caused by a bug..." boilerplate. I could not run the full `sqllogictest` suite locally (it needs a full `datafusion-core` build), so CI is the verification for the four updated `.slt` files. ## Scope Deliberately left out, to keep this reviewable: - the message is still verbose and still prefixes every branch error with `Error during planning:` — that is apache#19004; - `errors.join(",")` still runs the branch errors together without a separator, which is why the after-message reads `...(DataType: Int64).,Error during planning: ...`; - the arity phrasing suggested on apache#20070 (`log expects 1 to 2 arguments, got 3`) is a separate improvement. Happy to follow up on any of those in separate PRs.

Standardize UDF coercion error messages

f873568

github-actions Bot added sql SQL Planner logical-expr Logical plan and expressions optimizer Optimizer rules sqllogictest SQL Logic Tests (.slt) labels Jan 30, 2026

fix tests

cf5484e

Jefffrey marked this pull request as ready for review January 30, 2026 05:02

alamb reviewed Feb 3, 2026

View reviewed changes

Jefffrey mentioned this pull request Mar 2, 2026

Type signature matching errors are overly verbose and show internal details #20501

Open

timsaucer mentioned this pull request Mar 2, 2026

Improve formatting of datatypes #20605

Merged

myandpr mentioned this pull request Mar 18, 2026

fix: avoid internal errors for OneOf signature mismatches #21032

Closed

Jefffrey closed this Apr 15, 2026

Jefffrey deleted the udf-errors branch April 15, 2026 08:47

SingletonYe mentioned this pull request Oct 14, 2026

fix: report OneOf signature mismatches as planning errors #26329

Merged

Navigation Menu

Repository navigation

Standardize UDF coercion error messages - #20070

Standardize UDF coercion error messages#20070
Jefffrey wants to merge 2 commits into
apache:mainfrom
Jefffrey:udf-errors

Jefffrey commented Jan 30, 2026 •

edited

Loading

Uh oh!

Jefffrey commented Jan 30, 2026

Uh oh!

alamb left a comment

Uh oh!

alamb Feb 3, 2026

Uh oh!

alamb Feb 3, 2026

Uh oh!

Jefffrey commented Feb 4, 2026 •

edited

Loading

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Jefffrey commented Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

Jefffrey commented Jan 30, 2026

Uh oh!

alamb left a comment

Choose a reason for hiding this comment

Uh oh!

alamb Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

alamb Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

Jefffrey commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Jefffrey commented Jan 30, 2026 •

edited

Loading

Jefffrey commented Feb 4, 2026 •

edited

Loading