Skip to content

Substring not planned under Substrait when unicode_expressions` feature is disabled #17582

@kosiew

Description

@kosiew

Description:

When running the Substrait tests with DataFusion built without the unicode_expressions feature (in datafusion/substrait/Cargo.toml only nested_expressions is enabled), the cases::roundtrip_logical_plan::simple_scalar_function_substr test fails with:

NotImplemented("Substring could not be planned by registered expr planner. Hint: enable the unicode_expressions")

Concretely:

  • The UnicodeFunctionPlanner is not registered because the unicode_expressions feature is disabled.
  • As a result, the SUBSTR / substring function cannot be planned under the Substrait logical plan conversion.
  • This causes the simple_scalar_function_substr round‐trip logical plan test to fail under those build settings.

Steps to reproduce:

  1. Checkout the DataFusion repo.
  2. Build without unicode_expressions (so that only nested_expressions is enabled in the Substrait feature set).
  3. Run the Substrait tests — cargo test -p datafusion-substrait --test substrait_integration or specifically cases::roundtrip_logical_plan::simple_scalar_function_substr
  4. Observe the failure: NotImplemented("Substring could not be planned by registered expr planner. Hint: enable the unicode_expressions").

Expected behavior:

Either

  • The Substr / substring function should be plannable even when unicode_expressions is disabled (i.e. provide a fallback planner / expression implementation), or
  • At least the test suite should skip or mark this as expected behavior when unicode_expressions is off, rather than failing.

References:

  • The error message: Substring could not be planned by registered expr planner. Hint: enable the unicode_expressions
  • The failing test: cases::roundtrip_logical_plan::simple_scalar_function_substr

Issue was discovered in this PR comment - #17468 (comment)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions