Skip to content

Native engine panics on all-scalar inputs for Substring and StringSpace #3337

Closed

Description

@andygrove

Description

When Spark's ConstantFolding optimizer rule is disabled, all-scalar (literal-only) expressions reach Comet's native engine. The native engine panics with:

  • Substring(scalar) should be fold in Spark JVM side.
  • StringSpace(scalar) should be fold in Spark JVM side.

How to Reproduce

SET spark.sql.optimizer.excludedRules=org.apache.spark.sql.catalyst.optimizer.ConstantFolding;
SELECT substring('hello world', 1, 5);
SELECT space(5);
SELECT left('hello', 3);

Expected Behavior

The native engine should handle scalar inputs gracefully — either by computing the correct result or by falling back to Spark, not by panicking.

Affected Expressions

  • substring() / left()
  • space()

Activity

  1. kazantsev-maksim commented on Feb 5, 2026

    @kazantsev-maksim
    Contributor

    I'd like to work on it.

  2. added
    priority:highCrashes, panics, segfaults, major functional breakage
    priority:lowMinor issues, test failures, tooling, cosmetic
    and removed
    priority:highCrashes, panics, segfaults, major functional breakage
    on Mar 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:expressionsExpression evaluationbugSomething isn't workingcrashNative engine crash/panic/segfaultpriority:lowMinor issues, test failures, tooling, cosmetic

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions