Skip to content

[EPIC] Optimize performance for slow expressions #2986

Closed

Description

@andygrove

What is the problem the feature request solves?

The following expressions are slower with Comet enabled, according to the benchmarks in #2984

This epic is for tracking progress on optimizing these. Separate issues should be created and linked to from this table. Some issues already exist (look for issues tagged with the performance label).

Also, I'd like to point out that this table was generated by AI and contains some duplicate entries, and may also have errors.

Strings

Benchmark File Expression Spark Time (ms) Comet Relative Slowdown
CometStringExpressionBenchmark octet_length 373.0 0.4X 60.0%
CometStringExpressionBenchmark trim 435.0 0.4X 60.0%
CometStringExpressionBenchmark ltrim 434.0 0.4X 60.0%
CometStringExpressionBenchmark rtrim 436.0 0.4X 60.0%
CometStringExpressionBenchmark repeat 720.0 0.4X 60.0%
CometStringExpressionBenchmark concat 595.0 0.5X 50.0%
CometStringExpressionBenchmark startswith 396.0 0.5X 50.0%
CometStringExpressionBenchmark ascii 405.0 0.6X 40.0%
CometStringExpressionBenchmark bit_length 451.0 0.6X 40.0%
CometStringExpressionBenchmark concat_ws 702.0 0.6X 40.0%
CometStringExpressionBenchmark instr 3805.0 0.6X 40.0%
CometStringExpressionBenchmark endswith 414.0 0.6X 40.0%
CometStringExpressionBenchmark chr 27.0 0.8X 20.0%
CometStringExpressionBenchmark space 28.0 0.8X 20.0%
CometStringExpressionBenchmark translate 28908.0 0.8X 20.0%
CometStringExpressionBenchmark initCap 4560.0 0.9X 10.0%
CometStringExpressionBenchmark rlike 3396.0 0.9X 10.0%

Date/Timestamp

Benchmark File Expression Spark Time (ms) Comet Relative Slowdown
CometDatetimeExpressionBenchmark Timestamp Truncate 134.0 0.6X 40.0%
CometDatetimeExpressionBenchmark Date Truncate 34.0 0.8X 20.0%
CometDatetimeExpressionBenchmark Timestamp Extract - year 61.0 0.8X 20.0%
CometDatetimeExpressionBenchmark Date Extract - year 24.0 0.9X 10.0%
CometDatetimeExpressionBenchmark Date Arithmetic - date_add 24.0 0.9X 10.0%

Arrays

Benchmark File Expression Spark Time (ms) Comet Relative Slowdown
CometArrayExpressionBenchmark array_remove 12.0 0.5X 50.0%
CometArrayExpressionBenchmark array_compact 13.0 0.5X 50.0%
CometArrayExpressionBenchmark array_max 13.0 0.8X 20.0%
CometArrayExpressionBenchmark array_min 12.0 0.8X 20.0%
CometArrayExpressionBenchmark array_contains 15.0 0.9X 10.0%
CometArrayExpressionBenchmark array_distinct 14.0 0.9X 10.0%
CometArrayExpressionBenchmark array_append 12.0 0.9X 10.0%
CometArrayExpressionBenchmark arrays_overlap 12.0 0.9X 10.0%
CometArrayExpressionBenchmark array_insert 11.0 0.9X 10.0%
CometArrayExpressionBenchmark array_join 13.0 0.9X 10.0%

Hash

Benchmark File Expression Spark Time (ms) Comet Relative Slowdown
CometHashExpressionBenchmark xxhash64_multi 15.0 0.9X 10.0%
CometHashExpressionBenchmark murmur3_hash_single 13.0 0.9X 10.0%
CometHashExpressionBenchmark murmur3_hash_multi 14.0 0.9X 10.0%
CometHashExpressionBenchmark sha2_224 28.0 0.8X 20.0%
CometHashExpressionBenchmark sha2_256 29.0 0.8X 20.0%
CometHashExpressionBenchmark sha2_512 34.0 0.6X 40.0%
CometHashExpressionBenchmark sha2_384 34.0 0.7X 30.0%

Bitwise

Benchmark File Expression Spark Time (ms) Comet Relative Slowdown
CometBitwiseExpressionBenchmark shift_right_unsigned 10.0 0.9X 10.0%
CometBitwiseExpressionBenchmark shift_left 10.0 0.7X 30.0%
CometBitwiseExpressionBenchmark bitwise_or 12.0 0.8X 20.0%
CometBitwiseExpressionBenchmark bitwise_xor 11.0 0.8X 20.0%
CometBitwiseExpressionBenchmark bitwise_not 10.0 0.8X 20.0%
CometBitwiseExpressionBenchmark shift_right 10.0 0.8X 20.0%
CometBitwiseExpressionBenchmark bit_count 10.0 0.8X 20.0%

Cast

Benchmark File Expression Spark Time (ms) Comet Relative Slowdown
CometCastStringToNumericBenchmark CAST String to BYTE 59.0 0.8X 20.0%
CometCastStringToNumericBenchmark CAST String to SHORT 59.0 0.8X 20.0%
CometCastStringToNumericBenchmark CAST String to INT 56.0 0.8X 20.0%
CometCastStringToNumericBenchmark CAST String to LONG 59.0 0.8X 20.0%

Comparison

Benchmark File Expression Spark Time (ms) Comet Relative Slowdown
CometComparisonExpressionBenchmark greater_than 11.0 0.8X 20.0%
CometComparisonExpressionBenchmark is_null 10.0 0.8X 20.0%
CometComparisonExpressionBenchmark is_nan_float 10.0 0.8X 20.0%
CometComparisonExpressionBenchmark not_equal_to 13.0 0.9X 10.0%
CometComparisonExpressionBenchmark less_than 12.0 0.9X 10.0%
CometComparisonExpressionBenchmark less_than_or_equal 11.0 0.9X 10.0%
CometComparisonExpressionBenchmark greater_than_or_equal 11.0 0.9X 10.0%
CometComparisonExpressionBenchmark equal_null_safe 10.0 0.9X 10.0%
CometComparisonExpressionBenchmark is_not_null 10.0 0.9X 10.0%
CometComparisonExpressionBenchmark and 11.0 0.9X 10.0%
CometComparisonExpressionBenchmark or 11.0 0.9X 10.0%
CometComparisonExpressionBenchmark not 10.0 0.9X 10.0%
CometComparisonExpressionBenchmark in_list 10.0 0.9X 10.0%
CometComparisonExpressionBenchmark not_in_list 11.0 0.9X 10.0%

Math

Benchmark File Expression Spark Time (ms) Comet Relative Slowdown
CometMathExpressionBenchmark hex_int 11.0 0.7X 30.0%
CometMathExpressionBenchmark floor 10.0 0.8X 20.0%
CometMathExpressionBenchmark hex_long 11.0 0.8X 20.0%
CometMathExpressionBenchmark unhex 13.0 0.8X 20.0%
CometMathExpressionBenchmark unary_minus 10.0 0.8X 20.0%
CometMathExpressionBenchmark ceil 11.0 0.9X 10.0%
CometMathExpressionBenchmark round 19.0 0.9X 10.0%
CometMathExpressionBenchmark atan2 11.0 0.9X 10.0%
CometMathExpressionBenchmark log 11.0 0.9X 10.0%
CometMathExpressionBenchmark log10 11.0 0.9X 10.0%

Others

Benchmark File Expression Spark Time (ms) Comet Relative Slowdown
CometConditionalExpressionBenchmark Case When Expr 41.0 0.8X 20.0%
CometPredicateExpressionBenchmark in Expr 42.0 0.8X 20.0%
CometConditionalExpressionBenchmark If Expr 38.0 0.9X 10.0%

Describe the potential solution

No response

Additional context

No response

Activity

  1. added this to the 0.14.0 milestone on Dec 24, 2025
  2. coderfender commented on Dec 25, 2025

    @coderfender
    Contributor

    @andygrove thank you for filing this

  3. Brijesh-Thakkar commented on Dec 28, 2025

    @Brijesh-Thakkar

    @coderfender Can I work on this issue??
    We tried to speedup string operations right??
    in another pr in that we found that for "trim" the issue could be solved by improvements in datafusion not in comet, right @coderfender

  4. andygrove commented on Dec 29, 2025

    @andygrove
    MemberAuthor

    @coderfender Can I work on this issue?? We tried to speedup string operations right?? in another pr in that we found that for "trim" the issue could be solved by improvements in datafusion not in comet, right @coderfender

    @Brijesh-Thakkar feel free to work on items from this epic. In some cases, it makes sense to make the improvements in DataFusion and in some cases we need to make the improvements in Comet directly.

  5. Brijesh-Thakkar commented on Dec 29, 2025

    @Brijesh-Thakkar

    @andygrove Okk , Thanks for guiding me
    I will work on this issue but i think @getChan is alreadyy workingg, as he has linked a pr
    @getChan, please confirm

  6. andygrove commented on Dec 29, 2025

    @andygrove
    MemberAuthor

    @andygrove Okk , Thanks for guiding me I will work on this issue but i think @getChan is alreadyy workingg, as he has linked a pr @getChan, please confirm

    @Brijesh-Thakkar this issue is an EPIC listing dozens of expressions that need optimizing. A separate issue should be created for each expression.

  7. Brijesh-Thakkar commented on Dec 29, 2025

    @Brijesh-Thakkar

    @andygrove Okk , Thanks for guiding me I will work on this issue but i think @getChan is alreadyy workingg, as he has linked a pr @getChan, please confirm

    @Brijesh-Thakkar this issue is an EPIC listing dozens of expressions that need optimizing. A separate issue should be created for each expression.

    ohhh okk, understood
    I will surelyy try to optimise expressions

  8. coderfender commented on Dec 29, 2025

    @coderfender
    Contributor

    @andygrove seems like some benchmarks are failing with ANSI mode enabled (CometCastBenchmark) . Raised a PR to fix this : #3014

  9. Brijesh-Thakkar commented on Dec 31, 2025

    @Brijesh-Thakkar

    @andygrove I’ve opened an upstream PR in DataFusion to optimize the
    octet_length function by avoiding the generic Arrow length kernel and
    using a specialized implementation for string arrays.

    PR: apache/datafusion#19581

  10. Brijesh-Thakkar commented on Jan 1, 2026

    @Brijesh-Thakkar

    @andygrove I have opened PR In datafusion to optimise the bit_length function

    PR: apache/datafusion#19598

  11. raushanprabhakar1 commented on Jan 3, 2026

    @raushanprabhakar1

    Hi @andygrove , is there any documentation to which i can refer for running the benchmark in my local device? I was working on the expression optimization, the blocker i am facing is that i am not able to verify my optimzation result locally.

  12. coderfender commented on Jan 4, 2026

    @coderfender
    Contributor

    @raushanprabhakar1 , you can run a local benchmark using a command like below :
    SPARK_GENERATE_BENCHMARK_FILES=1 make benchmark-org.apache.spark.sql.benchmark.CometCastBenchmark

  13. 26 remaining items

  14. andygrove commented on Sep 28, 2026

    @andygrove
    MemberAuthor

    This table came from the old Scala microbenchmarks, which #5363 showed were timing scan and columnar-to-row transfer, excluded ConstantFolding only for Comet, and ran the string suites at 1024 rows, so many rows here are not real regressions (and #5372 found the translate row was measuring Spark). The expressions with concrete follow-ups have their own issues (#3022, #3025, #3027, #3029), and per-expression tuning now goes through the criterion benches tracked in #5396, #4942 and #5128. I think we can close this in favour of those.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions