Skip to content

Commit 9f8fade

Browse files
committed
test: benchmark the codegen-dispatch path against the operator fallback it replaces
Every expression this PR newly routes through the JVM codegen dispatcher used to fall the whole projection back to Spark, so the comparison that matters is dispatch against `spark.comet.exec.scalaUDF.codegen.enabled=false`, not against Spark. `CometCodegenDispatchBenchmark` runs both, plus a Comet-off reference, over `lpad`/`rpad` on binary, `encode`, the utf-8 form of `to_binary`, `to_time(fmt)`, a projection that mixes a dispatched call with native ones, and a grouped aggregate that only stays native if the projection under it does. Sub-batch (1024) and multi-batch (1M) row counts. Each table repeats its baseline so the machine's noise floor is visible next to the arms, and each case is checked to be fully native with dispatch on, not fully native with it off, and to have actually reached the dispatcher -- the first two shapes I tried failed that check because `length()` on binary has no native path, which would have made those tables measure nothing. Compilation is a one-time cost that the steady-state tables warm away, so a separate table times a cold kernel against the same query run twice. All three arms are run and their rows compared before anything is timed.
1 parent 517c466 commit 9f8fade

1 file changed

Lines changed: 407 additions & 0 deletions

File tree

0 commit comments

Comments
 (0)