Skip to content

Commit dd6ae32

Browse files
committed
fix: keep the null short-circuit for leaf-only multi-input trees
Addresses review feedback on #5219. The previous fix dropped the union-of-ordinals short-circuit for every multi-argument NullIntolerant tree, not only the ones that could swallow an error. The bug needs a subtree between the root and a leaf that can raise independently of nullness, like the Cast in add_months(CAST(s AS DATE), i). A BoundReference or Literal read cannot raise on its own, so when every direct child of the root is a leaf the tree is one level deep and the union short-circuit is still exact: any error comes from the root's own logic, which runs only once every input is known non-null, in Spark's shape and in ours alike. canShortCircuitNulls now accepts either a single input ordinal or a leaf-only-children root, restoring the fast path for the common no-cast multi-argument case (pmod, conv, make_timestamp) across the ~70 expressions that route through this dispatcher. defaultBody emits the disjunction over every ordinal the tree reads rather than only the first. Verified against Spark's generated code that the leaf-only rule holds even where Spark reorders children: DivModLike.doGenCode and Pmod.doGenCode evaluate the divisor first but still test the dividend's null before their divide/remainder-by-zero throw, so pmod(NULL, 0) returns NULL rather than raising.
1 parent 9354ad0 commit dd6ae32

3 files changed

Lines changed: 200 additions & 46 deletions

File tree

‎spark/src/main/scala/org/apache/comet/codegen/CometBatchKernelCodegen.scala‎

Lines changed: 56 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -337,9 +337,10 @@ object CometBatchKernelCodegen extends Logging with CometExprTraitShim with Come
337337
}
338338

339339
/**
340-
* Per-row body. For `NullIntolerant` expressions over a single input ordinal, prepends a
341-
* short-circuit on that ordinal so the whole `ev.code` cost is skipped on null rows. Otherwise
342-
* the standard shape: run `ev.code`, then `setNull` or write based on `ev.isNull`.
340+
* Per-row body. For `NullIntolerant` expressions whose input nulls fully determine a null
341+
* output (see [[canShortCircuitNulls]]), prepends a short-circuit on those ordinals so the
342+
* whole `ev.code` cost is skipped on null rows. Otherwise the standard shape: run `ev.code`,
343+
* then `setNull` or write based on `ev.isNull`.
343344
*
344345
* `subExprsCode` is the CSE helper-invocation block. It must run before `ev.code`. Inside the
345346
* short-circuit it lives in the else branch so null rows skip CSE too.
@@ -356,9 +357,13 @@ object CometBatchKernelCodegen extends Logging with CometExprTraitShim with Come
356357
// exposes `isNullAt(int)` rather than the raw Arrow `isNull(int)`. Pick the right method
357358
// for the ordinal so the short-circuit compiles for timestamp / int / float columns too,
358359
// not just VarChar / Decimal vectors that stay as raw Arrow types.
359-
val ord = inputOrdinals.head
360-
val nullCheck =
361-
s"this.col$ord.${CometBatchKernelCodegenInput.nullCheckMethod(inputSchema(ord))}(i)"
360+
//
361+
// Multi-ordinal trees test the disjunction of their ordinals. That is only reachable for
362+
// leaf-only-children roots, where it is exact; see [[canShortCircuitNulls]].
363+
val nullCheck = inputOrdinals
364+
.map(ord =>
365+
s"this.col$ord.${CometBatchKernelCodegenInput.nullCheckMethod(inputSchema(ord))}(i)")
366+
.mkString(" || ")
362367
// `NullIntolerant` only constrains "any input null -> output null"; it does NOT promise
363368
// that non-null inputs always produce non-null output. `MakeTimestamp(failOnError=false)`
364369
// is `NullIntolerant=true` but its `doGenCode` catches `DateTimeException` for invalid
@@ -414,24 +419,57 @@ object CometBatchKernelCodegen extends Logging with CometExprTraitShim with Come
414419
}
415420

416421
/**
417-
* Gates the [[defaultBody]] null short-circuit. Three conditions, all necessary:
422+
* Gates the [[defaultBody]] null short-circuit ("any input null implies null output", tested as
423+
* the disjunction of the tree's input ordinals before `ev.code` runs).
418424
*
419-
* - The tree reads exactly one input ordinal. Spark's null handling is per-node and
420-
* left-to-right (`BinaryExpression.nullSafeCodeGen` emits the left child's code
421-
* unconditionally, then tests the left child's null, then the right child's), so a
422-
* short-circuit on the union of several ordinals is not equivalent: it skips a subtree that
423-
* Spark would have evaluated, and with it any error that subtree raises. Under ANSI,
424-
* `add_months(cast(s as date), i)` on `('notadate', NULL)` raises `CAST_INVALID_INPUT` in
425-
* Spark, so returning null here would be wrong. With a single ordinal there is nothing left
426-
* for Spark to evaluate ahead of that ordinal's own null check, so the short-circuit is
427-
* exact. (A literal-only subtree that raises would be the one exception, but Catalyst's
428-
* `ConstantFolding` evaluates those at analysis time and never leaves one in the tree.)
425+
* The short-circuit is only equivalent to Spark when no subtree that Spark would have evaluated
426+
* ahead of a null check gets skipped. Spark's null handling is per-node and left-to-right:
427+
* `BinaryExpression.nullSafeCodeGen` emits the left child's code unconditionally, then tests
428+
* the left child's null, then the right child's. So for `add_months(cast(s as date), i)` on
429+
* `('notadate', NULL)` Spark evaluates the cast and, under ANSI, raises `CAST_INVALID_INPUT`; a
430+
* short-circuit on the union of ordinals would skip the cast and return null, swallowing the
431+
* error (#5218).
432+
*
433+
* Conditions, all necessary:
434+
*
435+
* - The tree reads at least one input ordinal. A literal-only tree has nothing to test.
436+
* (Catalyst's `ConstantFolding` evaluates those at analysis time, so this is defensive.)
429437
* - The root is `NullIntolerant`, so a null input really does mean a null result.
430438
* - Every node in the tree is null-propagating ([[allNullIntolerant]]); a `Coalesce` / `If` /
431439
* `CaseWhen` anywhere would break the chain.
440+
* - Either the tree reads exactly one ordinal, or every direct child of the root is a leaf
441+
* ([[rootChildrenAreLeaves]]). Both shapes make the short-circuit exact:
442+
* - One ordinal: there is nothing left for Spark to evaluate ahead of that ordinal's own
443+
* null check.
444+
* - Leaf-only children: the tree is one level deep, so the only code Spark runs ahead of
445+
* its null checks is `BoundReference` / `Literal` reads, which cannot raise. Any error
446+
* comes from the root's own logic, which runs only once every input is known non-null --
447+
* in both Spark's shape and ours. This keeps the fast path for the common no-cast
448+
* multi-argument case (`pmod(a, b)`, `conv(a, b, c)`, `make_timestamp(y, m, d, h, mi,
449+
* s)`) across the ~70 expressions that route through this dispatcher.
450+
*
451+
* The leaf-only rule relies on a `NullIntolerant` root not raising before it has checked every
452+
* input for null. Spark's own generated code honors that even where it reorders children: the
453+
* divide/remainder family (`DivModLike.doGenCode`, and `Pmod.doGenCode`, which carries its own
454+
* copy of the same shape) evaluates the divisor first, but still tests the dividend's null
455+
* *before* its divide/remainder-by-zero throw, so `pmod(NULL, 0)` returns null rather than
456+
* raising. A root that raised ahead of its null checks would contradict `NullIntolerant`.
432457
*/
433458
private def canShortCircuitNulls(expr: Expression, inputOrdinals: Seq[Int]): Boolean =
434-
inputOrdinals.size == 1 && isNullIntolerant(expr) && allNullIntolerant(expr)
459+
inputOrdinals.nonEmpty && isNullIntolerant(expr) && allNullIntolerant(expr) &&
460+
(inputOrdinals.size == 1 || rootChildrenAreLeaves(expr))
461+
462+
/**
463+
* True iff every direct child of the root is a leaf (`BoundReference` or `Literal`), i.e. the
464+
* tree is one level deep with no subtree between the root and its inputs. One of the conditions
465+
* on [[canShortCircuitNulls]]: it is what makes a multi-ordinal short-circuit exact, because a
466+
* leaf read cannot raise an error that Spark's left-to-right evaluation would have surfaced.
467+
*/
468+
private def rootChildrenAreLeaves(expr: Expression): Boolean =
469+
expr.children.forall {
470+
case _: BoundReference | _: Literal => true
471+
case _ => false
472+
}
435473

436474
/**
437475
* True iff every node in the tree propagates nulls (`NullIntolerant`, `BoundReference`, or

‎spark/src/test/scala/org/apache/comet/CometCodegenSourceSuite.scala‎

Lines changed: 91 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ import org.scalatest.funsuite.AnyFunSuite
2424
import org.apache.spark.SparkConf
2525
import org.apache.spark.serializer.JavaSerializer
2626
import org.apache.spark.sql.catalyst.InternalRow
27-
import org.apache.spark.sql.catalyst.expressions.{Add, AddMonths, BoundReference, Coalesce, Concat, CreateArray, CreateMap, DateFormatClass, ElementAt, Expression, GetStructField, LeafExpression, Length, Literal, MakeTimestamp, MicrosToTimestamp, MillisToTimestamp, MonthsBetween, Nondeterministic, Rand, Size, ToUnixTimestamp, Unevaluable, UnixMicros, UnixMillis, UnixSeconds, Upper}
27+
import org.apache.spark.sql.catalyst.expressions.{Add, AddMonths, BoundReference, Cast, Coalesce, Concat, CreateArray, CreateMap, DateFormatClass, ElementAt, Expression, GetStructField, LeafExpression, Length, Literal, MakeTimestamp, MicrosToTimestamp, MillisToTimestamp, MonthsBetween, Nondeterministic, Rand, Size, ToUnixTimestamp, Unevaluable, UnixMicros, UnixMillis, UnixSeconds, Upper}
2828
import org.apache.spark.sql.catalyst.expressions.codegen.{CodeFormatter, CodegenContext, CodegenFallback, ExprCode}
2929
import org.apache.spark.sql.types._
3030
import org.apache.spark.unsafe.types.UTF8String
@@ -41,9 +41,10 @@ import org.apache.comet.udf.codegen.CometScalaUDFCodegen
4141
* assert on the emitted Java directly, without invoking Janino. The goal is to catch regressions
4242
* in the optimizations we claim the dispatcher applies:
4343
*
44-
* - `NullIntolerant` short-circuit over a single input ordinal wraps `ev.code` in `if
45-
* (input-null) { setNull } else { ev.code; write }`, and is *not* emitted for trees that read
46-
* more than one ordinal (see the multi-input test for why).
44+
* - `NullIntolerant` short-circuit wraps `ev.code` in `if (input-null) { setNull } else {
45+
* ev.code; write }`. It is emitted for a tree reading a single ordinal, and for a
46+
* multi-ordinal tree whose root has only leaf children; it is *not* emitted once a subtree
47+
* sits between the root and a leaf (see the multi-input tests for why).
4748
* - Non-nullable column declaration emits `return false;` from `isNullAt(ord)`, and a
4849
* `BoundReference.nullable=false` (Catalyst sets this from schema-declared nullability) makes
4950
* Spark's `doGenCode` skip emitting its own `row.isNullAt(ord)` probe entirely.
@@ -167,17 +168,41 @@ class CometCodegenSourceSuite extends AnyFunSuite {
167168
s"got:\n$src")
168169
}
169170

170-
test("NullIntolerant short-circuit skipped for a multi-input tree (#5218)") {
171+
test("NullIntolerant short-circuit skipped when a subtree sits between root and leaf (#5218)") {
171172
// Even when every node is NullIntolerant, a short-circuit on the *union* of input ordinals is
172-
// not equivalent to Spark. Spark's null handling is per-node and left-to-right:
173-
// `BinaryExpression.nullSafeCodeGen` emits the left child's code unconditionally, then tests
174-
// the left child's null, then the right child's. So for `Add(cast(c0 AS INT), c1)` on a row
175-
// where c0 = 'abc' and c1 IS NULL, Spark evaluates the cast first and (under ANSI) raises
176-
// CAST_INVALID_INPUT. A union short-circuit would skip the cast and return null, swallowing
177-
// the error. See CometCodegenSuite's add_months test for the end-to-end witness.
173+
// not equivalent to Spark once a real subtree sits between the root and a leaf. Spark's null
174+
// handling is per-node and left-to-right: `BinaryExpression.nullSafeCodeGen` emits the left
175+
// child's code unconditionally, then tests the left child's null, then the right child's. So
176+
// for `Add(cast(c0 AS INT), c1)` on a row where c0 = 'abc' and c1 IS NULL, Spark evaluates the
177+
// cast first and (under ANSI) raises CAST_INVALID_INPUT. A union short-circuit would skip the
178+
// cast and return null, swallowing the error. See CometCodegenSuite's add_months test for the
179+
// end-to-end witness.
178180
//
179-
// `Add` over two BoundReferences is the minimal shape: NullIntolerant root, NullIntolerant
180-
// (leaf-only) children, two distinct ordinals.
181+
// The `Cast` is what disqualifies the short-circuit here: it is a non-leaf child of the root
182+
// that can raise on its own. The sibling test below covers the leaf-only-children shape, which
183+
// keeps the short-circuit.
184+
val strCol = ArrowColumnSpec(varCharVectorClass, nullable = true)
185+
val intCol = ArrowColumnSpec(
186+
CometBatchKernelCodegen.vectorClassBySimpleName("IntVector"),
187+
nullable = true)
188+
val expr = Add(
189+
Cast(BoundReference(0, StringType, nullable = true), IntegerType, ansiEnabled = true),
190+
BoundReference(1, IntegerType, nullable = true))
191+
val src = gen(expr, strCol, intCol)
192+
assert(
193+
!src.contains("this.col0.isNull(i) || this.col1.isNullAt(i)"),
194+
s"expected no union-of-inputs short-circuit when a Cast sits under the root; got:\n$src")
195+
assert(
196+
!src.contains("if (this.col0.isNull(i))") && !src.contains("if (this.col1.isNullAt(i))"),
197+
s"expected no pre-eval input-null short-circuit at all for this shape; got:\n$src")
198+
}
199+
200+
test("NullIntolerant short-circuit kept for a multi-input leaf-only tree (#5218)") {
201+
// Counterpart to the test above. When every direct child of the root is a `BoundReference` or
202+
// `Literal`, the tree is one level deep: the only code Spark runs ahead of its own null checks
203+
// is leaf reads, which cannot raise. Any error comes from the root's own logic, which runs only
204+
// once every input is known non-null -- in Spark's shape and in ours alike. So the union
205+
// short-circuit is exact and worth keeping for the common no-cast multi-argument case.
181206
val intCol = ArrowColumnSpec(
182207
CometBatchKernelCodegen.vectorClassBySimpleName("IntVector"),
183208
nullable = true)
@@ -186,11 +211,8 @@ class CometCodegenSourceSuite extends AnyFunSuite {
186211
BoundReference(1, IntegerType, nullable = true))
187212
val src = gen(expr, intCol, intCol)
188213
assert(
189-
!src.contains("this.col0.isNullAt(i) || this.col1.isNullAt(i)"),
190-
s"expected no union-of-inputs short-circuit for a two-input tree; got:\n$src")
191-
assert(
192-
!src.contains("if (this.col0.isNullAt(i))") && !src.contains("if (this.col1.isNullAt(i))"),
193-
s"expected no pre-eval input-null short-circuit at all for a two-input tree; got:\n$src")
214+
src.contains("if (this.col0.isNullAt(i) || this.col1.isNullAt(i))"),
215+
s"expected union-of-inputs short-circuit for a leaf-only two-input tree; got:\n$src")
194216
}
195217

196218
test("canHandle accepts CodegenFallback expressions (delegates to eval(row))") {
@@ -423,14 +445,17 @@ class CometCodegenSourceSuite extends AnyFunSuite {
423445
CodeFormatter.format(result.code))
424446
}
425447

426-
test("nullable NullIntolerant root keeps post-eval isNull guard (#4554)") {
448+
test("nullable NullIntolerant root keeps post-eval isNull guard inside short-circuit (#4554)") {
427449
// `NullIntolerant` only constrains "null in -> null out". An expression can still set
428450
// `ev.isNull = true` from non-null inputs — `MakeTimestamp(failOnError = false)` does this in
429451
// its `doGenCode` catch block when year/month/day/hour/min/sec components are out of range
430452
// (issue #4554). The dispatcher previously assumed NullIntolerant + non-null inputs implied a
431453
// non-null output and dropped the post-eval guard; that wrote stale `ev.value` bytes for
432-
// every invalid row. A nullable root must always keep a post-eval `if (ev.isNull) setNull`
433-
// check, whether or not the input-null short-circuit also applies.
454+
// every invalid row. The short-circuit on input nulls must coexist with a post-eval
455+
// `if (ev.isNull) setNull` check whenever the expression itself is nullable.
456+
//
457+
// All six children are `BoundReference`s, so this is a leaf-only tree and keeps the
458+
// union-of-ordinals short-circuit (#5218).
434459
val expr = MakeTimestamp(
435460
BoundReference(0, IntegerType, nullable = true),
436461
BoundReference(1, IntegerType, nullable = true),
@@ -453,17 +478,55 @@ class CometCodegenSourceSuite extends AnyFunSuite {
453478
IndexedSeq(intCol, intCol, intCol, intCol, intCol, decCol))
454479
val src = result.body
455480
val formatted = CodeFormatter.format(result.code)
456-
// This tree reads six ordinals, so there is no input-null short-circuit (#5218). Exactly one
457-
// setNull site must remain: the post-eval guard that propagates the `ev.isNull = true` set by
458-
// MakeTimestamp's catch block on invalid components. Pre-#4554 there were none.
481+
// Two distinct setNull sites must exist: the input-null short-circuit before `ev.code` runs,
482+
// and the post-eval guard that propagates `ev.isNull = true` set by MakeTimestamp's catch
483+
// block on invalid components. Pre-#4554 there was only one (the input short-circuit).
484+
val setNullOccurrences = "output\\.setNull\\(i\\);".r.findAllIn(src).length
485+
assert(
486+
setNullOccurrences >= 2,
487+
"expected at least two setNull sites (input short-circuit + post-eval ev.isNull guard); " +
488+
s"found $setNullOccurrences. Source:\n$formatted")
489+
// The short-circuit must test every ordinal the tree reads, not just the first.
490+
assert(
491+
src.contains(
492+
"if (this.col0.isNullAt(i) || this.col1.isNullAt(i) || " +
493+
"this.col2.isNullAt(i) || this.col3.isNullAt(i) || this.col4.isNullAt(i) || " +
494+
"this.col5.isNull(i))"),
495+
s"expected the short-circuit to test all six ordinals; source:\n$formatted")
496+
}
497+
498+
test("multi-input tree with a non-leaf child keeps only the post-eval guard (#5218)") {
499+
// The counterpart to the test above: the same nullable-NullIntolerant-root shape, but with a
500+
// `Cast` between the root and one leaf. That disqualifies the short-circuit, so exactly one
501+
// setNull site remains -- the post-eval `ev.isNull` guard, which is the actual #4554
502+
// regression and must survive independently of the short-circuit.
503+
val expr = MakeTimestamp(
504+
Cast(BoundReference(0, StringType, nullable = true), IntegerType, ansiEnabled = true),
505+
BoundReference(1, IntegerType, nullable = true),
506+
BoundReference(2, IntegerType, nullable = true),
507+
Literal(0),
508+
Literal(0),
509+
Literal(Decimal(0, 8, 6), DecimalType(8, 6)),
510+
timezone = None,
511+
timeZoneId = Some("UTC"),
512+
failOnError = false)
513+
assert(expr.nullable, "MakeTimestamp(failOnError=false) must be nullable for this test")
514+
val strCol = ArrowColumnSpec(varCharVectorClass, nullable = true)
515+
val intCol = ArrowColumnSpec(
516+
CometBatchKernelCodegen.vectorClassBySimpleName("IntVector"),
517+
nullable = true)
518+
val result =
519+
CometBatchKernelCodegen.generateSource(expr, IndexedSeq(strCol, intCol, intCol))
520+
val src = result.body
521+
val formatted = CodeFormatter.format(result.code)
459522
val setNullOccurrences = "output\\.setNull\\(i\\);".r.findAllIn(src).length
460523
assert(
461524
setNullOccurrences == 1,
462-
"expected exactly one setNull site (the post-eval ev.isNull guard, with no multi-input " +
463-
s"short-circuit); found $setNullOccurrences. Source:\n$formatted")
525+
"expected exactly one setNull site (the post-eval ev.isNull guard, with no short-circuit); " +
526+
s"found $setNullOccurrences. Source:\n$formatted")
464527
assert(
465-
!src.contains("if (this.col0.isNullAt(i))"),
466-
s"expected no input-null short-circuit for a six-input tree; source:\n$formatted")
528+
!src.contains("if (this.col0.isNull(i))"),
529+
s"expected no input-null short-circuit when a Cast sits under the root; source:\n$formatted")
467530
}
468531

469532
test("single-input nullable NullIntolerant root emits both short-circuit and post-eval guard") {

0 commit comments

Comments
 (0)