You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
round on float/double falls back to Spark, while bround already uses the codegen dispatcher #5576
round on a float or double falls the whole projection back to Spark:
case_: FloatType|DoubleType=>// ... Spark uses BigDecimal for rounding float/double ...Unsupported(Some("Comet does not support Spark's BigDecimal rounding"))
The reasoning in that comment is sound — Spark rounds floating point through BigDecimal, which goes via java.lang.Double.toString() and its rounding algorithm, and the native path cannot reproduce it. But "we can't do this natively" is precisely the case the dispatcher exists for: running Spark's own doGenCode inside the Comet pipeline gives the BigDecimal behavior byte for byte.
bround already does exactly this — CometBRound is a plain CometCodegenDispatch[BRound] (serde/math.scala:269) — so the two sibling functions behave completely differently today for no principled reason.
round(double_col, n) is everyday analytics SQL, which makes this probably the highest-traffic single fallback in the sweep. docs/source/user-guide/latest/expressions.md documents it as "Float/double inputs fall back".
Fix: mix CodegenDispatchFallback into CometRound. FloatType and DoubleType are both in CometBatchKernelCodegen.isSupportedDataType, so canHandle admits it.
Leave the negative-scale decimal arm alone for now — it is only reachable with spark.sql.legacy.allowNegativeScaleOfDecimal=true and is tracked as deferred on the EPIC.
Activity