You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 0a57b6a
Browse filesBrowse the repository at this point in the historyBrowse files
fix: decline Iceberg's decimal truncate instead of nulling early
Iceberg's `TruncateDecimal.invoke` returns a `Decimal` that can exceed the column's
declared precision, and Spark turns it into null only when the row is materialized.
An Arrow `Decimal128(precision, scale)` array has no encoding for that intermediate,
so the kernel nulled it during evaluation, which changes what an enclosing predicate
or hash sees. Report decimal inputs as `Unsupported` so they stay with Spark.
`CometStaticInvoke.getUnsupportedReasons()` now aggregates the per-function handlers'
notes, since `GenerateDocs` only asks the serde registered for the expression class
and the note would otherwise reach the Iceberg guide but not the compatibility page.
Also drops the claim that Comet and Spark agree on the sort key. `SortExec` orders
rows with `RowOrdering.create`, which evaluates the sort expression per comparison
rather than through an `UnsafeRowWriter`, so Spark's sort sees the oversized decimal.
That was wrong in `iceberg.md` and in the kernel comment.
0 commit comments