feat: Implement ANSI support for UnaryMinus #465
Description
What is the problem the feature request solves?
Comet does not support ANSI mode for UnaryMinus.
Create test data
val df = Seq(Int.MaxValue, Int.MinValue).toDF("a")
df.write.parquet("/tmp/int.parquet")
spark.read.parquet("/tmp/int.parquet").createTempView("t")
Test with ANSI mode disabled
Behavior is correct with ANSI mode disabled:
scala> spark.conf.set("spark.sql.ansi.enabled", false)
scala> spark.conf.set("spark.comet.enabled", false)
scala> spark.sql("select a, -a from t").show
+-----------+-----------+
| a| (- a)|
+-----------+-----------+
| 2147483647|-2147483647|
|-2147483648|-2147483648|
+-----------+-----------+
scala> spark.conf.set("spark.comet.enabled", true)
scala> spark.sql("select a, -a from t").show
24/05/23 13:55:00 WARN CometSparkSessionExtensions$CometExecRule: Comet cannot execute some parts of this plan natively because CollectLimit is not supported
+-----------+-----------+
| a| (- a)|
+-----------+-----------+
| 2147483647|-2147483647|
|-2147483648|-2147483648|
+-----------+-----------+
Test with ANSI mode enabled
With ANSI mode enabled, Spark throws an exception, but Comet does not.
spark.conf.set("spark.sql.ansi.enabled", true)
spark.conf.set("spark.comet.ansi.enabled", true)
scala> spark.conf.set("spark.comet.enabled", false)
scala> spark.sql("select a, -a from t").show
24/05/23 13:55:36 WARN CometSparkSessionExtensions$CometExecRule: Using Comet's experimental support for ANSI mode.
24/05/23 13:55:36 ERROR Executor: Exception in task 0.0 in stage 18.0 (TID 18)
org.apache.spark.SparkArithmeticException: [ARITHMETIC_OVERFLOW] integer overflow. If necessary set "spark.sql.ansi.enabled" to "false" to bypass this error.
scala> spark.conf.set("spark.comet.enabled", true)
scala> spark.sql("select a, -a from t").show
24/05/23 13:55:48 WARN CometSparkSessionExtensions$CometExecRule: Using Comet's experimental support for ANSI mode.
24/05/23 13:55:48 WARN CometSparkSessionExtensions$CometExecRule: Comet cannot execute some parts of this plan natively because CollectLimit is not supported
+-----------+-----------+
| a| (- a)|
+-----------+-----------+
| 2147483647|-2147483647|
|-2147483648|-2147483648|
+-----------+-----------+
Describe the potential solution
No response
Additional context
No response