Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-14338

Improve `SimplifyConditionals` rule to handle `null` in IF/CASEWHEN

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.0.0
    • Optimizer, SQL
    • None

    Description

      Currently, `SimplifyConditionals` handles `true` and `false` to optimize branches. This issue improves `SimplifyConditionals` to take advantage of `null` conditions for `if` and `CaseWhen` expressions, too.

      Before

      scala> sql("SELECT IF(null, 1, 0)").explain()
      == Physical Plan ==
      WholeStageCodegen
      :  +- Project [if (null) 1 else 0 AS (IF(CAST(NULL AS BOOLEAN), 1, 0))#4]
      :     +- INPUT
      +- Scan OneRowRelation[]
      scala> sql("select case when cast(null as boolean) then 1 else 2 end").explain()
      == Physical Plan ==
      WholeStageCodegen
      :  +- Project [CASE WHEN null THEN 1 ELSE 2 END AS CASE WHEN CAST(NULL AS BOOLEAN) THEN 1 ELSE 2 END#14]
      :     +- INPUT
      +- Scan OneRowRelation[]
      

      After

      scala> sql("SELECT IF(null, 1, 0)").explain()
      == Physical Plan ==
      WholeStageCodegen
      :  +- Project [0 AS (IF(CAST(NULL AS BOOLEAN), 1, 0))#4]
      :     +- INPUT
      +- Scan OneRowRelation[]
      scala> sql("select case when cast(null as boolean) then 1 else 2 end").explain()
      == Physical Plan ==
      WholeStageCodegen
      :  +- Project [2 AS CASE WHEN CAST(NULL AS BOOLEAN) THEN 1 ELSE 2 END#4]
      :     +- INPUT
      +- Scan OneRowRelation[]
      

      Hive

      hive> select if(null,1,2);
      OK
      2
      hive> select case when cast(null as boolean) then 1 else 2 end;
      OK
      2
      

      Attachments

        Activity

          People

            dongjoon Dongjoon Hyun
            dongjoon Dongjoon Hyun
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: