array_contains returns null instead of false for empty array with literal value

Description

array_contains(arr, 2) returns null instead of false when arr is an empty array (array()).

How to Reproduce

CREATE TABLE test(arr array<int>) USING parquet;
INSERT INTO test VALUES (array());
SELECT array_contains(arr, 2) FROM test;

Expected: false
Actual (Comet): null

Expected Behavior

Comet should return false for array_contains when the array is empty, matching Spark's behavior.

Add literal argument tests for all SQL file expression tests

Looks like it is fixed in df52

  test("native reader - array_contains") {
    withSQLConf(
      CometConf.COMET_EXEC_ENABLED.key -> "true",
      SQLConf.USE_V1_SOURCE_LIST.key -> "parquet",
      CometConf.COMET_ENABLED.key -> "true",
      CometConf.COMET_EXPLAIN_FALLBACK_ENABLED.key -> "true",
      CometConf.COMET_NATIVE_SCAN_IMPL.key -> "native_datafusion") {
      testSingleLineQuery(
        """
          |select cast(array() as array<int>) as a
          |""".stripMargin,
        "select array_contains(a, 2) from tbl")
    }
  }

Depends on #3536

empty array is fixed in DF52

another problem with

> select array_has([1, null, 3], 2);
+--------------------------------------------------------+
| array_has(make_array(Int64(1),NULL,Int64(3)),Int64(2)) |
+--------------------------------------------------------+
| false                                                  |
+--------------------------------------------------------+

Spark returns NULL in this case and this behavior is being explained in https://issues.apache.org/jira/browse/SPARK-55749

So DataFusion needs to have Spark compatible array_contains function

Depends on apache/datafusion#20611

I'd like to work on this.

Depends on #3574

fix: mark expressions with known correctness issues as incompatible

chore: Mark expressions with known correctness issues as incompatible (…

This expression (ArrayContains) is now marked as Incompatible in Comet, which means it falls back to Spark by default and only runs natively when spark.comet.expr.allowIncompatible=true is explicitly set. Since default behavior is now correct (via Spark fallback), this is no longer a bug. Relabeling as an enhancement to track implementing compatible native behavior in the future.

Navigation Menu

Repository navigation

array_contains returns null instead of false for empty array with literal value #3346

Description

How to Reproduce

Expected Behavior

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

array_contains returns null instead of false for empty array with literal value #3346

Description

Description

How to Reproduce

Expected Behavior

Activity

comphead commented on Feb 25, 2026

comphead commented on Feb 25, 2026

comphead commented on Feb 27, 2026

comphead commented on Feb 28, 2026

n0r0shi commented on Feb 28, 2026

comphead commented on Mar 6, 2026

andygrove commented on Mar 27, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions