Repository navigation
Include NullEquality in join executor's explain output #17611
Description
Is your feature request related to a problem or challenge?
@comphead has come up with this great idea: #17319 (comment)
If we re-write IS NOT DISTINCT FROM into equality in hash join (see write-up in #17319 for details), it's better to mark them in the EXPLAIN output, otherwise it might freak out careful users.
Describe the solution you'd like
I think we don't have to change the output for default behavior (null does not equal to null), however if we modify that behavior from the above-mentioned optimization, it should be marked in the join executors. (Hash Join and Sort Merge Join supports this feature)
DataFusion CLI v49.0.2
> create table t1(v1 int);
0 row(s) fetched.
Elapsed 0.025 seconds.
> create table t2(v1 int);
0 row(s) fetched.
Elapsed 0.001 seconds.
> EXPLAIN SELECT *
FROM t1
JOIN t2 ON t1.v1 IS NOT DISTINCT FROM t2.v1;
+---------------+------------------------------------------------------------+
| plan_type | plan |
+---------------+------------------------------------------------------------+
| physical_plan | ┌───────────────────────────┐ |
| | │ CoalesceBatchesExec │ |
| | │ -------------------- │ |
| | │ target_batch_size: │ |
| | │ 8192 │ |
| | └─────────────┬─────────────┘ |
| | ┌─────────────┴─────────────┐ |
| | │ HashJoinExec │ |
| | │ -------------------- ├──────────────┐ |
| | │ on: (v1 = v1) │ │ |
| | └─────────────┬─────────────┘ │ |
| | ┌─────────────┴─────────────┐┌─────────────┴─────────────┐ |
| | │ DataSourceExec ││ DataSourceExec │ |
| | │ -------------------- ││ -------------------- │ |
| | │ bytes: 0 ││ bytes: 0 │ |
| | │ format: memory ││ format: memory │ |
| | │ rows: 0 ││ rows: 0 │ |
| | └───────────────────────────┘└───────────────────────────┘ |
| | |
+---------------+------------------------------------------------------------+
1 row(s) fetched.
Elapsed 0.032 seconds.
> set datafusion.explain.format = indent;
0 row(s) fetched.
Elapsed 0.000 seconds.
> EXPLAIN SELECT *
FROM t1
JOIN t2 ON t1.v1 IS NOT DISTINCT FROM t2.v1;
+---------------+----------------------------------------------------------------------+
| plan_type | plan |
+---------------+----------------------------------------------------------------------+
| logical_plan | Inner Join: t1.v1 = t2.v1 |
| | TableScan: t1 projection=[v1] |
| | TableScan: t2 projection=[v1] |
| physical_plan | CoalesceBatchesExec: target_batch_size=8192 |
| | HashJoinExec: mode=Partitioned, join_type=Inner, on=[(v1@0, v1@0)] |
| | DataSourceExec: partitions=1, partition_sizes=[0] |
| | DataSourceExec: partitions=1, partition_sizes=[0] |
| | |
+---------------+----------------------------------------------------------------------+
2 row(s) fetched.
Elapsed 0.002 seconds.We can add a new entry Null Equality: NULL equals NULL in the HashJoinExec node
Describe alternatives you've considered
No response
Additional context
No response
Activity
[-]Include 'NullEquality` in join executor's explain output[/-][+]Include `NullEquality` in join executor's explain output[/+]