Repository navigation
Commit 16b08db
authored
fix: preserve NULL semantics when simplifying col ~ '.*' (apache#24380)
## Which issue does this PR close?
- Closes apache#24379.
## Rationale for this change
`simplify_regex_expr` rewrites `col ~ '.*'` to `col IS NOT NULL`. For a
NULL input that returns `false`, but `NULL ~ '.*'` is `NULL` under
three-valued logic — so the rewrite produces wrong results in a
projection context:
```sql
SELECT s, s ~ '.*' FROM (VALUES (CAST(NULL AS VARCHAR)), ('x')) t(s);
-- NULL row currently returns `false`; it should be NULL
```
The `!~` (`RegexNotMatch`) branch of the same rule is already NULL-aware
(`col IS NULL AND NULL`); only the `~` branch dropped the NULL.
## What changes are included in this PR?
- Rewrite `col ~ '.*'` to `col IS NOT NULL OR NULL` — `true` for a
non-NULL string, `NULL` for a NULL input.
- In a WHERE filter both FALSE and NULL reject the row, so filter
*results* are unchanged; only the plan text and projection-context
values differ. Existing filter-plan expectations in `simplify_expr.slt`
and the `test_simplify_regex_special_cases` unit test are updated
accordingly, and a projection regression test is added.
## Are these changes tested?
Yes.
- New projection regression test in `simplify_expr.slt` asserting `col ~
'.*'` returns `true`/`true`/`NULL` for `'foo'`/`''`/`NULL`.
- Updated the two filter-context plan expectations (logical + physical)
that previously encoded the `IS NOT NULL` rewrite.
- `simplify_expr.slt`, the `regexp/*` SLTs, and the optimizer simplify
unit tests all pass.
## Are there any user-facing changes?
`col ~ '.*'` in a projection now returns `NULL` for a NULL input instead
of `false`, matching SQL semantics. No API changes.1 parent d3c47b4 commit 16b08db
3 files changed
Lines changed: 28 additions & 7 deletions
File tree
- datafusion
- optimizer/src/simplify_expressions
- sqllogictest/test_files
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
41 | | - | |
| 41 | + | |
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
| |||
75 | 75 | | |
76 | 76 | | |
77 | 77 | | |
78 | | - | |
79 | | - | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
80 | 83 | | |
81 | 84 | | |
82 | 85 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1026 | 1026 | | |
1027 | 1027 | | |
1028 | 1028 | | |
1029 | | - | |
| 1029 | + | |
1030 | 1030 | | |
1031 | 1031 | | |
1032 | 1032 | | |
1033 | 1033 | | |
1034 | 1034 | | |
1035 | 1035 | | |
1036 | 1036 | | |
1037 | | - | |
| 1037 | + | |
1038 | 1038 | | |
1039 | 1039 | | |
1040 | 1040 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | | - | |
| 37 | + | |
38 | 38 | | |
39 | 39 | | |
40 | | - | |
| 40 | + | |
41 | 41 | | |
42 | 42 | | |
43 | 43 | | |
| |||
50 | 50 | | |
51 | 51 | | |
52 | 52 | | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
53 | 71 | | |
54 | 72 | | |
55 | 73 | | |
| |||
0 commit comments