Repository navigation
Commit b90cb07
fix: preserve NULL semantics when simplifying col ~ '.*' (apache#24380)
## Which issue does this PR close?
- Closes apache#24379.
## Rationale for this change
`simplify_regex_expr` rewrites `col ~ '.*'` to `col IS NOT NULL`. For a
NULL input that returns `false`, but `NULL ~ '.*'` is `NULL` under
three-valued logic — so the rewrite produces wrong results in a
projection context:
```sql
SELECT s, s ~ '.*' FROM (VALUES (CAST(NULL AS VARCHAR)), ('x')) t(s);
-- NULL row currently returns `false`; it should be NULL
```
The `!~` (`RegexNotMatch`) branch of the same rule is already NULL-aware
(`col IS NULL AND NULL`); only the `~` branch dropped the NULL.
## What changes are included in this PR?
- Rewrite `col ~ '.*'` to `col IS NOT NULL OR NULL` — `true` for a
non-NULL string, `NULL` for a NULL input.
- In a WHERE filter both FALSE and NULL reject the row, so filter
*results* are unchanged; only the plan text and projection-context
values differ. Existing filter-plan expectations in `simplify_expr.slt`
and the `test_simplify_regex_special_cases` unit test are updated
accordingly, and a projection regression test is added.
## Are these changes tested?
Yes.
- New projection regression test in `simplify_expr.slt` asserting `col ~
'.*'` returns `true`/`true`/`NULL` for `'foo'`/`''`/`NULL`.
- Updated the two filter-context plan expectations (logical + physical)
that previously encoded the `IS NOT NULL` rewrite.
- `simplify_expr.slt`, the `regexp/*` SLTs, and the optimizer simplify
unit tests all pass.
## Are there any user-facing changes?
`col ~ '.*'` in a projection now returns `NULL` for a NULL input instead
of `false`, matching SQL semantics. No API changes.
(cherry picked from commit 16b08db)1 parent 7d9f743 commit b90cb07
3 files changed
Lines changed: 28 additions & 7 deletions
File tree
- datafusion
- optimizer/src/simplify_expressions
- sqllogictest/test_files
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
41 | | - | |
| 41 | + | |
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
| |||
75 | 75 | | |
76 | 76 | | |
77 | 77 | | |
78 | | - | |
79 | | - | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
80 | 83 | | |
81 | 84 | | |
82 | 85 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1013 | 1013 | | |
1014 | 1014 | | |
1015 | 1015 | | |
1016 | | - | |
| 1016 | + | |
1017 | 1017 | | |
1018 | 1018 | | |
1019 | 1019 | | |
1020 | 1020 | | |
1021 | 1021 | | |
1022 | 1022 | | |
1023 | 1023 | | |
1024 | | - | |
| 1024 | + | |
1025 | 1025 | | |
1026 | 1026 | | |
1027 | 1027 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | | - | |
| 37 | + | |
38 | 38 | | |
39 | 39 | | |
40 | | - | |
| 40 | + | |
41 | 41 | | |
42 | 42 | | |
43 | 43 | | |
| |||
50 | 50 | | |
51 | 51 | | |
52 | 52 | | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
53 | 71 | | |
54 | 72 | | |
55 | 73 | | |
| |||
0 commit comments