Repository navigation
Commit c111f0a
authored
fix: coerce SIMILAR TO operands to a common string type (#23704)
## Which issue does this PR close?
- Closes #22886
## Rationale for this change
`SIMILAR TO` panics with `failed to downcast array` whenever its
operands end up as arrays of different physical types:
```sql
SELECT 'a' SIMILAR TO NULL; -- panics during planning (constant folding)
CREATE TABLE t AS SELECT * FROM (VALUES ('user auth failed')) v(s);
CREATE TABLE p AS SELECT * FROM (VALUES ('(auth|login)')) v(pat);
SELECT arrow_cast(t.s, 'Utf8View') SIMILAR TO p.pat FROM t CROSS JOIN p; -- panics at runtime
```
Root cause: `SIMILAR TO` is planned as a regex binary operator
(`RegexMatch` et al.), whose kernel downcasts both arrays to the left
operand's array type. But unlike `LIKE` and the regex operators (`~`,
`~*`, ...), the `TypeCoercion` analyzer never coerced `Expr::SimilarTo`
operands — it was listed in the "nothing to coerce" arm of
`TypeCoercionRewriter` — so any type mismatch reached the kernel
un-normalized and hit a blind `.expect()`. Literal patterns take a
scalar fast path, which is why the common `col SIMILAR TO 'pattern'`
form never showed this.
## What changes are included in this PR?
Two commits:
1. **Coerce `SIMILAR TO` operands in the analyzer**
(`datafusion/optimizer/src/analyzer/type_coercion.rs`)
- Extract the existing `LIKE`-operand coercion into
`coerce_like_operands` (behavior for `LIKE`/`ILIKE` is unchanged,
including the `Dictionary(_, Utf8)` special case and error texts).
- Apply `regex_coercion` to `Expr::SimilarTo` — the same coercion the
physical regex operators it is planned into use
(`expr-common/src/type_coercion/binary.rs:218`). A `NULL` pattern is
coerced to a typed NULL and evaluates to NULL (matching `expr ~ NULL`),
and mixed string types are unified before planning.
- Regression coverage in `type_coercion.slt` (both reproducers, NULL
variants, and the planning-error path) plus analyzer unit tests.
2. **Defense in depth in the regex kernels**
(`datafusion/physical-expr/src/expressions/binary/kernels.rs`)
- Replace `.expect("failed to downcast array")` in
`regexp_is_match_flag!` / `regexp_is_match_flag_scalar!` with
`exec_err!`, so any expression path that bypasses the analyzer (e.g. a
hand-built plan going straight to physical planning) returns an error
instead of panicking. Includes a unit test constructing a mismatched
`RegexMatch` directly.
## Are these changes tested?
Yes:
- sqllogictest: 7 new cases in `type_coercion.slt` (issue reproducers,
`NULL`/`NOT SIMILAR TO NULL`, `LargeUtf8` × `Utf8`, incompatible-type
planning error).
- Unit tests: analyzer coercion plans (NULL pattern, cross string types,
error case) and a kernel-level test proving mismatched arrays now error
instead of panic.
- Verified `./dev/rust_lint.sh`, `cargo test -p datafusion-optimizer
--lib`, `cargo test -p datafusion-physical-expr --lib`, and the relevant
sqllogictest files (`type_coercion`, `strings`, `scalar`, `regexp`,
`string`) — all green.
## Are there any user-facing changes?
Only the bug fix: queries that previously panicked now either evaluate
correctly (`Utf8View` × `Utf8` etc.) or return NULL / a planning error.
`LIKE`/`ILIKE` behavior is unchanged.1 parent 5b65e70 commit c111f0a
4 files changed
Lines changed: 294 additions & 34 deletions
File tree
- datafusion
- optimizer/src/analyzer
- physical-expr/src/expressions
- binary
- sqllogictest/test_files
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
43 | 43 | | |
44 | 44 | | |
45 | 45 | | |
46 | | - | |
| 46 | + | |
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
| |||
442 | 442 | | |
443 | 443 | | |
444 | 444 | | |
| 445 | + | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
| 458 | + | |
| 459 | + | |
| 460 | + | |
| 461 | + | |
| 462 | + | |
| 463 | + | |
| 464 | + | |
| 465 | + | |
| 466 | + | |
| 467 | + | |
| 468 | + | |
| 469 | + | |
| 470 | + | |
| 471 | + | |
| 472 | + | |
| 473 | + | |
| 474 | + | |
| 475 | + | |
| 476 | + | |
445 | 477 | | |
446 | 478 | | |
447 | 479 | | |
| |||
588 | 620 | | |
589 | 621 | | |
590 | 622 | | |
591 | | - | |
592 | | - | |
593 | | - | |
594 | | - | |
595 | | - | |
596 | | - | |
597 | | - | |
598 | | - | |
599 | | - | |
600 | | - | |
601 | | - | |
602 | | - | |
603 | | - | |
604 | | - | |
605 | | - | |
606 | | - | |
607 | | - | |
| 623 | + | |
| 624 | + | |
| 625 | + | |
| 626 | + | |
| 627 | + | |
| 628 | + | |
| 629 | + | |
| 630 | + | |
608 | 631 | | |
609 | 632 | | |
610 | 633 | | |
| |||
613 | 636 | | |
614 | 637 | | |
615 | 638 | | |
| 639 | + | |
| 640 | + | |
| 641 | + | |
| 642 | + | |
| 643 | + | |
| 644 | + | |
| 645 | + | |
| 646 | + | |
| 647 | + | |
| 648 | + | |
| 649 | + | |
| 650 | + | |
| 651 | + | |
| 652 | + | |
| 653 | + | |
| 654 | + | |
| 655 | + | |
| 656 | + | |
| 657 | + | |
| 658 | + | |
| 659 | + | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
| 663 | + | |
| 664 | + | |
616 | 665 | | |
617 | 666 | | |
618 | 667 | | |
| |||
812 | 861 | | |
813 | 862 | | |
814 | 863 | | |
815 | | - | |
816 | 864 | | |
817 | 865 | | |
818 | 866 | | |
| |||
2240 | 2288 | | |
2241 | 2289 | | |
2242 | 2290 | | |
| 2291 | + | |
| 2292 | + | |
| 2293 | + | |
| 2294 | + | |
| 2295 | + | |
| 2296 | + | |
| 2297 | + | |
| 2298 | + | |
| 2299 | + | |
| 2300 | + | |
| 2301 | + | |
| 2302 | + | |
| 2303 | + | |
| 2304 | + | |
| 2305 | + | |
| 2306 | + | |
| 2307 | + | |
| 2308 | + | |
| 2309 | + | |
| 2310 | + | |
| 2311 | + | |
| 2312 | + | |
| 2313 | + | |
| 2314 | + | |
| 2315 | + | |
| 2316 | + | |
| 2317 | + | |
| 2318 | + | |
| 2319 | + | |
| 2320 | + | |
| 2321 | + | |
| 2322 | + | |
| 2323 | + | |
| 2324 | + | |
| 2325 | + | |
| 2326 | + | |
| 2327 | + | |
| 2328 | + | |
| 2329 | + | |
| 2330 | + | |
| 2331 | + | |
| 2332 | + | |
| 2333 | + | |
| 2334 | + | |
| 2335 | + | |
| 2336 | + | |
| 2337 | + | |
| 2338 | + | |
| 2339 | + | |
| 2340 | + | |
| 2341 | + | |
| 2342 | + | |
| 2343 | + | |
| 2344 | + | |
| 2345 | + | |
| 2346 | + | |
| 2347 | + | |
| 2348 | + | |
| 2349 | + | |
| 2350 | + | |
| 2351 | + | |
| 2352 | + | |
| 2353 | + | |
| 2354 | + | |
| 2355 | + | |
| 2356 | + | |
| 2357 | + | |
| 2358 | + | |
| 2359 | + | |
| 2360 | + | |
| 2361 | + | |
| 2362 | + | |
| 2363 | + | |
| 2364 | + | |
| 2365 | + | |
| 2366 | + | |
| 2367 | + | |
| 2368 | + | |
| 2369 | + | |
| 2370 | + | |
| 2371 | + | |
| 2372 | + | |
| 2373 | + | |
| 2374 | + | |
| 2375 | + | |
| 2376 | + | |
| 2377 | + | |
| 2378 | + | |
| 2379 | + | |
| 2380 | + | |
| 2381 | + | |
| 2382 | + | |
| 2383 | + | |
| 2384 | + | |
| 2385 | + | |
| 2386 | + | |
| 2387 | + | |
| 2388 | + | |
| 2389 | + | |
| 2390 | + | |
| 2391 | + | |
| 2392 | + | |
| 2393 | + | |
| 2394 | + | |
| 2395 | + | |
| 2396 | + | |
| 2397 | + | |
2243 | 2398 | | |
2244 | 2399 | | |
2245 | 2400 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1279 | 1279 | | |
1280 | 1280 | | |
1281 | 1281 | | |
1282 | | - | |
| 1282 | + | |
1283 | 1283 | | |
1284 | 1284 | | |
1285 | 1285 | | |
| |||
3331 | 3331 | | |
3332 | 3332 | | |
3333 | 3333 | | |
| 3334 | + | |
| 3335 | + | |
| 3336 | + | |
| 3337 | + | |
| 3338 | + | |
| 3339 | + | |
| 3340 | + | |
| 3341 | + | |
| 3342 | + | |
| 3343 | + | |
| 3344 | + | |
| 3345 | + | |
| 3346 | + | |
| 3347 | + | |
| 3348 | + | |
| 3349 | + | |
| 3350 | + | |
| 3351 | + | |
| 3352 | + | |
| 3353 | + | |
| 3354 | + | |
| 3355 | + | |
| 3356 | + | |
| 3357 | + | |
| 3358 | + | |
| 3359 | + | |
| 3360 | + | |
3334 | 3361 | | |
3335 | 3362 | | |
3336 | 3363 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
30 | | - | |
| 30 | + | |
31 | 31 | | |
32 | 32 | | |
33 | 33 | | |
| |||
162 | 162 | | |
163 | 163 | | |
164 | 164 | | |
165 | | - | |
166 | | - | |
167 | | - | |
168 | | - | |
169 | | - | |
170 | | - | |
171 | | - | |
172 | | - | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
173 | 186 | | |
174 | 187 | | |
175 | 188 | | |
| |||
210 | 223 | | |
211 | 224 | | |
212 | 225 | | |
213 | | - | |
214 | | - | |
215 | | - | |
216 | | - | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
217 | 235 | | |
218 | 236 | | |
219 | 237 | | |
| |||
0 commit comments