Repository navigation
Commit 41fc083
committed
fix: keep collated array_join and array_intersect native via codegen dispatch
Address review feedback on #4713: report Incompatible instead of
Unsupported for non-default collations so both route through the JVM
codegen dispatcher (Spark's own doGenCode) and stay native while matching
Spark, reconciling the three #2190 collation cases with CometReverse.
array_join concatenation is collation-independent, so only the output
string's collation metadata is dropped. array_intersect membership is
collation-aware, so the dispatcher (not the byte-based native path)
produces the correct result. Add collation SQL tests for both.1 parent 0571632 commit 41fc083
3 files changed
Lines changed: 57 additions & 20 deletions
File tree
- spark/src
- main/scala/org/apache/comet/serde
- test/resources/sql-tests/expressions/array
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
193 | 193 | | |
194 | 194 | | |
195 | 195 | | |
196 | | - | |
197 | | - | |
198 | | - | |
199 | | - | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
200 | 199 | | |
201 | | - | |
| 200 | + | |
202 | 201 | | |
203 | 202 | | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
204 | 208 | | |
205 | | - | |
| 209 | + | |
206 | 210 | | |
207 | 211 | | |
208 | 212 | | |
| |||
393 | 397 | | |
394 | 398 | | |
395 | 399 | | |
396 | | - | |
397 | | - | |
| 400 | + | |
| 401 | + | |
398 | 402 | | |
399 | 403 | | |
400 | | - | |
401 | | - | |
402 | | - | |
| 404 | + | |
403 | 405 | | |
404 | 406 | | |
405 | | - | |
406 | | - | |
407 | | - | |
| 407 | + | |
| 408 | + | |
| 409 | + | |
| 410 | + | |
| 411 | + | |
408 | 412 | | |
409 | | - | |
| 413 | + | |
410 | 414 | | |
411 | 415 | | |
412 | 416 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
20 | | - | |
21 | | - | |
22 | | - | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
23 | 24 | | |
24 | 25 | | |
25 | | - | |
| 26 | + | |
26 | 27 | | |
27 | 28 | | |
28 | 29 | | |
29 | | - | |
| 30 | + | |
30 | 31 | | |
0 commit comments