Repository navigation
Commit 22d4159
committed
perf: keep CASE lazy for string-parsing casts, and give NullShortCircuit its own check
Counting a LEGACY or TRY string-to-integer cast as infallible also put CASE
and IF on their eager path, which parses every row for each branch that some
row chooses. With a selective predicate, that made IF about 8x slower and a
three-branch CASE about 3x slower.
Cast::is_infallible keeps its original set: casts that cannot fail and cost
little more than reading their input. Cast::cannot_fail adds the LEGACY and
TRY string-to-integer casts. case_when.rs exposes both checks. CASE uses
is_infallible, so it chooses the path it chose before. NullShortCircuit uses
cannot_fail, since it masks an argument only to keep it from failing on rows
that Spark skips.
The conditional bench gains IF and CASE shapes whose branches parse strings.1 parent dd6da42 commit 22d4159
4 files changed
Lines changed: 122 additions & 31 deletions
File tree
- native/spark-expr
- benches
- src
- conditional_funcs
- conversion_funcs
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
45 | 45 | | |
46 | 46 | | |
47 | 47 | | |
48 | | - | |
49 | | - | |
50 | | - | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
51 | 52 | | |
52 | 53 | | |
53 | 54 | | |
| |||
57 | 58 | | |
58 | 59 | | |
59 | 60 | | |
60 | | - | |
| 61 | + | |
61 | 62 | | |
62 | 63 | | |
63 | 64 | | |
| |||
84 | 85 | | |
85 | 86 | | |
86 | 87 | | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
87 | 93 | | |
88 | 94 | | |
89 | 95 | | |
| |||
92 | 98 | | |
93 | 99 | | |
94 | 100 | | |
| 101 | + | |
| 102 | + | |
95 | 103 | | |
96 | 104 | | |
97 | 105 | | |
| |||
105 | 113 | | |
106 | 114 | | |
107 | 115 | | |
| 116 | + | |
| 117 | + | |
108 | 118 | | |
109 | 119 | | |
110 | 120 | | |
| |||
309 | 319 | | |
310 | 320 | | |
311 | 321 | | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
312 | 354 | | |
313 | 355 | | |
314 | 356 | | |
315 | 357 | | |
316 | 358 | | |
317 | 359 | | |
318 | | - | |
| 360 | + | |
319 | 361 | | |
320 | 362 | | |
321 | 363 | | |
| |||
374 | 416 | | |
375 | 417 | | |
376 | 418 | | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
| 422 | + | |
| 423 | + | |
| 424 | + | |
| 425 | + | |
| 426 | + | |
377 | 427 | | |
378 | 428 | | |
379 | 429 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
385 | 385 | | |
386 | 386 | | |
387 | 387 | | |
388 | | - | |
| 388 | + | |
389 | 389 | | |
390 | | - | |
391 | | - | |
392 | | - | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
393 | 395 | | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
| 408 | + | |
394 | 409 | | |
395 | 410 | | |
396 | 411 | | |
| |||
399 | 414 | | |
400 | 415 | | |
401 | 416 | | |
402 | | - | |
| 417 | + | |
| 418 | + | |
| 419 | + | |
| 420 | + | |
| 421 | + | |
403 | 422 | | |
404 | 423 | | |
405 | 424 | | |
| |||
417 | 436 | | |
418 | 437 | | |
419 | 438 | | |
420 | | - | |
| 439 | + | |
421 | 440 | | |
422 | 441 | | |
423 | 442 | | |
| |||
1231 | 1250 | | |
1232 | 1251 | | |
1233 | 1252 | | |
1234 | | - | |
| 1253 | + | |
| 1254 | + | |
1235 | 1255 | | |
1236 | 1256 | | |
1237 | 1257 | | |
1238 | 1258 | | |
1239 | 1259 | | |
1240 | 1260 | | |
1241 | 1261 | | |
1242 | | - | |
1243 | | - | |
1244 | | - | |
1245 | | - | |
1246 | | - | |
1247 | | - | |
1248 | | - | |
1249 | | - | |
| 1262 | + | |
| 1263 | + | |
| 1264 | + | |
| 1265 | + | |
| 1266 | + | |
1250 | 1267 | | |
1251 | 1268 | | |
1252 | 1269 | | |
1253 | | - | |
| 1270 | + | |
1254 | 1271 | | |
1255 | 1272 | | |
1256 | 1273 | | |
| 1274 | + | |
1257 | 1275 | | |
1258 | 1276 | | |
1259 | 1277 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
18 | | - | |
| 18 | + | |
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| |||
66 | 66 | | |
67 | 67 | | |
68 | 68 | | |
69 | | - | |
| 69 | + | |
70 | 70 | | |
71 | 71 | | |
72 | 72 | | |
| |||
487 | 487 | | |
488 | 488 | | |
489 | 489 | | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
490 | 503 | | |
491 | 504 | | |
492 | 505 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
119 | 119 | | |
120 | 120 | | |
121 | 121 | | |
122 | | - | |
123 | | - | |
124 | | - | |
125 | | - | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
126 | 125 | | |
127 | 126 | | |
128 | 127 | | |
| |||
138 | 137 | | |
139 | 138 | | |
140 | 139 | | |
141 | | - | |
142 | | - | |
143 | | - | |
144 | | - | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
145 | 155 | | |
146 | 156 | | |
147 | 157 | | |
| |||
0 commit comments