Repository navigation
Commit 7738f74
authored
fix: return error instead of capacity overflow panic in generate_series (apache#22323)
## Which issue does this PR close?
- Closes apache#22188.
## Rationale for this change
Captured in issue description
## What changes are included in this PR?
Bounds the element count at `isize:MAX / size_of::<i64>()` in
`generate_range_values` and returns a DataFusion execution error when
the limit is exceeded, so the function returns a normal DataFusion error
instead of panicking inside `Vec::reserve`.
Only the scalar UDF path path in
`datafusion/functions-nested/src/range.rs` is affected, the table-valued
generate_series already streams and is unchanged.
This PR just takes care of the panic and does not attempt a larger
refactor towards streaming or introducing a max-elements config value.
Happy to follow up on either.
## Are these changes tested?
Added sqllogictest to tests in
`datafusion/sqllogictest/test_files/array/array_range.slt` to cover the
issues presented in the issue.
## Are there any user-facing changes?
Queries that previously crashed the process with capacity overflow now
returns `Execution error: Range too large to materialize: would produce
{count} elements ({MAX_RANGE_ELEMENTS})`
Behavior for valid ranges is unchanged.1 parent 821260f commit 7738f74
2 files changed
Lines changed: 40 additions & 11 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
329 | 329 | | |
330 | 330 | | |
331 | 331 | | |
332 | | - | |
| 332 | + | |
333 | 333 | | |
334 | 334 | | |
335 | 335 | | |
| |||
538 | 538 | | |
539 | 539 | | |
540 | 540 | | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
| 555 | + | |
| 556 | + | |
541 | 557 | | |
542 | 558 | | |
543 | 559 | | |
| |||
546 | 562 | | |
547 | 563 | | |
548 | 564 | | |
549 | | - | |
| 565 | + | |
550 | 566 | | |
551 | | - | |
| 567 | + | |
552 | 568 | | |
553 | 569 | | |
554 | 570 | | |
| |||
558 | 574 | | |
559 | 575 | | |
560 | 576 | | |
561 | | - | |
| 577 | + | |
562 | 578 | | |
563 | | - | |
564 | | - | |
565 | | - | |
| 579 | + | |
| 580 | + | |
566 | 581 | | |
567 | 582 | | |
568 | 583 | | |
| |||
578 | 593 | | |
579 | 594 | | |
580 | 595 | | |
581 | | - | |
| 596 | + | |
582 | 597 | | |
583 | | - | |
584 | | - | |
585 | | - | |
| 598 | + | |
| 599 | + | |
586 | 600 | | |
587 | 601 | | |
588 | 602 | | |
| |||
592 | 606 | | |
593 | 607 | | |
594 | 608 | | |
| 609 | + | |
595 | 610 | | |
596 | 611 | | |
597 | 612 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
41 | 48 | | |
42 | 49 | | |
43 | 50 | | |
| |||
352 | 359 | | |
353 | 360 | | |
354 | 361 | | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
355 | 369 | | |
356 | 370 | | |
357 | 371 | | |
| |||
0 commit comments