Repository navigation
Commit 7cdea62
Report an empty dictionary id section explicitly, and clean up tests
- An empty dictionary id section now fails with "the page has no data, not even the bit width" instead of the generic "reading past the end of the encoded values".
- Run the test for runs of length 0 on multiple buffers too.
- In `ParquetIOSuite`, find the RLE section with `indexOfSlice` and list causes with `ExceptionUtils.getThrowableList`.
Co-authored-by: Isaac <[email protected]>1 parent bf06bd8 commit 7cdea62
3 files changed
Lines changed: 29 additions & 15 deletions
File tree
- sql/core/src
- main/java/org/apache/spark/sql/execution/datasources/parquet
- test/scala/org/apache/spark/sql/execution/datasources/parquet
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
62 | 62 | | |
63 | 63 | | |
64 | 64 | | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
65 | 68 | | |
66 | 69 | | |
67 | 70 | | |
| |||
96 | 99 | | |
97 | 100 | | |
98 | 101 | | |
| 102 | + | |
99 | 103 | | |
100 | 104 | | |
101 | 105 | | |
| |||
121 | 125 | | |
122 | 126 | | |
123 | 127 | | |
| 128 | + | |
124 | 129 | | |
125 | 130 | | |
126 | 131 | | |
| |||
1000 | 1005 | | |
1001 | 1006 | | |
1002 | 1007 | | |
| 1008 | + | |
| 1009 | + | |
| 1010 | + | |
| 1011 | + | |
1003 | 1012 | | |
1004 | 1013 | | |
1005 | 1014 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
28 | 28 | | |
29 | 29 | | |
30 | 30 | | |
| 31 | + | |
31 | 32 | | |
32 | 33 | | |
33 | 34 | | |
| |||
2509 | 2510 | | |
2510 | 2511 | | |
2511 | 2512 | | |
2512 | | - | |
2513 | | - | |
2514 | | - | |
2515 | | - | |
2516 | | - | |
| 2513 | + | |
| 2514 | + | |
| 2515 | + | |
2517 | 2516 | | |
2518 | 2517 | | |
2519 | 2518 | | |
| |||
2530 | 2529 | | |
2531 | 2530 | | |
2532 | 2531 | | |
2533 | | - | |
| 2532 | + | |
2534 | 2533 | | |
2535 | 2534 | | |
2536 | 2535 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
435 | 435 | | |
436 | 436 | | |
437 | 437 | | |
438 | | - | |
| 438 | + | |
| 439 | + | |
| 440 | + | |
439 | 441 | | |
440 | 442 | | |
441 | 443 | | |
442 | 444 | | |
443 | 445 | | |
444 | | - | |
445 | | - | |
446 | | - | |
447 | | - | |
448 | | - | |
449 | | - | |
450 | | - | |
451 | | - | |
| 446 | + | |
| 447 | + | |
| 448 | + | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
| 457 | + | |
452 | 458 | | |
453 | 459 | | |
454 | 460 | | |
| |||
0 commit comments