Repository navigation
Commit a4e3909
### Rationale for this change
`MapArray` validation uses `MayHaveNulls()` to check that the map child and keys contain no nulls. When an all-valid validity bitmap is present with an unknown null count, `MayHaveNulls()` can report that the array may contain nulls even though all values are valid.
During a cast of a `MapArray`, this can cause validation to return an invalid status which is passed to `ARROW_CHECK_OK`, resulting in a process abort instead of a recoverable error.
### What changes are included in this PR?
- Use `GetNullCount()` instead of `MayHaveNulls()` when validating the `MapArray` child and keys.
- Add a regression test covering an all-valid validity bitmap with `kUnknownNullCount`.
### Are these changes tested?
Yes, with existing and additional unit tests.
### Are there any user-facing changes?
Only a bugfix.
### This PR contains a "Critical Fix".
This fixes a bug that can cause a process crash when casting a valid `MapArray` whose keys carry an all-valid validity bitmap with an unknown null count. This is easily reproduced in Python:
```python
import pyarrow as pa
m = pa.array([{"a": "1"}, {"b": "2"}], type=pa.map_(pa.string(), pa.string()))
m.filter(pa.array([False, True])).cast(pa.map_(pa.large_string(), pa.large_string()))
```
* GitHub Issue: #51029
Lead-authored-by: Anurag Raut <[email protected]>
Co-authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
1 parent a62870c commit a4e3909
2 files changed
Lines changed: 42 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1532 | 1532 | | |
1533 | 1533 | | |
1534 | 1534 | | |
| 1535 | + | |
| 1536 | + | |
| 1537 | + | |
| 1538 | + | |
| 1539 | + | |
| 1540 | + | |
| 1541 | + | |
| 1542 | + | |
| 1543 | + | |
| 1544 | + | |
| 1545 | + | |
| 1546 | + | |
| 1547 | + | |
| 1548 | + | |
| 1549 | + | |
| 1550 | + | |
| 1551 | + | |
| 1552 | + | |
| 1553 | + | |
| 1554 | + | |
| 1555 | + | |
| 1556 | + | |
| 1557 | + | |
| 1558 | + | |
| 1559 | + | |
| 1560 | + | |
| 1561 | + | |
| 1562 | + | |
| 1563 | + | |
| 1564 | + | |
| 1565 | + | |
| 1566 | + | |
| 1567 | + | |
| 1568 | + | |
| 1569 | + | |
| 1570 | + | |
| 1571 | + | |
1535 | 1572 | | |
1536 | 1573 | | |
1537 | 1574 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
115 | 115 | | |
116 | 116 | | |
117 | 117 | | |
118 | | - | |
| 118 | + | |
119 | 119 | | |
120 | 120 | | |
121 | 121 | | |
| |||
826 | 826 | | |
827 | 827 | | |
828 | 828 | | |
829 | | - | |
| 829 | + | |
830 | 830 | | |
831 | 831 | | |
832 | 832 | | |
| |||
835 | 835 | | |
836 | 836 | | |
837 | 837 | | |
838 | | - | |
| 838 | + | |
839 | 839 | | |
840 | 840 | | |
841 | 841 | | |
| |||
896 | 896 | | |
897 | 897 | | |
898 | 898 | | |
899 | | - | |
| 899 | + | |
900 | 900 | | |
901 | 901 | | |
902 | 902 | | |
903 | 903 | | |
904 | 904 | | |
905 | | - | |
| 905 | + | |
906 | 906 | | |
907 | 907 | | |
908 | 908 | | |
| |||
0 commit comments