Repository navigation
[C++] binary_slice with its default stop raises "Negative buffer resize" #51222
Description
Describe the bug
pyarrow.compute.binary_slice(arr, start) with stop left at its default raises
ArrowInvalid: Negative buffer resize once a value is longer than a few hundred bytes. Passing an
explicit large stop works, and utf8_slice_codeunits handles the same default correctly since
#36575; the binary kernel still has the overflow in its output-size arithmetic.
pyarrow 25.0.1, Python 3.13.9, macOS 26.6 (arm64), wheel from PyPI.
import pyarrow as pa, pyarrow.compute as pc
b = pa.array([b"abcdef", b"x" * 300], pa.binary())
print([len(x) for x in pc.binary_slice(b, 1, 1000).to_pylist()]) # explicit stop: fine
print([len(x) for x in pc.utf8_slice_codeunits(pa.array(["abcdef", "x" * 300]), 1).to_pylist()])
print(pc.binary_slice(b, 1).to_pylist()) # default stopOutput:
[5, 299]
[5, 299]
Traceback (most recent call last):
...
pyarrow.lib.ArrowInvalid: Negative buffer resize: -4
Expected behavior
binary_slice(b, 1) returns [b"bcdef", b"x" * 299], as utf8_slice_codeunits does for the string
equivalent.
Related: #34929 (umbrella for slice-option overflow), #36575 (the utf8 fix).
Component(s)
C++, Python
Activity
Upstream tracker: binary_slice default-stop overflow filed as apache/…
singhpratech commentedon Sep 8, 2026
The overflow is in
SliceBytesTransform::MaxCodeunits(scalar_string_ascii.cc on main):(opt.stop - opt.start + opt.step - 1) / opt.stepwith the defaultstopofnumeric_limits<int64_t>::max()wraps negative whenstart > 0, and that negative bound is whatbecomes "Negative buffer resize". #36575 fixed the same expression for
utf8_slice_codeunits;the byte-slice transform still has the original form.
Dricmoy commentedon Sep 16, 2026
take
Upstream tracker: apache/arrow#51222 taken by a contributor
apacheGH-51222: Address binary slice review feedback
UPSTREAM: apache/arrow-go#1306 merged 2026-10-13 (ArraySpan.SetSlice …
pyarrow 26.0.0: the matrix and the Python suites run on it (venv, CI …
apacheGH-51222: Clamp binary slice estimate bounds
apacheGH-51222: [C++] Avoid overflow in binary_slice output sizing
apacheGH-51222: Address binary slice review feedback
apacheGH-51222: Clamp binary slice estimate bounds
apacheGH-51222: [C++] Avoid overflow in binary_slice with extreme ste…