Skip to content

bug: to_io_slice and chunks_vectored ignore logical length on shortened non-contiguous Buffer #8063

Closed

Description

@codeAnqiang-ma

Describe the bug

For a non-contiguous Buffer, to_io_slice() and the bytes::Buf::chunks_vectored() impl walk every underlying part to its physical end and ignore the logical size field. truncate / slice / split_to / split_off shrink size without dropping parts, so after any of them both methods hand back more bytes than remaining().

Two contracts are broken:

  1. The bytes::Buf::chunks_vectored contract. Its docs state that "the sum of the lengths of all the buffers written to dst will be less than or equal to Buf::remaining()".
  2. The type's own invariant. Inner::NonContiguous is documented as "the logic view of the buffer starts at parts[idx][offset] and spans size bytes across subsequent parts", and the sibling methods chunk() and Iterator::next both honor it with .min(*size). These two methods destructure with .. and drop size entirely.

to_io_slice is public and documented as "Convert buffer into a slice of [IoSlice] for vectored write", so a vectored write over a shortened view would emit the bytes that were supposed to be cut off.

Steps to Reproduce

use bytes::Buf;
use bytes::Bytes;
use opendal_core::Buffer;

// 6 physical bytes across two parts
let buf = Buffer::from(vec![Bytes::from("abc"), Bytes::from("def")]);
// logical view of the first 2 bytes
let view = buf.slice(0..2);

assert_eq!(view.remaining(), 2);
assert_eq!(view.chunk(), b"ab"); // correct

let total: usize = view.to_io_slice().iter().map(|s| s.len()).sum();
assert_eq!(total, 2); // fails: total is 6, and the bytes are "abcdef"

All four ways of shortening a view are affected. Measured on main (65b3e5d), same buffer ["abc", "def"]:

view remaining() chunk() Iterator to_io_slice() chunks_vectored()
full 6 3 6 6 6
slice(0..2) 2 2 2 6 6
slice(1..4) 3 2 3 5 5
split_to(2) 2 2 2 6 6
split_off(2) head 2 2 2 6 6

chunk() and Iterator are correct in every case; the two vectored methods over-report in every shortened case.

Expected Behavior

to_io_slice() and chunks_vectored() should expose exactly the logical view — the same bytes that chunk() and Iterator yield — so their total never exceeds remaining().

Additional Context

The same two methods were already corrected once for the sibling half of this problem: #4481 ("fix(core): Buffer offset misuse") fixed them to apply offset only to the first part. The size clamp was not added at that time, so the logical-length half of the invariant is still missing.

Impact, stated honestly: the only in-tree vectored write backend, compfs, obtains its bytes through Buffer::by_ref().collect() (the Iterator impl, which respects size), so it does not hit this path — there is no active data corruption in-tree today. This is a correctness defect at the public API and trait-contract level: to_io_slice is pub and documented for vectored writes, and chunks_vectored is a bytes::Buf trait impl, so downstream users and any future in-tree consumer that performs a vectored write over a shortened view would write the extra tail bytes.

I have a minimal fix ready that mirrors the existing chunk() clamp, plus regression tests covering slice, split_to and split_off. I will open the PR right away.

  • Yes, I would like to submit a PR.

Activity

  1. added
    bugSomething isn't working
    releases-note/fixThe PR fixes a bug or has a title that begins with "fix"
    on Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingreleases-note/fixThe PR fixes a bug or has a title that begins with "fix"

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions