Skip to content

fix(ipc): Avoid panic on malformed compressed buffer prefix - #9802

Merged
alamb merged 1 commit into
apache:mainfrom
pchintar:compressed_buffer_prefix
Apr 25, 2026
Merged

alamb merged 1 commit into
apache:mainfrom
pchintar:compressed_buffer_prefix

Conversation

@pchintar

@pchintar pchintar commented Apr 24, 2026 •

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

The IPC format specifies that compressed buffers are encoded as:

[8 bytes uncompressed length] + compressed data

The current implementation assumes this invariant when reading the prefix during decompression.

However, in the reader path, buffers are constructed from metadata (offset, length) and passed to the decompression logic without validating that they contain at least the required 8-byte prefix.

In contrast, Parquet defensively validates input before reading fixed-size prefixes, e.g.:

while input_len >= PREFIX_LEN { ... }
if input_len < expected_compressed_size as usize { return Err(...) }

This ensures malformed or truncated input does not cause panics.

In IPC, however, the prefix is read without a prior length check, so malformed/truncated input (buffer < 8 bytes) can lead to a panic instead of returning an error.

What changes are included in this PR?

  • Add a bounds check in read_uncompressed_size to ensure buffer.len() >= 8
  • Return ArrowError::IpcError when the prefix is missing
  • Propagate the error in decompress_to_buffer

Are these changes tested?

Yes.

  • Added test_read_uncompressed_size_rejects_short_prefix

    • Verifies that a buffer shorter than 8 bytes returns an error instead of panicking
  • Existing compression tests remain unchanged and pass

Are there any user-facing changes?

No.

@github-actions github-actions Bot added the arrow Changes to the arrow crate label Apr 24, 2026

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @pchintar

@alamb
alamb merged commit 4e4a430 into apache:main Apr 25, 2026
28 checks passed
@alamb

alamb commented Apr 25, 2026

Copy link
Copy Markdown
Contributor

Thanks again @pchintar

Rich-T-kid pushed a commit to Rich-T-kid/arrow-rs that referenced this pull request Jun 2, 2026
)

# Which issue does this PR close?

- Closes apache#9801 .

# Rationale for this change

The IPC format specifies that compressed buffers are encoded as:

> `[8 bytes uncompressed length] + compressed data` 

The current implementation assumes this invariant when reading the
prefix during decompression.

However, in the reader path, buffers are constructed from metadata
(`offset`, `length`) and passed to the decompression logic without
validating that they contain at least the required 8-byte prefix.

In contrast, Parquet defensively validates input before reading
fixed-size prefixes, e.g.:

```rust
while input_len >= PREFIX_LEN { ... }
if input_len < expected_compressed_size as usize { return Err(...) }
```

This ensures malformed or truncated input does not cause panics.

In IPC, however, the prefix is read without a prior length check, so
malformed/truncated input (buffer < 8 bytes) can lead to a panic instead
of returning an error.

# What changes are included in this PR?

* Add a bounds check in `read_uncompressed_size` to ensure `buffer.len()
>= 8`
* Return `ArrowError::IpcError` when the prefix is missing
* Propagate the error in `decompress_to_buffer`

# Are these changes tested?

Yes.

* Added `test_read_uncompressed_size_rejects_short_prefix`

* Verifies that a buffer shorter than 8 bytes returns an error instead
of panicking
* Existing compression tests remain unchanged and pass

# Are there any user-facing changes?

No.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

arrow Changes to the arrow crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Avoid panic on malformed compressed buffer prefix in IPC

2 participants