Repository navigation
Validate string SerDe bounds before allocation - #783
Conversation
|
Thanks for this, @jaideeppyne — the fix and tests look good. We’re holding it briefly so we can merge it together with related work going into the 9.1.0 release, and we’ve marked it as a draft so it isn’t merged early. No changes are needed from you. We’ll follow up when it’s ready to merge. |
|
Note to Reviewers: Please go ahead and review this PR even though it is in Draft. See my note to the author above. |
|
@jaideeppyne Thank you again for this fix, which shipped in Apache DataSketches Java 9.1.0. The DataSketches PMC is preparing a security advisory that covers this issue, and we’d like to credit you in it as a finder. Would you like to be credited? If so, please tell us how your name should appear: your GitHub handle, your full name, or both, and optionally an affiliation. If you’d rather not be credited, that’s fine too. You’re welcome to reply here, or by email to [email protected]. We plan to publish the advisory now without your name, and we can add your name later when you reply. |
|
Thanks, Lee. Yes, please credit me as |
Problem
The UTF-8 and UTF-16 string SerDes allocate arrays from serialized counts and length prefixes before checking that the input segment contains those bytes. Invalid input can therefore throw
NegativeArraySizeExceptionorOutOfMemoryErrorinstead of the expected bounds exception.Change
Check the minimum prefix space before allocating the output array, and validate each string's payload before allocating its byte buffer. Valid serialized bytes and zero-item behavior are unchanged.
The new tests exercise both SerDes with impossible item counts, negative and oversized length prefixes, truncated prefixes, invalid offsets, and offset Unicode/empty-string round trips. They also verify that decoded strings remain independent of later input mutations.
Verification
Tests-first and full verification on JDK 25:
NegativeArraySizeExceptionandOutOfMemoryErrorpaths.mvn -B clean verify: 2,305 tests passed, no failures, errors, or skips.Source:
628ac813eb143c287ed9da5594e74f5222fc29be. No dependencies or serialization formats changed.Prepared with OpenAI Codex assistance, including implementation and tests.