Skip to content

[DV-10] [P2] Reject overflowing and malformed chunked responses in the loopback HTTP client #570

Closed

Description

@wsdt

Priority: P2
Area: Loopback HTTP
Repository / target branch: wavect/semaprax / wavect/v090
Reviewed commit: 5428b0805fa3cf35859c843a53e30c3252ef378c
Review date: 6 October 2026

Defect

The public harness loopback HttpClient parses chunk lengths as usize, then checks out.len() + size > max with unchecked addition. After one valid one-byte chunk, a peer can send a chunk-size line equal to usize::MAX. This tiny response panics in the normal Rust test/debug profile before the client returns its structured size/malformed-response diagnostic.

The same decoder accepts malformed framing independently of overflow: after a data chunk it blindly skips size + 2 bytes without verifying the required CRLF, and it treats a zero-size line as complete without requiring the final empty line/trailer terminator. It can therefore return a successful HTTP response for bytes that are not a complete valid chunked body.

This issue applies to crates/semaprax-harness/src/host/http.rs, the public loopback HTTP facility. The separate streaming transport in crates/semaprax-toolchain/src/harness_bridge/transport.rs has a different decoder and is not claimed to share this exact defect.

Executed evidence

Two real loopback TCP fixtures exercise the public HttpClient::request API with max_response_bytes = 64:

  1. The server sends 1\r\nx\r\n followed by a hexadecimal usize::MAX chunk size. A catch_unwind around the client catches attempt to add with overflow at host/http.rs:335. The entire received response is small; no large allocation or actual memory exhaustion is needed.
  2. The server sends body bytes 1\r\nxZZ0\r\n. ZZ replaces the data chunk's required CRLF and the final empty line is missing. The client nevertheless returns Ok(HttpResponse { status: 200, body: [120] }), i.e. body x.

The first observation is a debug/overflow-checking-profile panic; it is not presented as a reproduced release-build panic. The malformed-framing acceptance follows the same branch independently of arithmetic overflow. Both tests run the unchanged production implementation in an isolated copy.

Smallest useful change

Compare the parsed size to the remaining budget with checked/subtractive arithmetic before slicing or adding offsets. Validate the chunk data's trailing CRLF exactly. A zero chunk must be followed by a complete admitted trailer section and terminating empty line; EOF inside it must be refused. Bound trailer parsing and preserve the existing response-size cap, timeout, fixed approved endpoint, no-redirect policy and credential handling.

Keep the fix local to the existing decoder or a small established HTTP parsing primitive. Do not widen transport scope or introduce remote/TLS support as part of this correction.

Acceptance criteria

  • usize::MAX, near-overflow values and aggregate chunk-size overflow produce a structured refusal in debug and release profiles, never a panic.
  • Wrong/missing data CRLF, truncated chunk headers/data, missing final empty line and malformed trailers are refused.
  • Exact-cap valid bodies, multiple chunks, supported chunk extensions and complete valid trailers still decode correctly.
  • Tests exercise the public loopback API as well as any parser unit seam; do not only test a replacement parser detached from its caller.
  • A bounded malformed response cannot bypass the size check or be accepted as a successful complete response.

Existing issue boundary

#419 introduced the bounded HTTP facility. #70 concerns end-to-end deadlines in the separate language TCP provider, not this chunked-response decoder. #549 concerns newline-framed daemon input, not HTTP body framing. No retrieved issue targets this exact decoder or overflow/framing failure.

Verification handoff

Read AGENTS.md and docs/HARNESS-HOST-V1.md. Put regressions in the existing host network test module. Record the fixing commit, profile, commands and actual pass/fail results; preserve distinctions between debug panic evidence and malformed-body acceptance evidence.

Pinned source references

Activity

  1. wsdt commented on Oct 6, 2026

    @wsdt
    MemberAuthor

    Fixed in b5e3f37 (pushed to main). dechunk parses hex-only sizes with an overflow-free bound (size > max - out.len()), an unrepresentable size is SPX-HPC033 (never a panic), the data CRLF is checked exactly, and the last chunk requires a bounded trailer section plus the final empty line (SPX-HPC036). Evidence: host::tests::net chunk_sizes_that_overflow_are_refused_not_panics, malformed_chunk_framing_is_refused (7 bodies + oversized trailer), valid_chunked_bodies_extensions_trailers_and_exact_cap_still_decode over real loopback TCP.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions