Rewind trailing 0x1F byte in GZipStream on seekable leaveOpen streams by rzikm · Pull Request #132979 · dotnet/runtime

rzikm · 2026-08-31T18:06:32Z

Why

When you decompress a gzip member from a seekable base stream opened with leaveOpen: true, and a single stray 0x1F byte (gzip's ID1 magic byte) follows the member, GZipStream consumed that byte but never rewound it. The base stream was left positioned one byte past the member, so the trailing 0x1F was silently lost to whoever read the base stream next.

Root cause

After a member finishes, Inflater.ResetStreamForLeftoverInput() tries to detect a concatenated member by peeking at the leftover input for the 2-byte magic 1F 8B. It cannot verify that magic from a single leftover byte, so with exactly one leftover 0x1F it optimistically resets zlib. The next Inflate() then consumes the 0x1F as the start of a would-be header, GetAvailableInput() drops to 0, and TryRewindStream has nothing left to rewind.

Approach

Inflater now records an "unconfirmed lone-0x1F probe" — but only when exactly one leftover 0x1F byte triggered the optimistic reset. The probe never fires for a genuine multi-byte concatenation (1F 8B ...), and it is cleared as soon as new input arrives or a real concatenation header is seen, so it can never coexist with actual decompressed output.

When the base stream then hits EOF on a seekable stream, DeflateStream recognizes the pending probe, rewinds the byte, and marks the inflater terminally finished. InflateVerified short-circuits once end-of-stream is confirmed so subsequent reads don't re-advance the base stream (avoiding a re-read that would undo the rewind). This is wired at all four decompression sites: sync ReadCore, async Core, and sync/async CopyFromSourceToDestination.

Notable points for review

Intentional behavioral change under strict validation (the default-off internal s_useStrictValidation AppContext switch): a trailing 0x1F on a seekable stream now rewinds instead of throwing InvalidDataException, making it consistent with the existing trailing-0xFF behavior. The non-seekable strict path is unchanged and still throws.
Out of scope: the "separate-chunk" variant, where the finished member and the lone 0x1F arrive in separate Read/Write chunks, does not set the probe and is not addressed here (it requires a base stream that fragments reads exactly at the member boundary; the reported repro is same-chunk).

Tests

Adds a regression [Theory] TrailingByteAfterGzipMember_IsRewound covering footers 0x00/0x1F/0x8B/0xFF across Read/ReadByte/ReadAsync/Copy/CopyAsync (16 cases). Verified the test fails on the 0x1F cases without the fix and passes with it; the full System.IO.Compression.Tests suite passes (2628 tests, 0 failures).

Fixes: #132946

Note

This pull request was authored with GitHub Copilot.

…n streams When decompressing a gzip member from a seekable stream opened with leaveOpen: true, a lone trailing 0x1F byte (gzip ID1) following the member was consumed but never rewound, leaving the base stream positioned one byte past the member and losing the trailing byte. The Inflater cannot verify gzip's 2-byte magic (1F 8B) from a single leftover byte, so ResetStreamForLeftoverInput optimistically resets zlib and the next Inflate consumes the 0x1F as a would-be header, making GetAvailableInput report 0 so TryRewindStream rewinds nothing. Inflater now records an "unconfirmed lone-0x1F probe" only when exactly one leftover 0x1F triggers the reset. On base-stream EOF with a seekable stream, DeflateStream recognizes the probe, rewinds the byte, and marks the inflater terminally finished (InflateVerified short-circuits so subsequent reads do not re-advance the base stream). The probe is cleared when new input arrives or a multi-byte concatenation header is seen, so it never coexists with real output. Wired at all four decompression sites (sync ReadCore, async Core, sync/async CopyFromSourceToDestination). Adds regression theory TrailingByteAfterGzipMember_IsRewound covering 0x00/0x1F/0x8B/0xFF footers across Read/ReadByte/ReadAsync/Copy/CopyAsync. Fixes: dotnet#132946 Co-authored-by: Copilot App <[email protected]> Copilot-Session: edec8cdb-a9b6-4719-9e28-2d95bce20fde

azure-pipelines · 2026-08-31T18:07:16Z

Azure Pipelines: Successfully started running 3 pipeline(s). 13 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run.

dotnet-policy-service · 2026-08-31T18:08:04Z

Tagging subscribers to this area: @karelz, @dotnet/area-system-io-compression
See info in area-owners.md if you want to be subscribed.

Copilot

🟡 Changes recommended

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR adjusts System.IO.Compression’s gzip concatenation handling so that when a gzip member ends on a seekable base stream (leaveOpen: true) and the next byte is a single stray 0x1F (gzip ID1), that byte can be treated as trailing data and the base stream is rewound to preserve it for subsequent consumers.

Changes:

Track a special “unconfirmed lone-0x1F concatenation probe” in Inflater and expose it to DeflateStream.
Update DeflateStream read/copy paths to (on seekable streams) mark the inflater as terminally finished and rewind the probed byte at EOF, preventing re-consumption on subsequent reads.
Add a regression [Theory] covering multiple footers and read/copy sync/async scenarios.

File summaries

File	Description
src/libraries/System.IO.Compression/tests/CompressionStreamUnitTests.Gzip.cs	Adds regression coverage ensuring a single trailing footer byte (including `0x1F`) is preserved via stream rewind across multiple consumption patterns.
src/libraries/System.IO.Compression/src/System/IO/Compression/DeflateZLib/Inflater.cs	Records and clears a lone-`0x1F` “probe” state and supports a terminal end-of-stream mode to avoid re-consuming the rewound byte.
src/libraries/System.IO.Compression/src/System/IO/Compression/DeflateZLib/DeflateStream.cs	Integrates probe/end-of-stream handling into read + copy-to code paths and adjusts rewind logic to account for the probed byte.

Review details

Suppressed comments (1)

src/libraries/System.IO.Compression/src/System/IO/Compression/DeflateZLib/DeflateStream.cs:1036

Same as the async CopyTo path: this strict-validation check currently skips throwing when HasUnconfirmedGZipProbe is true, regardless of whether the underlying stream is seekable. To keep non-seekable strict-validation behavior unchanged, only suppress the truncated-data exception when _deflateStream._stream.CanSeek (i.e., when the probe can be rewound).

                    if (s_useStrictValidation && !_deflateStream._inflater.Finished() && !_deflateStream._inflater.HasUnconfirmedGZipProbe)
                    {
                        ThrowTruncatedInvalidData();
                    }

Files reviewed: 3/3 changed files
Comments generated: 1
Review effort level: Lite

The lone-0x1F trailing-byte probe can only be rewound on a seekable base stream. In the sync/async CopyFromSourceToDestination paths the strict truncated-data check was suppressed whenever HasUnconfirmedGZipProbe was set, regardless of seekability, which silently accepted truncated input on non-seekable streams under strict validation. Gate the suppression on _stream.CanSeek so non-seekable strict validation still throws, matching the ReadCore/Core paths. Co-authored-by: Copilot App <[email protected]> Copilot-Session: edec8cdb-a9b6-4719-9e28-2d95bce20fde

Copilot

🟡 Changes recommended

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Files reviewed: 3/3 changed files
Comments generated: 1
Review effort level: Lite

Covers the intentional behavior change where a trailing lone GZip ID1 (0x1F) on a seekable stream is rewound rather than throwing under strict validation, while a non-seekable stream still throws. Uses RemoteExecutor so the static readonly UseStrictValidation switch is set before type load and does not contaminate other tests. Co-authored-by: Copilot App <[email protected]> Copilot-Session: edec8cdb-a9b6-4719-9e28-2d95bce20fde

Copilot

🟡 Changes recommended

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

src/libraries/System.IO.Compression/src/System/IO/Compression/DeflateZLib/DeflateStream.cs:1041

Same issue as the async path: CopyFromSourceToDestination can hang if CopyTo is called again after finishing and rewinding due to the lone-0x1F probe. The second call will re-copy the rewound 0x1F into the inflater and Write will spin because _endOfStream forces Inflate* to return 0 without consuming input while NeedsInput stays false. Add an early no-op return when decompression has already completed/rewound (or EndOfStreamReached is set).

                    // Now, use the source stream's CopyToAsync to push directly to our inflater via this helper stream
                    _deflateStream._stream.CopyTo(this, _arrayPoolBuffer.Length);
                    // A lone trailing GZip ID1 (0x1F) probe only suppresses the truncated-data error when the
                    // base stream is seekable, since only then can the byte be rewound below. On non-seekable
                    // streams strict validation still throws, preserving the existing behavior.
                    if (s_useStrictValidation && !_deflateStream._inflater.Finished() &&
                        !(_deflateStream._inflater.HasUnconfirmedGZipProbe && _deflateStream._stream.CanSeek))

Files reviewed: 3/3 changed files
Comments generated: 1
Review effort level: Lite

Once a lone trailing GZip ID1 (0x1F) probe is resolved as end-of-stream and the base stream rewound, the inflater is terminally finished. The copy paths (CopyFromSourceToDestination[Async]) would re-read the rewound byte on a subsequent CopyTo and spin forever, because the inflater returns 0 without consuming input (via _endOfStream) while NeedsInput stays false. Add an early no-op return when EndOfStreamReached is set. The Read paths were already safe because ReadCore's loop breaks on InflatorIsFinished. Adds a regression theory covering a second decompression pass across all read/copy scenarios. Co-authored-by: Copilot App <[email protected]> Copilot-Session: edec8cdb-a9b6-4719-9e28-2d95bce20fde

Copilot

🔵 Needs a closer look

Review details

Files reviewed: 3/3 changed files
Comments generated: 0 new
Review effort level: Lite

alinpahontu2912

lgtm

Copilot AI lite review requested due to automatic review settings August 31, 2026 18:06

github-actions Bot added the area-System.IO.Compression label Aug 31, 2026

Copilot started reviewing on behalf of rzikm August 31, 2026 18:07

View session

dotnet-policy-service Bot assigned rzikm Aug 31, 2026

Copilot AI reviewed Aug 31, 2026

View reviewed changes

Comment thread src/libraries/System.IO.Compression/src/System/IO/Compression/DeflateZLib/DeflateStream.cs Resolved

build-analysis Bot mentioned this pull request Aug 31, 2026

Android WebSocket failure #121518

Closed

Copilot AI review requested due to automatic review settings September 1, 2026 07:28

Copilot started reviewing on behalf of rzikm September 1, 2026 07:28

View session

Copilot AI reviewed Sep 1, 2026

View reviewed changes

Comment thread src/libraries/System.IO.Compression/tests/CompressionStreamUnitTests.Gzip.cs Resolved

Copilot AI review requested due to automatic review settings September 1, 2026 07:42

Copilot started reviewing on behalf of rzikm September 1, 2026 07:42

View session

Copilot AI reviewed Sep 1, 2026

View reviewed changes

Comment thread src/libraries/System.IO.Compression/src/System/IO/Compression/DeflateZLib/DeflateStream.cs Resolved

Copilot AI review requested due to automatic review settings September 1, 2026 07:59

Copilot started reviewing on behalf of rzikm September 1, 2026 08:00

View session

Copilot AI reviewed Sep 1, 2026

View reviewed changes

This was referenced Sep 1, 2026

GZipStream does not rewind a trailing 0x1F leftover on seekable leaveOpen streams #132946

Closed

Build break: swiftc crash (Abort trap: 6) building SwiftAbiStress native test component on osx-arm64 #133001

Closed

build-analysis Bot mentioned this pull request Sep 1, 2026

[Test Failure] System.Net.Security.Tests.SslStreamTlsResumeTests.DifferentHost_NoResume #132851

Closed

rzikm requested a review from a team September 10, 2026 15:39

alinpahontu2912 approved these changes Sep 11, 2026

View reviewed changes

rzikm merged commit 720ba8f into dotnet:main Sep 11, 2026
79 checks passed

dotnet-maestro Bot mentioned this pull request Sep 12, 2026

[main] Source code updates from dotnet/runtime dotnet/dotnet#9263

Merged

dotnet-milestone-bot Bot added this to the 12.0-preview1 milestone Sep 14, 2026

github-actions Bot locked and limited conversation to collaborators Oct 15, 2026

Navigation Menu

Repository navigation

Rewind trailing 0x1F byte in GZipStream on seekable leaveOpen streams - #132979

Rewind trailing 0x1F byte in GZipStream on seekable leaveOpen streams#132979
rzikm merged 4 commits into
dotnet:mainfrom
rzikm:rzikm/gzipstream-trailing-1f-rewind

rzikm commented Aug 31, 2026

Uh oh!

azure-pipelines Bot commented Aug 31, 2026

Uh oh!

dotnet-policy-service Bot commented Aug 31, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

alinpahontu2912 left a comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

rzikm commented Aug 31, 2026

Why

Root cause

Approach

Notable points for review

Tests

Uh oh!

azure-pipelines Bot commented Aug 31, 2026

Uh oh!

dotnet-policy-service Bot commented Aug 31, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (1)

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Suppressed comments (1)

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

alinpahontu2912 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants