Skip to content

[ci-scan] Test failure: ZstandardStreamUnitTests OutOfMemoryException on tvOS CoreCLR #132459

Closed

Description

@davidnguyen-tech

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1555007

Build error leg or test failing: tvos-arm64 Release AllSubsets_CoreCLR - System.IO.Compression.Tests

Error Details

[FAIL] System.IO.Compression.ZstandardStreamUnitTests.CompressionLevel_SizeInOrder(testFile: "UncompressedTestFiles/TestDocument.doc")
System.OutOfMemoryException : Insufficient memory to continue the execution of the program.
   at System.IO.Compression.ZstandardUtils.Throw(ZSTD_error error)
   at System.IO.Compression.ZstandardEncoder.CompressCore(ReadOnlySpan`1 source, Span`1 destination, Int32& bytesConsumed, Int32& bytesWritten, ZstdEndDirective endDirective)
   at System.IO.Compression.ZstandardStream.WriteCoreAsync(ReadOnlyMemory`1 buffer, CancellationToken cancellationToken, Boolean isFinalBlock, Boolean flush, Boolean throwOnActiveRwOp)

[FAIL] System.IO.Compression.ZstandardStreamUnitTests.TestCompressCtor
Assert.All() Failure: 3 out of 13 items in the collection did not pass.
[10]: Error: Insufficient memory to continue the execution of the program.
[11]: Error: Insufficient memory to continue the execution of the program.
[12]: Error: Insufficient memory to continue the execution of the program.

Error Message

{
  "ErrorMessage": [
    "[FAIL] System.IO.Compression.ZstandardStreamUnitTests",
    "Insufficient memory to continue the execution of the program."
  ],
  "ErrorPattern": "",
  "BuildRetry": true,
  "ExcludeConsoleLog": false
}

Failure details:

System.IO.Compression.Tests intermittently fails on physical tvOS CoreCLR when Zstandard tests use CompressionLevel.SmallestSize.

Four failures share the same native Zstd allocation failure:

  • ZstandardStreamUnitTests.CompressionLevel_SizeInOrder("UncompressedTestFiles/TestDocument.doc")
  • ZstandardStreamUnitTests.CompressionLevel_SizeInOrder("UncompressedTestFiles/TestDocument.pdf")
  • ZstandardStreamUnitTests.CompressionLevel_SizeInOrder("UncompressedTestFiles/sum")
  • ZstandardStreamUnitTests.TestCompressCtor

The first three fail on the SmallestSize operation. TestCompressCtor builds a zero-based factory list containing one CompressionMode.Compress factory followed by three factories for each compression level. The Zstandard overrides forward those factories to the concrete constructors. The failed items map to:

  • [10]: new ZstandardStream(stream, CompressionLevel.SmallestSize)
  • [11]: new ZstandardStream(stream, CompressionLevel.SmallestSize, leaveOpen: true)
  • [12]: new ZstandardStream(stream, CompressionLevel.SmallestSize, leaveOpen: false)

Affected leg:

  • tvos-arm64 Release AllSubsets_CoreCLR

Test results:

Console log:

Regression status:

Not established. The same tests passed in the two preceding daily builds on the same AppleTV5,3 device model:

The KBE signature was verified against the failing Helix console. Its allocation-error line is absent from both passing consoles.

Configuration:

  • .NET: 11.0.0-preview.6.26359.118
  • Repository commit: a36e3c1c9a993f0b4c6db87f033105f0bb364f3b
  • Runtime: CoreCLR composite ReadyToRun
  • Target: physical tvOS 26.2 (23K54), arm64
  • Device: AppleTV5,3 / J42dAP
  • Device memory: 1.97 GB
  • Helix host: DNCENGMAC108, macOS 15.7

Analysis:

CompressionLevel.SmallestSize maps to Zstd's maximum quality:

internal static int GetQualityFromCompressionLevel(CompressionLevel compressionLevel) =>
compressionLevel switch
{
// zstd supports negative quality levels, all negative levels map to the
// same behavior (essentially no compression). Quality 0 means "default" = 3.
// 1 is therefore the fastest compression level with some compression.
CompressionLevel.NoCompression => Quality_Min,
CompressionLevel.Fastest => 1,
CompressionLevel.Optimal => Quality_Default,
CompressionLevel.SmallestSize => Quality_Max,
_ => throw new ArgumentOutOfRangeException(nameof(compressionLevel), compressionLevel, SR.ArgumentOutOfRange_Enum)

The test already replaces SmallestSize with Optimal for 32-bit processes because maximum-quality Zstandard compression needs 800+ MB of working memory and can fail intermittently:

public override Stream CreateStream(Stream stream, CompressionLevel level)
{
if (PlatformDetection.Is32BitProcess && level == CompressionLevel.SmallestSize)
{
// Zstandard smallest size requires too much working memory
// (800+ MB) and causes intermittent allocation errors on 32-bit
// processes in CI.
level = CompressionLevel.Optimal;
}
return new ZstandardStream(stream, level);
}
public override Stream CreateStream(Stream stream, CompressionLevel level, bool leaveOpen)
{
if (PlatformDetection.Is32BitProcess && level == CompressionLevel.SmallestSize)
{
// Zstandard smallest size requires too much working memory
// (800+ MB) and causes intermittent allocation errors on 32-bit
// processes in CI.
level = CompressionLevel.Optimal;
}
return new ZstandardStream(stream, level, leaveOpen);

The evidence is consistent with an intermittent native allocation limit or memory-pressure condition on a constrained physical device. It does not yet prove a product correctness regression.

BuildRetry is set to true as a temporary mitigation. Build Analysis may retry the whole build once when this signature matches a first-attempt failure. The two preceding equivalent runs passed, so a retry has a reasonable chance of clearing transient device memory pressure while the test's memory assumption is investigated.

The cpu_resource report and mlaunch exit 137 are not the root failure:

  • The resource report says Action taken: none.
  • Both preceding passing work items also produced a cpu_resource report.
  • Both preceding passing work items also logged Process mlaunch exited with 137 after XHarness detected the test-end marker.

Recommended action:

Determine whether maximum-quality compression is expected to succeed reliably within the supported tvOS device memory budget.

If the allocation failure is an expected platform constraint, extend the test's constrained-memory handling to physical tvOS or isolate the maximum-quality coverage so it does not fail unrelated compression coverage.

Note

This issue was prepared with GitHub Copilot assistance and reviewed by the submitting developer.

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=1555007
Error message validated: [[FAIL] System.IO.Compression.ZstandardStreamUnitTests Insufficient memory to continue the execution of the program.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 8/18/2026 11:52:22 AM UTC

Report

Build Repository Test Pull Request
1555007 dotnet/runtime System.IO.Compression.Tests.WorkItemExecution

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 0 1

Activity

  1. dotnet-policy-service commented on Aug 18, 2026

    @dotnet-policy-service
    Contributor

    Tagging subscribers to 'os-tvos': @vitek-karas, @kotlarmilos, @steveisok, @akoeplinger
    See info in area-owners.md if you want to be subscribed.

  2. dotnet-policy-service commented on Aug 18, 2026

    @dotnet-policy-service
    Contributor

    Tagging subscribers to this area: @karelz, @dotnet/area-system-io-compression
    See info in area-owners.md if you want to be subscribed.

  3. davidnguyen-tech commented on Aug 18, 2026

    @davidnguyen-tech
    MemberAuthor
  4. added this to the Future milestone on Aug 21, 2026
  5. added a commit that references this issue on Aug 27, 2026
    9632f7f
  6. locked and limited conversation to collaborators on Sep 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions