Skip to content

Mac mac_unopt failing due to infra failure timeout #189748

Open

Description

@Renzo-Olivares

Help us understand the severity of this issue

  • breakage; unable to contribute or trigger builds will no workarounds
  • inconvenient workarounds exist, but significant effort required
  • needed for a Flutter team-wide priority (already agreed upon)
  • nice-to-have; none of the above

What do you need help with

Mac mac_unopt is failing due to an infra failure. 4 re-runs and still faling. Logs below and here.

This build timed out.
The build was cancelled: Step('process logs (2).gsutil upload logs 891cee207bcc79cb7657472c6e8c2beb1e49f3b6')

Task timed out.

host_debug_unopt_arm64:

FAILED TESTS (2/7618):
    2831 ms: /Volumes/Work/s/w/ir/cache/builder/engine/src/out/ci/host_debug_unopt_arm64/impeller_unittests Play/AiksTest.CanRenderTextFrameWithInvertedTransform/OpenGLESSDF (try #1)
    1322 ms: /Volumes/Work/s/w/ir/cache/builder/engine/src/out/ci/host_debug_unopt_arm64/impeller_unittests Play/MatrixFilterContentsTest.RenderCoverageMatchesGetCoverageSubpassScale/OpenGLESSDF (try #1)
SUMMARY:
  /Volumes/Work/s/w/ir/cache/builder/engine/src/out/ci/host_debug_unopt_arm64/impeller_unittests Play/AiksTest.CanRenderTextFrameWithInvertedTransform/OpenGLESSDF passed 1 / 2 times.
  /Volumes/Work/s/w/ir/cache/builder/engine/src/out/ci/host_debug_unopt_arm64/impeller_unittests Play/MatrixFilterContentsTest.RenderCoverageMatchesGetCoverageSubpassScale/OpenGLESSDF passed 1 / 2 times.


!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!


Failed Command:

python3 /Volumes/Work/s/w/ir/cache/builder/engine/src/flutter/third_party/gtest-parallel/gtest-parallel /Volumes/Work/s/w/ir/cache/builder/engine/src/out/ci/host_debug_unopt_arm64/impeller_unittests --repeat=2 --gtest_filter=-*OpenGLES -- --enable_vulkan_validation

Exit Code: 251

Looks like there is some related issues

Activity

  1. jason-simmons commented on Jul 20, 2026

    @jason-simmons
    Contributor

    impeller_unittests has been seeing flakes when running OpenGLESSDF test cases.

    Example from the log above:

    [1443/7618] Play/MatrixFilterContentsTest.RenderCoverageMatchesGetCoverageSubpassScale/OpenGLESSDF (1322 ms)
    [INFO:flutter/testing/test_timeout_listener.cc(75)] Test timeout of 300 seconds per test case will be enforced.
    �[0;33mNote: Google Test filter = Play/MatrixFilterContentsTest.RenderCoverageMatchesGetCoverageSubpassScale/OpenGLESSDF
    �[m�[0;32m[==========] �[mRunning 1 test from 1 test suite.
    �[0;32m[----------] �[mGlobal test environment set-up.
    �[0;32m[----------] �[m1 test from Play/MatrixFilterContentsTest
    �[0;32m[ RUN      ] �[mPlay/MatrixFilterContentsTest.RenderCoverageMatchesGetCoverageSubpassScale/OpenGLESSDF
    2026-07-20 14:23:23.055 impeller_unittests[4845:653070] Metal API Validation Enabled
    2026-07-20 14:23:23.055 impeller_unittests[4845:653070] Metal GPU Validation Enabled
    [1443/7618] Play/MatrixFilterContentsTest.RenderCoverageMatchesGetCoverageSubpassScale/OpenGLESSDF returned/aborted with exit code -5 (1322 ms)
    

    The gtest-parallel script reports a returned/aborted with exit code -5 (SIGTRAP) error from the test process.

    I've seen other logs where the test fails with a segfault stack trace from within ANGLE.
    (example: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac%20Production%20Engine%20Drone/1033846/overview)

    [2531/7656] Play/DlGoldenTest.TextBlurMaskFilterRespectCTM/OpenGLESSDF (374 ms)
    [INFO:flutter/testing/test_timeout_listener.cc(75)] Test timeout of 300 seconds per test case will be enforced.
    �[0;33mNote: Google Test filter = Play/DlGoldenTest.TextBlurMaskFilterRespectCTM/OpenGLESSDF
    �[m�[0;32m[==========] �[mRunning 1 test from 1 test suite.
    �[0;32m[----------] �[mGlobal test environment set-up.
    �[0;32m[----------] �[m1 test from Play/DlGoldenTest
    �[0;32m[ RUN      ] �[mPlay/DlGoldenTest.TextBlurMaskFilterRespectCTM/OpenGLESSDF
    2026-07-17 10:52:46.834 impeller_unittests[55996:342825] Metal API Validation Enabled
    2026-07-17 10:52:46.834 impeller_unittests[55996:342825] Metal GPU Validation Enabled
    [ERROR:flutter/fml/backtrace.cc(108)] Caught signal SIGSEGV during program execution.
    Frame 0: 0x121f84660 sh::VarField::operator==()
    Frame 1: 0x121f8454c std::__fl::equal_to<>::operator()[abi:nn210000]()
    Frame 2: 0x121f8b3bc std::__fl::__hash_table<>::find<>()
    Frame 3: 0x121f712f8 std::__fl::unordered_set<>::find[abi:nn210000]()
    Frame 4: 0x121f71200 sh::SymbolEnv::isUBO()
    Frame 5: 0x12201ebd8 (anonymous namespace)::GenMetalTraverser::emitFunctionParameter()
    Frame 6: 0x12201e724 (anonymous namespace)::GenMetalTraverser::emitFunctionSignature()
    Frame 7: 0x12201376c (anonymous namespace)::GenMetalTraverser::visitFunctionDefinition()
    Frame 8: 0x121e64490 sh::TIntermFunctionDefinition::visit()
    Frame 9: 0x121e66f78 sh::TIntermTraverser::traverseFunctionDefinition()
    Frame 10: 0x121e64144 sh::TIntermFunctionDefinition::traverse()
    Frame 11: 0x1220141d8 (anonymous namespace)::GenMetalTraverser::visitBlock()
    Frame 12: 0x121e64830 sh::TIntermBlock::visit()
    Frame 13: 0x121e67120 sh::TIntermTraverser::traverseBlock()
    Frame 14: 0x121e641a8 sh::TIntermBlock::traverse()
    Frame 15: 0x12200ffa0 sh::EmitMetal()
    Frame 16: 0x1220306e0 sh::TranslatorMSL::translateImpl()
    Frame 17: 0x122033dec sh::TranslatorMSL::translate()
    Frame 18: 0x121d42578 sh::TCompiler::compile()
    Frame 19: 0x121d5e644 sh::Compile()
    Frame 20: 0x121534a24 rx::ShaderTranslateTask::translate()
    Frame 21: 0x1213a851c gl::(anonymous namespace)::CompileTask::compileImpl()
    Frame 22: 0x1213a837c gl::(anonymous namespace)::CompileTask::operator()()
    Frame 23: 0x121885580 angle::AsyncWorkerPool::threadLoop()
    Frame 24: 0x12188e204 std::__fl::__invoke[abi:nn210000]<>()
    Frame 25: 0x12188e14c std::__fl::__thread_execute[abi:nn210000]<>()
    Frame 26: 0x12188dca8 std::__fl::__thread_proxy[abi:nn210000]<>()
    Frame 27: 0x1974f7bc8 _pthread_start
    Frame 28: 0x1974f2b80 thread_start
    
    [2531/7656] Play/DlGoldenTest.TextBlurMaskFilterRespectCTM/OpenGLESSDF returned/aborted with exit code -11 (374 ms)
    

    I've been unable to reproduce these issues locally.

    This does not happen consistently, and it does not appear to be associated with any particular test case. But when it happens, it's always an OpenGLESSDF test rendering with ANGLE.

    @flar

  2. added and removed
    team-infraOwned by Infrastructure team
    on Jul 27, 2026
  3. jtmcdole commented on Jul 27, 2026

    @jtmcdole
    Member

    I don't believe this is team-infra. Over to team-engine

  4. cbracken commented on Jul 28, 2026

    @cbracken
    Member

    As a recent datapoint, looking at passing runs, they're getting pretty close to the timeout:

  5. cbracken commented on Aug 3, 2026

    @cbracken
    Member

    Looking at https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20mac_unopt we're now timing out or flaking 30-50% of all runs.

    I will absolutely regret this but if no one has time for this, I can take a look.

  6. added
    P1High-priority issues at the top of the work list
    on Aug 3, 2026
  7. gaaclarke commented on Aug 3, 2026

    @gaaclarke
    Member

    @flar what is your current recommendation here?

  8. gaaclarke commented on Aug 11, 2026

    @gaaclarke
    Member

    I'm dropping this to p2 because we've disabled the tests (putting them back into the state they were in)

  9. added
    P2Important issues not at the top of the work list
    and removed
    P1High-priority issues at the top of the work list
    on Aug 11, 2026
  10. vashworth commented on Nov 2, 2026

    @vashworth
    Contributor

    we've disabled the tests

    FYI for when we bring these tests back,Play/BlendFilterContentsTest.AdvancedBlendColorAlignsColorTo4/OpenGLESSDF fails consistently on macOS 26: #195511

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Important issues not at the top of the work listteam-engineOwned by Engine teamtriaged-engineTriaged by Engine team

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions