You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Mac_arm64_ios hot_mode_dev_cycle_ios__benchmark is 2.04% flaky #190995
The post-submit test builder Mac_arm64_ios hot_mode_dev_cycle_ios__benchmark had a flaky ratio 2.04% for the past (up to) 100 commits, which is above our 2.00% threshold.
On physical iOS 17+ devices, launching debug JIT apps directly via devicectl device process launch exits immediately with signal 11 (SIGSEGV) due to OS-level memory protection preventing JIT execution without an attached debugger.
2. Xcode Debugger Fallback
The Flutter tool catches the launch exit and falls back to opening and debugging the workspace through Xcode using the JXA automation script (packages/flutter_tools/bin/xcode_debug.js).
3. Symlink Mismatch in getWorkspaceDocument
In xcode_debug.js, getWorkspaceDocument matches open Xcode workspaces against the target directory using string comparison:
On Swarming CI bots, working directories contain symbolic links (e.g., /opt/s/w/ir/x/t/...). Xcode resolves symlinks when opening documents, returning canonical paths, whereas the tool passes unresolved paths. Because the paths do not match literally:
xcode_debug.js repeatedly logs Failed to get workspace.
It eventually reaches the 14-minute timeout: Timed out waiting for workspace to load.
4. Flakiness Mechanism
After the Xcode fallback times out on the physical device, the benchmark harness falls back to running on iphonesimulator. The simulator run succeeds, but the overall task is marked flaky due to the initial device failure and timeout.
Implemented Fix
1. Canonicalize Paths in JXA (xcode_debug.js)
Import Foundation and resolve symlinks in both open document paths and command arguments using NSURL.URLByResolvingSymlinksInPath before comparison:
Fixed a misleading log message that previously reported lldb exited with code $status when the monitored process was actually devicectl device process launch.
I don't think we should change this file without very thorough manual testing. I think the AI is likely wrong.
On physical iOS 17+ devices, launching debug JIT apps directly via devicectl device process launch exits immediately with signal 11 (SIGSEGV) due to OS-level memory protection preventing JIT execution without an attached debugger.
We should look into fixing 1, although, I think the AI's assumptions on why it's failing are also wrong.
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.
Activity
bkonyi commentedon Aug 12, 2026
Root Cause Analysis
1. Physical Device Launch Behavior (iOS 17+)
On physical iOS 17+ devices, launching debug JIT apps directly via
devicectl device process launchexits immediately withsignal 11(SIGSEGV) due to OS-level memory protection preventing JIT execution without an attached debugger.2. Xcode Debugger Fallback
The Flutter tool catches the launch exit and falls back to opening and debugging the workspace through Xcode using the JXA automation script (
packages/flutter_tools/bin/xcode_debug.js).3. Symlink Mismatch in
getWorkspaceDocumentIn
xcode_debug.js,getWorkspaceDocumentmatches open Xcode workspaces against the target directory using string comparison:On Swarming CI bots, working directories contain symbolic links (e.g.,
/opt/s/w/ir/x/t/...). Xcode resolves symlinks when opening documents, returning canonical paths, whereas the tool passes unresolved paths. Because the paths do not match literally:xcode_debug.jsrepeatedly logsFailed to get workspace.Timed out waiting for workspace to load.4. Flakiness Mechanism
After the Xcode fallback times out on the physical device, the benchmark harness falls back to running on
iphonesimulator. The simulator run succeeds, but the overall task is marked flaky due to the initial device failure and timeout.Implemented Fix
1. Canonicalize Paths in JXA (
xcode_debug.js)Import
Foundationand resolve symlinks in both open document paths and command arguments usingNSURL.URLByResolvingSymlinksInPathbefore comparison:2. Log Message Cleanup (
core_devices.dart)Fixed a misleading log message that previously reported
lldb exited with code $statuswhen the monitored process was actuallydevicectl device process launch.vashworth commentedon Aug 12, 2026
I don't think we should change this file without very thorough manual testing. I think the AI is likely wrong.
We should look into fixing 1, although, I think the AI's assumptions on why it's failing are also wrong.
It's appears to be crashing on launch, we likely need to VNC to the bot to get the crash logs to diagnose why it's crashing
fluttergithubbot commentedon Aug 26, 2026
[prod pool] flaky ratio for the past (up to) 100 commits between 2026-08-20 and 2026-08-25 is 2.00%. Flaky number: 2; total number: 100.
One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_arm64_ios%20hot_mode_dev_cycle_ios__benchmark/11597
Commit: 40b3290
Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_arm64_ios%20hot_mode_dev_cycle_ios__benchmark/11597
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_arm64_ios%20hot_mode_dev_cycle_ios__benchmark/11567
Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Mac_arm64_ios%20hot_mode_dev_cycle_ios__benchmark
vashworth commentedon Aug 27, 2026
Okay I was able to get the crash logs and confirmed it is crashing on launch
vashworth commentedon Aug 27, 2026
Okay this crash can happen when you launch a debug build: #149214
However, we purposely pass
--start-stoppedtodevicectlso that it will start in a "suspended state" until a debugger is attached.Gemini speculates it could be because "This issue usually happens because the system reuses an already running/suspended instance of your app"
Terminate existing process before launching with devicectl (flutter#1…
fluttergithubbot commentedon Sep 2, 2026
[prod pool] flaky ratio for the past (up to) 100 commits between 2026-08-27 and 2026-09-01 is 2.00%. Flaky number: 2; total number: 100.
One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_arm64_ios%20hot_mode_dev_cycle_ios__benchmark/11690
Commit: 3796c36
Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_arm64_ios%20hot_mode_dev_cycle_ios__benchmark/11690
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_arm64_ios%20hot_mode_dev_cycle_ios__benchmark/11671
Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Mac_arm64_ios%20hot_mode_dev_cycle_ios__benchmark
vashworth commentedon Sep 2, 2026
Looks like recent failure was due to the iOS device not being found - let's wait another week
fluttergithubbot commentedon Sep 9, 2026
[prod pool] flaky ratio for the past (up to) 100 commits between 2026-09-01 and 2026-09-08 is 1.04%. Flaky number: 1; total number: 96.
One recent flaky example for a same commit: https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_arm64_ios%20hot_mode_dev_cycle_ios__benchmark/11840
Commit: 2d39b7d
Flaky builds:
https://ci.chromium.org/ui/p/flutter/builders/prod/Mac_arm64_ios%20hot_mode_dev_cycle_ios__benchmark/11840
Recent test runs:
https://flutter-dashboard.appspot.com/#/build?taskFilter=Mac_arm64_ios%20hot_mode_dev_cycle_ios__benchmark
fluttergithubbot commentedon Sep 16, 2026
[prod pool] flaky ratio for the past (up to) 100 commits between 2026-09-09 and 2026-09-15 is 0.00%. Flaky number: 0; total number: 98.
Piinks commentedon Sep 21, 2026
Closing as no longer flaking
github-actions commentedon Oct 5, 2026
This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of
flutter doctor -vand a minimal reproduction of the issue.