It seems like that for Apple devices, it can happen that:
- We create the TCP tunnel in MacOS
- We start the app
- The TCP tunnel disconnects again
- App fails to connect and runs without it, logs app end tag in its stdout
- XHarness sees the end tag, which sort of mean success, but also never received
testResults.xml over the broken TCP, which means failure
- Together, it qualifies as APP_CRASH as the app failed to connect and send stuff but only because XHarness cannot tell the TCP connection broke
Example log:
[14:51:49] dbug: Starting TCP tunnel between mac port: 58234 and device port 58234
[14:51:49] dbug: [TCP tunnel]
[14:51:49] dbug: [TCP tunnel] Running /private/tmp/helix/working/A42E0919/p/microsoft.dotnet.xharness.cli/1.0.0-prerelease.22375.5/tools/net7.0/any/../../../runtimes/any/native/mlaunch/bin/mlaunch --sdkroot /Applications/Xcode13.app --tcp-tunnel=58234:58234 --devname bd7aed3884d86af1ca7c2311e8706b5a95f3a300 -v -v -v -v -v
[14:51:49] dbug: [TCP tunnel] Using Xcode 13.0 found in /Applications/Xcode13.app
[14:51:49] dbug: [TCP tunnel] Xamarin.Hosting: Device discovery started
[14:51:49] dbug: [TCP tunnel] Xamarin.Hosting: Device discovery event: Connected (bd7aed3884d86af1ca7c2311e8706b5a95f3a300)
[14:51:49] dbug: [TCP tunnel] Xamarin.Hosting: Connected to DNCENGTVOS-071 (bd7aed3884d86af1ca7c2311e8706b5a95f3a300) in 00:00:00.0020177
[14:51:49] dbug: [TCP tunnel] Xamarin.Hosting: Creating USB tunnel between the port 58234 on the device and the port 58234 on the mac.
[14:51:49] dbug: [TCP tunnel] Xamarin.Hosting: Attempting USB tunnel between the port 58234 on the device and the port 58234 (31459) on the mac: 61
[14:51:49] dbug: [TCP tunnel] Xamarin.Hosting: Failed to connect to port 58234 (31459) on device (error: 61)
...
[14:51:54] dbug: [TCP tunnel] Please connect the device 'Tcp tunnel started on device port 58234 to host port 58234 for device bd7aed3884d86af1ca7c2311e8706b5a95f3a300'...
[14:51:54] dbug: TCP tunnel created on port 58234
[14:51:54] dbug: Starting the application
...
[14:51:58] dbug: [TCP tunnel] Xamarin.Hosting: Failed to connect to port 58234 (31459) on device (error: 61)
[14:51:58] dbug: [TCP tunnel] Xamarin.Hosting: Attempting USB tunnel between the port 58234 on the device and the port 58234 (31459) on the mac: 61
[14:51:58] dbug: [TCP tunnel] Xamarin.Hosting: Failed to connect to port 58234 (31459) on device (error: 61)
[14:51:58] dbug: Detected test end tag in application's output
[14:51:58] dbug: Process mlaunch exited with 137
[14:51:58] dbug: Test run completed
...
[14:52:03] dbug: [TCP tunnel] Xamarin.Hosting: Attempting USB tunnel between the port 58234 on the device and the port 58234 (31459) on the mac: 61
[14:52:03] dbug: [TCP tunnel] Xamarin.Hosting: Failed to connect to port 58234 (31459) on device (error: 61)
[14:52:03] dbug: [TCP tunnel] Xamarin.Hosting: Attempting USB tunnel between the port 58234 on the device and the port 58234 (31459) on the mac: 61
[14:52:03] dbug: [TCP tunnel] Xamarin.Hosting: Failed to connect to port 58234 (31459) on device (error: 61)
[14:52:03] dbug: Killing process tree of 64383...
[14:52:03] dbug: Pids to kill: 64383
[14:52:03] dbug: [TCP tunnel] Killing process 64389 as it was cancelled
[14:52:03] dbug: [TCP tunnel] Process mlaunch exited with 137
[14:52:03] dbug: Device log captured in /tmp/helix/working/A42E0919/w/A1AD0929/uploads/device-DNCENGTVOS-071-20220806_145148.log
[14:52:03] dbug: Test run started but crashed and no test results were reported
...
XHarness exit code: 80 (APP_CRASH)
Full log: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-8c0cc100331f483699/System.Console.Manual.Tests/1/console.75d6ad8c.log?%3Fhelixlogtype%3Dresult
It seems like that for Apple devices, it can happen that:
testResults.xmlover the broken TCP, which means failureExample log:
Full log: https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-heads-main-8c0cc100331f483699/System.Console.Manual.Tests/1/console.75d6ad8c.log?%3Fhelixlogtype%3Dresult