-
Notifications
You must be signed in to change notification settings - Fork 5.4k
Description
Test failures inside the library tests for Native AOT don't show as failed in the CI pipeline even when a test fails.
Example (all checks/jobs/steps below show up as green):
This PR: https://github.com/dotnet/runtime/pull/70173/checks?check_run_id=6716857093
Under Checks->.NET Helix->Build Analysis->Runtime brings you to here: https://dev.azure.com/dnceng/public/_build/results?buildId=1803448&view=results
Find Job "Build windows arm64 Release NativeAOT" with step "Send to Helix" which shows: https://dev.azure.com/dnceng/public/_build/results?buildId=1803448&view=logs&j=7bb91cf6-d57c-5393-cf5c-c884f1e50962&t=7729b4d4-9255-54d8-01fd-e831ebbb744b
From there, you can see "Sent Helix Job; see work items at https://helix.dot.net/api/jobs/7528557f-cbe9-4665-8fa3-6028facd5084/workitems?api-version=2019-06-17"
Follow that link and you will see the "System.Reflection.Tests" details URL: https://helix.dot.net/api/jobs/7528557f-cbe9-4665-8fa3-6028facd5084/workitems/System.Reflection.Tests?api-version=2019-06-17
And from there, you can see the "ConsoleOutputUri https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-70173-merge-7528557fcbe946658f/System.Reflection.Tests/1/console.ba43aa3a.log?helixlogtype=result"
Which shows this:
System.PlatformNotSupportedException : Operation is not supported on this platform.
at System.Reflection.Assembly.GetCallingAssembly() + 0x6c
at System.Reflection.TestAssembly.ClassToInvoke.CallMeActual() + 0xc
at System.Reflection.TestAssembly.ClassToInvoke.CallMe_AggressiveInlining() + 0x10
at System.Reflection.Tests!<BaseAddress>+0x681698
at System.InvokeUtils.CallDynamicInvokeMethod(Object, IntPtr, IntPtr, IntPtr, Object, Object[], BinderBundle, Boolean, Boolean) + 0xec
Running these tests manually locally also results in this error.