Skip to content

NativeAOT Library Test Failures Don't Fail CI #70223

@Beau-Gosse-dev

Description

@Beau-Gosse-dev

Test failures inside the library tests for Native AOT don't show as failed in the CI pipeline even when a test fails.

Example (all checks/jobs/steps below show up as green):
This PR: https://github.com/dotnet/runtime/pull/70173/checks?check_run_id=6716857093

Under Checks->.NET Helix->Build Analysis->Runtime brings you to here: https://dev.azure.com/dnceng/public/_build/results?buildId=1803448&view=results

Find Job "Build windows arm64 Release NativeAOT" with step "Send to Helix" which shows: https://dev.azure.com/dnceng/public/_build/results?buildId=1803448&view=logs&j=7bb91cf6-d57c-5393-cf5c-c884f1e50962&t=7729b4d4-9255-54d8-01fd-e831ebbb744b

From there, you can see "Sent Helix Job; see work items at https://helix.dot.net/api/jobs/7528557f-cbe9-4665-8fa3-6028facd5084/workitems?api-version=2019-06-17"

Follow that link and you will see the "System.Reflection.Tests" details URL: https://helix.dot.net/api/jobs/7528557f-cbe9-4665-8fa3-6028facd5084/workitems/System.Reflection.Tests?api-version=2019-06-17

And from there, you can see the "ConsoleOutputUri https://helixre107v0xdeko0k025g8.blob.core.windows.net/dotnet-runtime-refs-pull-70173-merge-7528557fcbe946658f/System.Reflection.Tests/1/console.ba43aa3a.log?helixlogtype=result"

Which shows this:

System.PlatformNotSupportedException : Operation is not supported on this platform.
   at System.Reflection.Assembly.GetCallingAssembly() + 0x6c
   at System.Reflection.TestAssembly.ClassToInvoke.CallMeActual() + 0xc
   at System.Reflection.TestAssembly.ClassToInvoke.CallMe_AggressiveInlining() + 0x10
   at System.Reflection.Tests!<BaseAddress>+0x681698
   at System.InvokeUtils.CallDynamicInvokeMethod(Object, IntPtr, IntPtr, IntPtr, Object, Object[], BinderBundle, Boolean, Boolean) + 0xec

Running these tests manually locally also results in this error.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions