Define a merged runner for the nativeaot subtree and run every test out-of-proc and don't reference the xunit generator#107002
Conversation
…ut-of-proc and don't reference the xunit generator.
jkoritzinsky
added
test-enhancement
|
Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas |
| <PropertyGroup> | ||
| <DisableProjectBuild Condition="'$(IsMergedTestRunnerAssembly)' == 'true' and '$(BuildAllTestsAsStandalone)' == 'true' and '$(HasMergedInTests)' != 'true'">true</DisableProjectBuild> | ||
| <OutputType Condition="('$(IsMergedTestRunnerAssembly)' == 'true' and '$(BuildAllTestsAsStandalone)' != 'true') or ('$(RequiresProcessIsolation)' == 'true' and '$(CLRTestKind)' != 'SharedLibrary')">Exe</OutputType> | ||
| <OutputType Condition="'$(SkipInferOutputType)' != 'true' and (('$(IsMergedTestRunnerAssembly)' == 'true' and '$(BuildAllTestsAsStandalone)' != 'true') or ('$(RequiresProcessIsolation)' == 'true' and '$(CLRTestKind)' != 'SharedLibrary'))">Exe</OutputType> |
There was a problem hiding this comment.
I've a question. What is this SkipInferOutputType tag?
There was a problem hiding this comment.
I had to add this to get this test to behave correctly. Basically the test system today infers the OutputType property value based on other properties. Because of file ordering, OutputType is non-empty by the time we're doing it, so we need to add another property as the hint to not do that work.
There was a problem hiding this comment.
Ah I see, thanks for explaining!
|
/azp run runtime-coreclr outerloop |
|
Azure Pipelines successfully started running 1 pipeline(s). |
MichalStrehovsky
left a comment
MichalStrehovsky
left a comment
There was a problem hiding this comment.
Are we running tests? Clicking the "Build windows-x64 Checked NativeAOT" leg, I see this in the Helix step:
Sent Helix Job; see work items at https://helix.dot.net/api/jobs/21c99c0d-f2cf-4505-bea9-f5b4ce041277/workitems?api-version=2019-06-17
Clicking that, and going to nativeaot detailsuri: https://helix.dot.net/api/jobs/21c99c0d-f2cf-4505-bea9-f5b4ce041277/workitems/nativeaot?api-version=2019-06-17 and from there the consoleoutputuri: https://helixre107v0xdcypoyl9e7f.blob.core.windows.net/dotnet-runtime-refs-pull-107002-merge-21c99c0df2cf4505be/nativeaot/1/console.e18a6ba6.log?helixlogtype=result
I'm only seeing test "runs" to the tune of:
19:11:40.357 Running test: nativeaot\SmokeTests\HardwareIntrinsics\X64Avx\X64Avx.cmd
Test executable 'C:\h\w\B4BE09A2\w\A94C08F0\e\nativeaot\nativeaot\native\..\..\SmokeTests\HardwareIntrinsics\X64Avx\X64Avx.cmd' not found, skipping.
19:11:40.358 Passed test: nativeaot\SmokeTests\HardwareIntrinsics\X64Avx\X64Avx.cmd
|
I'll figure out what makes our Helix runs different from local (and why the Helix payloads don't work). |
…ation and set as build-as-standalone. Only skip when the test suite is being built in "all tests as standalone" mode.
|
@MichalStrehovsky I've fixed the payload construction. Lets see if CI is happy now. |
jkoritzinsky
dismissed
MichalStrehovsky’s stale review
Tests are running. Issue has been resolved.
…ut-of-proc and don't reference the xunit generator (dotnet#107002)
3 participants
This is a very simplistic port to the merged runner infrastructure just so we can get the last few directories onto it to enable removing the old infrastructure.
I've validated locally that all of the tests actually run with this change.