Getting test output that breaks Visual Studio Devops Build pipeline (MSTest.Adapter v2.2.3) #799
Description
Description
I was holding back on v2.2.1 and using v2.1.2 because of issue #789
When I upgraded to v2.2.2 and then to v2.2.3 I find the tests are running inside Visual Studio and the breaking behavior in 2.2.1 is resolved; however after running all the tests In the Test Output window I notice the following message
---------- Starting test run ----------
MSTest Executor: Test Parallelization enabled for C:\Path\To\My\Test\Project\bin\x64\Debug\net48\Tests.Integration.dll (Workers: 12, Scope: ClassLevel).
An exception occurred while invoking executor 'executor://mstestadapter/v2': The parameter cannot be null or empty.
Parameter name: message
========== Test run finished: 693 Tests run in 48.9 sec (693 Passed, 0 Failed, 0 Skipped) ==========
This doesn't cause any issues in Visual studio but when they are executed in Visual Studio DevOps pipeline, the mention of the word exception causes the Pipeline task to fail, even though all tests have passed.
I have rolled back to 2.1.2 multiple times, and after reverting, the above error message goes away.
Steps to reproduce
- Take test project with MSTest.Adapter version 2.1.2
- Upgrade to 2.2.3
Expected behavior
All tests should pass without any messages mentioning any exception, e.g. when running same tests with v2.1.2 give the following message at end of run
---------- Starting test run ----------
MSTest Executor: Test Parallelization enabled for C:\Path\To\My\Test\Project\bin\x64\Debug\net48\Tests.Integration.dll (Workers: 12, Scope: ClassLevel).
========== Test run finished: 693 Tests run in 50.4 sec (693 Passed, 0 Failed, 0 Skipped) ==========
Actual behavior
I get the following message
---------- Starting test run ----------
MSTest Executor: Test Parallelization enabled for C:\Path\To\My\Test\Project\bin\x64\Debug\net48\Tests.Integration.dll (Workers: 12, Scope: ClassLevel).
An exception occurred while invoking executor 'executor://mstestadapter/v2': The parameter cannot be null or empty.
Parameter name: message
========== Test run finished: 693 Tests run in 48.9 sec (693 Passed, 0 Failed, 0 Skipped) ==========
I could be missing some Run parameters, but I couldn't figure out which one exactly, quickly. Any pointers appreciated.
Environment
Visual Studio 2019 16.9.1
Windows 10 Enterprise x64
Test environment set to x64 explicitly
Nuget Package Style: PackageReference
Package references in the project
<ItemGroup>
<PackageReference Include="Castle.Core" Version="4.4.1" />
<PackageReference Include="EfCore.TestSupport" Version="3.2.0" />
<PackageReference Include="FluentAssertions" Version="5.10.3" />
<PackageReference Include="Microsoft.EntityFrameworkCore.InMemory" Version="3.1.13" />
<PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite.NetTopologySuite" Version="3.1.13" />
<PackageReference Include="Microsoft.EntityFrameworkCore.SqlServer.NetTopologySuite" Version="3.1.13" />
<PackageReference Include="Microsoft.Extensions.Configuration.Abstractions" Version="5.0.0" />
<PackageReference Include="Moq" Version="4.16.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.9.1" />
<PackageReference Include="MSTest.TestAdapter" Version="2.2.3" />
<PackageReference Include="MSTest.TestFramework" Version="2.2.3" />
<PackageReference Include="Newtonsoft.Json.Schema" Version="3.0.14" />
<PackageReference Include="sqlite" Version="3.13.0" />
<PackageReference Include="System.ValueTuple" Version="4.5.0" />
</ItemGroup>