Skip to content

[don't merge] Try filtering out tests fails on net9#15450

Closed
nohwnd wants to merge 5 commits into
microsoft:mainfrom
nohwnd:try-filtering-out-tests-fails-on-net9
Closed

[don't merge] Try filtering out tests fails on net9#15450
nohwnd wants to merge 5 commits into
microsoft:mainfrom
nohwnd:try-filtering-out-tests-fails-on-net9

Conversation

@nohwnd

@nohwnd nohwnd commented Mar 9, 2026

Copy link
Copy Markdown
Member

I see local failures in unexpected place locally, trying if that will fail also in ci. if not something is very wrong.

RunTestsShouldThrowOnStackOverflowException

        var errorMessage = runnerInfo.TargetFramework == "net462"
            ? $"The active test run was aborted. Reason: Test host process crashed : Process is terminated due to StackOverflowException.{Environment.NewLine}"
            : $"The active test run was aborted. Reason: Test host process crashed : Stack overflow.{Environment.NewLine}";
image image

Copilot AI review requested due to automatic review settings March 9, 2026 09:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR attempts to avoid/diagnose unexpected failures on newer .NET (net9) by skipping a large set of acceptance/integration tests on .NET (Core) and by forcing those tests to run under the VSTest runner via a per-working-directory global.json.

Changes:

  • Force dotnet test integration runs to use VSTest by writing/validating a global.json in the test working directory.
  • Add a new MSTest condition attribute and apply it broadly to skip many integration tests on .NET (Core) to avoid duplicated out-of-proc runs.
  • Switch repo-level test runner configuration from dotnet.config to global.json.

Reviewed changes

Copilot reviewed 37 out of 37 changed files in this pull request and generated no comments.

Show a summary per file
File Description
test/Microsoft.TestPlatform.TestUtilities/IntegrationTestBase.cs Writes/validates per-working-dir global.json to force VSTest runner
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/TestPlatformNugetPackageTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/TestCaseFilterTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/TelemetryTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/SkipIntegrationTestOnNetConditionAttribute.cs Introduces new condition attribute used to filter test execution by runtime
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/SelfContainedAppTests.cs Skips integration test on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/RunsettingsTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/ResultsDirectoryTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/RecursiveResourcesLookupTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/ProcessesInteractionTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/PostProcessingTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/PortableNugetPackageTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/PlatformTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/MultitargetingTestHostTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/LoggerTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/ListExtensionsTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/FrameworkTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/FilePatternParserTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/ExecutionThreadApartmentStateTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/ExecutionTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/EventLogCollectorTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/DotnetTestTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/DotnetTestMSBuildOutputTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/DotnetHostArchitectureVerifierTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/DotnetArchitectureSwitchTests.Windows.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/DiscoveryTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/DisableAppdomainTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/DifferentTestFrameworkSimpleTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/DeprecateExtensionsPathWarningTests.cs Adds a TODO comment in the test class header
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/DebugAssertTests.cs Skips integration test on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/DataCollectorTests.Coverlet.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/DataCollectorAttachmentsProcessorsFactoryTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/DataCollectionTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/CodeCoverageTests.cs Skips integration tests on .NET (Core) via new condition attribute
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/BlameDataCollectorTests.cs Skips integration tests on .NET (Core) and adds a new test category
global.json Adds repo-level test runner config using global.json
dotnet.config Removes previous test runner config
Comments suppressed due to low confidence (4)

test/Microsoft.TestPlatform.TestUtilities/IntegrationTestBase.cs:1

  • workingDirectory! is dereferenced before the null-check, so calling InvokeDotnetTest(..., workingDirectory: null) will throw immediately (even though the following if attempts to guard it). Compute globalJsonPath only after verifying workingDirectory is non-null, or early-return/throw with a clear message when it is null.
    test/Microsoft.TestPlatform.TestUtilities/IntegrationTestBase.cs:1
  • The JSON being written is invalid (trailing comma after "vstest", and an extra comma before the closing brace). This will likely cause dotnet test to ignore or error on the file, defeating the intent to enforce the runner. Write valid JSON (no trailing commas and correct brace structure).
    test/Microsoft.TestPlatform.TestUtilities/IntegrationTestBase.cs:1
  • Validating global.json by substring match is brittle (formatting/whitespace, different property ordering, or equivalent JSON could fail validation). Prefer parsing JSON and checking test.runner semantically, and consider using Environment.NewLine for message formatting consistency.
    test/Microsoft.TestPlatform.Acceptance.IntegrationTests/SkipIntegrationTestOnNetConditionAttribute.cs:1
  • The attribute name and internal field name are confusing relative to behavior: ConditionMode.Include + _include = ...StartsWith(".NET Framework") means “run only on .NET Framework”, not “skip on net”. Consider renaming the attribute to reflect actual behavior (e.g., RunOnlyOnNetFrameworkAttribute) and rename _include to something explicit like _isNetFramework.

@nohwnd nohwnd marked this pull request as draft March 9, 2026 10:00
@nohwnd nohwnd changed the title Try filtering out tests fails on net9 [don't merge] Try filtering out tests fails on net9 Mar 9, 2026
@nohwnd nohwnd mentioned this pull request Mar 9, 2026
@nohwnd

nohwnd commented Mar 9, 2026

Copy link
Copy Markdown
Member Author

The failing test skips itself in Release mode, fixing in the linked PR.

@nohwnd nohwnd closed this Mar 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants