You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The acceptance test suite has several tests with extremely broad parameterization matrices (runner × TFM × isolation combos), making them the most expensive tests by total wall-clock time. From a recent net48_x86 TRX analysis:
RunTestsFromMultipleMSTestAssemblies alone runs 140 times and consumes ~6.5 minutes of wall-clock time.
Suggestion
Review the parameterization for these high-frequency tests and determine whether the full matrix is necessary. Many of these tests likely validate framework-agnostic behavior and could run on a reduced set of runner/TFM combinations without losing meaningful coverage.
Possible approaches:
Reduce DataSource attributes to cover only boundary TFMs (e.g. oldest + newest)
Split into a "full matrix" category that runs only in CI, with a smaller matrix for local/PR runs
Problem
The acceptance test suite has several tests with extremely broad parameterization matrices (runner × TFM × isolation combos), making them the most expensive tests by total wall-clock time. From a recent net48_x86 TRX analysis:
RunTestsFromMultipleMSTestAssembliesRunMultipleTestAssembliesRunTestsWithCustomTestHostLauncher…OutdatedITestHostLauncherRunAllTestsWithMixedTFMs…RunAllTestsRunMultipleMSTestAssemblies…Combinations2DiscoverTestsUsingDiscoveryEventHandler1RunTestsWithTestCaseFilterDiscoverTestsUsingDiscoveryEventHandler2AndTelemetryOptedOutRunTestsWithCustomTestHostLauncher…AttachesToDebuggerRunTestsFromMultipleMSTestAssembliesalone runs 140 times and consumes ~6.5 minutes of wall-clock time.Suggestion
Review the parameterization for these high-frequency tests and determine whether the full matrix is necessary. Many of these tests likely validate framework-agnostic behavior and could run on a reduced set of runner/TFM combinations without losing meaningful coverage.
Possible approaches: