Skip to content

TestDifferentGenericMethodTestCases is flaky #6191

@Youssef1313

Description

@Youssef1313
test/IntegrationTests/MSTest.Acceptance.IntegrationTests/GenericTestMethodTests.cs(21,1): error : Assert.IsTrue failed. Expression 'AssertOutputMatchesRegex' failed for member 'TestDifferentGenericMethodTestCases' at line 21 of file '/_/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/GenericTestMethodTests.cs'. Output of the test host is:
Command: /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/nfxHK/GenericTestMethodTests/bin/Release/net9.0/GenericTestMethodTests
====================
ExitCode: 2
====================
StandardOutput:
MSTest v3.10.0-ci (UTC 7/27/2025) [osx-x64 - .NET 9.0.3]
failed AMethodWithBadConstraints (0) (1s 054ms)
  GenericArguments[0], 'System.Int32', on 'Void AMethodWithBadConstraints[T](T)' violates the constraint of type 'T'.
    at System.RuntimeType.ValidateGenericArguments(MemberInfo definition, RuntimeType[] genericArguments, Exception e)
    at System.Reflection.RuntimeMethodInfo.MakeGenericMethod(Type[] methodInstantiation)
failed NonParameterizedTestMethod (0ms)
  The generic test method 'NonParameterizedTestMethod' doesn't have arguments, so the generic parameter cannot be inferred.
    at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Extensions.MethodInfoExtensions.ConstructGenericMethod(MethodInfo methodInfo, Object[] arguments) in /_/src/Adapter/MSTestAdapter.PlatformServices/Extensions/MethodInfoExtensions.cs:272
    at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Extensions.MethodInfoExtensions.GetInvokeResultAsync(MethodInfo methodInfo, Object classInstance, Object[] arguments) in /_/src/Adapter/MSTestAdapter.PlatformServices/Extensions/MethodInfoExtensions.cs:165
    at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestMethodInfo.ExecuteInternalAsync(Object[] arguments, CancellationTokenSource timeoutTokenSource) in /_/src/Adapter/MSTestAdapter.PlatformServices/Execution/TestMethodInfo.cs:408
failed ParameterizedMethodSimple (1) (8ms)
  Assert.Fail failed. Test method 'ParameterizedMethodSimple' did run with parameter '1' and type 'System.Byte'.
    at TestClass.ParameterizedMethodSimple[T](T parameter) in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/nfxHK/GenericTestMethodTests/UnitTest1.cs:23
    at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
    at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
failed ParameterizedMethodSimple (2) (1ms)
  Assert.Fail failed. Test method 'ParameterizedMethodSimple' did run with parameter '2' and type 'System.Int32'.
    at TestClass.ParameterizedMethodSimple[T](T parameter) in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/nfxHK/GenericTestMethodTests/UnitTest1.cs:23
    at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
    at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
failed ParameterizedMethodSimple ("Hello world") (0ms)
  Assert.Fail failed. Test method 'ParameterizedMethodSimple' did run with parameter 'Hello world' and type 'System.String'.
    at TestClass.ParameterizedMethodSimple[T](T parameter) in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/nfxHK/GenericTestMethodTests/UnitTest1.cs:23
    at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
    at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
failed ParameterizedMethodSimple (null) (2ms)
  Test method TestClass.ParameterizedMethodSimple threw exception: 
  System.InvalidOperationException: The type of the generic parameter 'T' could not be inferred.
failed ParameterizedMethodTwoGenericParametersAndFourMethodParameters (1,"Hello world",2,3) (1ms)
  Test method TestClass.ParameterizedMethodTwoGenericParametersAndFourMethodParameters threw exception: 
  System.InvalidOperationException: Found two conflicting types for generic parameter 'T2'. The conflicting types are 'System.Byte' and 'System.Int32'.
failed ParameterizedMethodTwoGenericParametersAndFourMethodParameters (null,"Hello world","Hello again",3) (1ms)
  Assert.Fail failed. Test method 'ParameterizedMethodTwoGenericParametersAndFourMethodParameters' did run with parameters '<null>', 'Hello world', 'Hello again', '3' and generic types 'System.Int32', 'System.String'.
    at TestClass.ParameterizedMethodTwoGenericParametersAndFourMethodParameters[T1,T2](T2 p1, String p2, T2 p3, T1 p4) in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/nfxHK/GenericTestMethodTests/UnitTest1.cs:31
    at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
    at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
failed ParameterizedMethodTwoGenericParametersAndFourMethodParameters ("Hello hello","Hello world",null,null) (0ms)
  Test method TestClass.ParameterizedMethodTwoGenericParametersAndFourMethodParameters threw exception: 
  System.InvalidOperationException: The type of the generic parameter 'T1' could not be inferred.
failed ParameterizedMethodTwoGenericParametersAndFourMethodParameters (null,null,null,null) (0ms)
  Test method TestClass.ParameterizedMethodTwoGenericParametersAndFourMethodParameters threw exception: 
  System.InvalidOperationException: The type of the generic parameter 'T1' could not be inferred.
failed ParameterizedMethodSimpleParams (1) (0ms)
  Cannot create an instance of T[] because Type.ContainsGenericParameters is true.
    at System.RuntimeType.CreateInstanceCheckThis()
    at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture)
    at Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.ReflectionOperations2.CreateInstance(Type type, Object[] parameters) in /_/src/Adapter/MSTestAdapter.PlatformServices/Services/ReflectionOperations2.cs:55
failed ParameterizedMethodSimpleParams (1,2) (0ms)
  Cannot create an instance of T[] because Type.ContainsGenericParameters is true.
    at System.RuntimeType.CreateInstanceCheckThis()
    at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture)
    at Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.ReflectionOperations2.CreateInstance(Type type, Object[] parameters) in /_/src/Adapter/MSTestAdapter.PlatformServices/Services/ReflectionOperations2.cs:55
failed ParameterizedMethodSimpleParams ("Hello world") (0ms)
  Cannot create an instance of T[] because Type.ContainsGenericParameters is true.
    at System.RuntimeType.CreateInstanceCheckThis()
    at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture)
    at Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.ReflectionOperations2.CreateInstance(Type type, Object[] parameters) in /_/src/Adapter/MSTestAdapter.PlatformServices/Services/ReflectionOperations2.cs:55
failed ParameterizedMethodSimpleParams (null) (0ms)
  Cannot create an instance of T[] because Type.ContainsGenericParameters is true.
    at System.RuntimeType.CreateInstanceCheckThis()
    at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture)
    at Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.ReflectionOperations2.CreateInstance(Type type, Object[] parameters) in /_/src/Adapter/MSTestAdapter.PlatformServices/Services/ReflectionOperations2.cs:55
failed ParameterizedMethodSimpleParams (null,"Hello world") (0ms)
  Cannot create an instance of T[] because Type.ContainsGenericParameters is true.
    at System.RuntimeType.CreateInstanceCheckThis()
    at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture)
    at Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.ReflectionOperations2.CreateInstance(Type type, Object[] parameters) in /_/src/Adapter/MSTestAdapter.PlatformServices/Services/ReflectionOperations2.cs:55
failed ParameterizedMethodWithNestedGeneric (System.Collections.Generic.List`1[System.String],System.Collections.Generic.List`1[System.String]) (1ms)
  Assert.Fail failed. Test method 'ParameterizedMethodWithNestedGeneric' did run with first list [Hello, World] and second list [Unit, Testing]
    at TestClass.ParameterizedMethodWithNestedGeneric[T](List`1 a, List`1 b) in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/nfxHK/GenericTestMethodTests/UnitTest1.cs:48
    at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
    at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
failed ParameterizedMethodWithNestedGeneric (System.Collections.Generic.List`1[System.Int32],System.Collections.Generic.List`1[System.Int32]) (636ms)
  Assert.Fail failed. Test method 'ParameterizedMethodWithNestedGeneric' did run with first list [0, 1] and second list [2, 3]
    at TestClass.ParameterizedMethodWithNestedGeneric[T](List`1 a, List`1 b) in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/nfxHK/GenericTestMethodTests/UnitTest1.cs:48
    at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
    at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
Test run summary: Failed! - /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/nfxHK/GenericTestMethodTests/bin/Release/net9.0/GenericTestMethodTests.dll (net9.0|x64)
  total: 17
  failed: 17
  succeeded: 0
  skipped: 0
  duration: 4s 660ms
====================
StandardError:

Metadata

Metadata

Assignees

Labels

Flaky-TestThis label marks issues that track flaky tests in CI.

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions