-
Notifications
You must be signed in to change notification settings - Fork 291
TestDifferentGenericMethodTestCases is flaky #6191
Copy link
Copy link
Labels
Flaky-TestThis label marks issues that track flaky tests in CI.This label marks issues that track flaky tests in CI.
Description
test/IntegrationTests/MSTest.Acceptance.IntegrationTests/GenericTestMethodTests.cs(21,1): error : Assert.IsTrue failed. Expression 'AssertOutputMatchesRegex' failed for member 'TestDifferentGenericMethodTestCases' at line 21 of file '/_/test/IntegrationTests/MSTest.Acceptance.IntegrationTests/GenericTestMethodTests.cs'. Output of the test host is:
Command: /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/nfxHK/GenericTestMethodTests/bin/Release/net9.0/GenericTestMethodTests
====================
ExitCode: 2
====================
StandardOutput:
MSTest v3.10.0-ci (UTC 7/27/2025) [osx-x64 - .NET 9.0.3]
failed AMethodWithBadConstraints (0) (1s 054ms)
GenericArguments[0], 'System.Int32', on 'Void AMethodWithBadConstraints[T](T)' violates the constraint of type 'T'.
at System.RuntimeType.ValidateGenericArguments(MemberInfo definition, RuntimeType[] genericArguments, Exception e)
at System.Reflection.RuntimeMethodInfo.MakeGenericMethod(Type[] methodInstantiation)
failed NonParameterizedTestMethod (0ms)
The generic test method 'NonParameterizedTestMethod' doesn't have arguments, so the generic parameter cannot be inferred.
at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Extensions.MethodInfoExtensions.ConstructGenericMethod(MethodInfo methodInfo, Object[] arguments) in /_/src/Adapter/MSTestAdapter.PlatformServices/Extensions/MethodInfoExtensions.cs:272
at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Extensions.MethodInfoExtensions.GetInvokeResultAsync(MethodInfo methodInfo, Object classInstance, Object[] arguments) in /_/src/Adapter/MSTestAdapter.PlatformServices/Extensions/MethodInfoExtensions.cs:165
at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestMethodInfo.ExecuteInternalAsync(Object[] arguments, CancellationTokenSource timeoutTokenSource) in /_/src/Adapter/MSTestAdapter.PlatformServices/Execution/TestMethodInfo.cs:408
failed ParameterizedMethodSimple (1) (8ms)
Assert.Fail failed. Test method 'ParameterizedMethodSimple' did run with parameter '1' and type 'System.Byte'.
at TestClass.ParameterizedMethodSimple[T](T parameter) in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/nfxHK/GenericTestMethodTests/UnitTest1.cs:23
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
failed ParameterizedMethodSimple (2) (1ms)
Assert.Fail failed. Test method 'ParameterizedMethodSimple' did run with parameter '2' and type 'System.Int32'.
at TestClass.ParameterizedMethodSimple[T](T parameter) in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/nfxHK/GenericTestMethodTests/UnitTest1.cs:23
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
failed ParameterizedMethodSimple ("Hello world") (0ms)
Assert.Fail failed. Test method 'ParameterizedMethodSimple' did run with parameter 'Hello world' and type 'System.String'.
at TestClass.ParameterizedMethodSimple[T](T parameter) in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/nfxHK/GenericTestMethodTests/UnitTest1.cs:23
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
failed ParameterizedMethodSimple (null) (2ms)
Test method TestClass.ParameterizedMethodSimple threw exception:
System.InvalidOperationException: The type of the generic parameter 'T' could not be inferred.
failed ParameterizedMethodTwoGenericParametersAndFourMethodParameters (1,"Hello world",2,3) (1ms)
Test method TestClass.ParameterizedMethodTwoGenericParametersAndFourMethodParameters threw exception:
System.InvalidOperationException: Found two conflicting types for generic parameter 'T2'. The conflicting types are 'System.Byte' and 'System.Int32'.
failed ParameterizedMethodTwoGenericParametersAndFourMethodParameters (null,"Hello world","Hello again",3) (1ms)
Assert.Fail failed. Test method 'ParameterizedMethodTwoGenericParametersAndFourMethodParameters' did run with parameters '<null>', 'Hello world', 'Hello again', '3' and generic types 'System.Int32', 'System.String'.
at TestClass.ParameterizedMethodTwoGenericParametersAndFourMethodParameters[T1,T2](T2 p1, String p2, T2 p3, T1 p4) in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/nfxHK/GenericTestMethodTests/UnitTest1.cs:31
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
failed ParameterizedMethodTwoGenericParametersAndFourMethodParameters ("Hello hello","Hello world",null,null) (0ms)
Test method TestClass.ParameterizedMethodTwoGenericParametersAndFourMethodParameters threw exception:
System.InvalidOperationException: The type of the generic parameter 'T1' could not be inferred.
failed ParameterizedMethodTwoGenericParametersAndFourMethodParameters (null,null,null,null) (0ms)
Test method TestClass.ParameterizedMethodTwoGenericParametersAndFourMethodParameters threw exception:
System.InvalidOperationException: The type of the generic parameter 'T1' could not be inferred.
failed ParameterizedMethodSimpleParams (1) (0ms)
Cannot create an instance of T[] because Type.ContainsGenericParameters is true.
at System.RuntimeType.CreateInstanceCheckThis()
at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture)
at Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.ReflectionOperations2.CreateInstance(Type type, Object[] parameters) in /_/src/Adapter/MSTestAdapter.PlatformServices/Services/ReflectionOperations2.cs:55
failed ParameterizedMethodSimpleParams (1,2) (0ms)
Cannot create an instance of T[] because Type.ContainsGenericParameters is true.
at System.RuntimeType.CreateInstanceCheckThis()
at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture)
at Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.ReflectionOperations2.CreateInstance(Type type, Object[] parameters) in /_/src/Adapter/MSTestAdapter.PlatformServices/Services/ReflectionOperations2.cs:55
failed ParameterizedMethodSimpleParams ("Hello world") (0ms)
Cannot create an instance of T[] because Type.ContainsGenericParameters is true.
at System.RuntimeType.CreateInstanceCheckThis()
at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture)
at Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.ReflectionOperations2.CreateInstance(Type type, Object[] parameters) in /_/src/Adapter/MSTestAdapter.PlatformServices/Services/ReflectionOperations2.cs:55
failed ParameterizedMethodSimpleParams (null) (0ms)
Cannot create an instance of T[] because Type.ContainsGenericParameters is true.
at System.RuntimeType.CreateInstanceCheckThis()
at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture)
at Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.ReflectionOperations2.CreateInstance(Type type, Object[] parameters) in /_/src/Adapter/MSTestAdapter.PlatformServices/Services/ReflectionOperations2.cs:55
failed ParameterizedMethodSimpleParams (null,"Hello world") (0ms)
Cannot create an instance of T[] because Type.ContainsGenericParameters is true.
at System.RuntimeType.CreateInstanceCheckThis()
at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture)
at Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.ReflectionOperations2.CreateInstance(Type type, Object[] parameters) in /_/src/Adapter/MSTestAdapter.PlatformServices/Services/ReflectionOperations2.cs:55
failed ParameterizedMethodWithNestedGeneric (System.Collections.Generic.List`1[System.String],System.Collections.Generic.List`1[System.String]) (1ms)
Assert.Fail failed. Test method 'ParameterizedMethodWithNestedGeneric' did run with first list [Hello, World] and second list [Unit, Testing]
at TestClass.ParameterizedMethodWithNestedGeneric[T](List`1 a, List`1 b) in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/nfxHK/GenericTestMethodTests/UnitTest1.cs:48
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
failed ParameterizedMethodWithNestedGeneric (System.Collections.Generic.List`1[System.Int32],System.Collections.Generic.List`1[System.Int32]) (636ms)
Assert.Fail failed. Test method 'ParameterizedMethodWithNestedGeneric' did run with first list [0, 1] and second list [2, 3]
at TestClass.ParameterizedMethodWithNestedGeneric[T](List`1 a, List`1 b) in /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/nfxHK/GenericTestMethodTests/UnitTest1.cs:48
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
Test run summary: Failed! - /Users/runner/work/1/s/artifacts/tmp/Debug/testsuite/nfxHK/GenericTestMethodTests/bin/Release/net9.0/GenericTestMethodTests.dll (net9.0|x64)
total: 17
failed: 17
succeeded: 0
skipped: 0
duration: 4s 660ms
====================
StandardError:
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
Flaky-TestThis label marks issues that track flaky tests in CI.This label marks issues that track flaky tests in CI.