Regression? Strictness set in @MockitoSettings ignored after upgrade from 4.5.1 to 4.6.0 #2656
Description
Upgrading from Mockito 4.5.1 to 4.6.0 and it looks to me as though #2650 or something around there has introduced a regression.
I'm seeing PotentialStubbingProblem exceptions where previously there were none, because the test class is annotated with @MockitoSettings(strictness = Strictness.LENIENT).
The issue seems to be that StrictnessSelector prefers the strictness set in the mock to the test level strictness, and the mock defaults to STRICT_STUBS, so always overrides the strictness set in @MockitoSettings.
I've added a failing example test that demonstrates the issue here: big-andy-coates#1
The above test fails with:
expected:
Optional.empty
but was:
Optional[org.mockito.exceptions.misusing.PotentialStubbingProblem:
Strict stubbing argument mismatch. Please check:
- this invocation of 'test' method:
rootMock.test("Bar");
-> at org.mockitousage.ProductionCode.simpleMethod(ProductionCode.java:9)
- has following stubbing(s) with different arguments:
1. rootMock.test("Foo");
-> at org.mockitousage.StrictnessTest$LenientMockitoSettings.should_not_throw_on_potential_stubbing_issue(StrictnessTest.java:176)
Typically, stubbing argument mismatch indicates user mistake when writing tests.
Mockito fails early so that you can debug potential problem easily.
However, there are legit scenarios when this exception generates false negative signal:
- stubbing the same method multiple times using 'given().will()' or 'when().then()' API
Please use 'will().given()' or 'doReturn().when()' API for stubbing.
- stubbed method is intentionally invoked with different arguments by code under test
Please use default or 'silent' JUnit Rule (equivalent of Strictness.LENIENT).
For more information see javadoc for PotentialStubbingProblem class.]
- The mockito message in the stacktrace have useful information, but it didn't help
- The problematic code (if that's possible) is copied here;
Note that some configuration are impossible to mock via Mockito - Provide versions (mockito / jdk / os / any other relevant information)
- Provide a Short, Self Contained, Correct (Compilable), Example of the issue
(same as any question on stackoverflow.com) - Read the contributing guide