refactor: convert rerun-only tests to use fixtures#1764
refactor: convert rerun-only tests to use fixtures#1764carlos-granados merged 1 commit intoBehat:3.xfrom
Conversation
features/rerun-only.feature
Outdated
| Scenario: Run one feature with 2 failed and 3 passing scenarios | ||
| When I run "behat --no-colors -f progress features/apples.feature" | ||
| When I run "behat features/apples.feature" | ||
| Then it should fail with: | ||
| """ | ||
| ..F.............F.... |
There was a problem hiding this comment.
@carlos-granados sorry to keep picking on these rerun features.
I think the process you're working through is very helpfully showing where the noise of the setup in the past has made it hard to reason about what the features are actually covering.
I've realised this first scenario is IMO not a scenario at all. It is not testing anything about the rerun feature, only that the fixture we have created fails as expected.
This is actually a precondition of the later scenarios (where we start with the same `Given I run "behat features/apples.feature" but don't assert anything about the result).
The later scenarios assume that if we've run the same steps to setup the fixture, it will fail in the same way as this scenario did - but this is not actually guaranteed (there could have been some filesystem issue for example).
I think the content of this scenario should either be inlined as the first steps of the actual scenarios, or a Background if it is consistent for all scenarios (it isn't in this case).
This would also apply to the feature we looked at the other day, and to #1766
features/rerun-only.feature
Outdated
| @@ -130,43 +52,22 @@ Feature: Rerun | |||
| """ | |||
|
|
|||
| Scenario: Fixing scenario removes it from the rerun log | |||
There was a problem hiding this comment.
I think this scenario isn't named quite right for what it covers. Run nothing if there were no previous failures or something would make more sense?
There was a problem hiding this comment.
Or maybe this is actually two separate scenarios:
- If Behat has never run before, nothing runs (even with the unchanged
apples.feature- Behat passes because that failing scenario is never executed). - If Behat has run & failed, then was fixed, then
--rerun-onlypassed, then the next run does not run any scenarios.
There was a problem hiding this comment.
Agree that this is better defined as two separate scenarios, I have updated the code
c396653 to
cbc529a
Compare
cbc529a to
21c2836
Compare
21c2836 to
a26568d
Compare
acoulton
left a comment
There was a problem hiding this comment.
thanks @carlos-granados that's a great improvement
In this case we have re-used the fixtures that we had for the
rerun.featuretests as they were almost identical, we have just added a new feature because the behaviour when we were fixing the tests was slightly different