-
-
Notifications
You must be signed in to change notification settings - Fork 613
Description
#1515 improved the output for the basic It should pass and It should fail steps - but also highlighted that these are generally "fallback" assertions because in many cases we are (or should be) using It should pass with: or It should fail with: to assert that the behat output includes expected content.
These steps have to perform partial matches on the output to avoid excess verbosity in our feature files and coupling to irrelevant details.
However, working on #1521 I realised that the output of these steps is also quite hard to work with if the expectation fails. They're implemented using PHPUnit's Assert::assertStringContainsString which isn't ideal for this usecase because:
- It can only render the complete actual and expected strings, which in our case are often big blocks of text. If the difference is a single word (or e.g. a missing
.) it's virtually impossible to see without copying the two blocks and pasting into a diffing tool. - It renders explicit
\nfor newlines in the actual content (maybe other special characters) but prints the expected content as-is. As well as making the difference harder to spot, it also makes it difficult to paste the actual into a diff tool or indeed back into the scenario if you're happy with the actual output and just want to update the expectation.
We could improve DX quite a bit by making these assertions produce a more helpful output.
One option would be to experiment with sebastian/diff and see if we can get it to produce a diff which ignores any extra content before or after the expected string. Alternatively we could maybe use str_contains for the actual assertion, and then just show a diff of the whole output if it fails - which would probably still be easier to read than the status quo.