Implement JSON formatter#1674
Conversation
| @@ -0,0 +1,125 @@ | |||
| { | |||
| "$schema": "https://json-schema.org/draft/2020-12/schema", | |||
| "$id": "https://example.com/schemas/test-report.schema.json", | |||
There was a problem hiding this comment.
should this use behat.org instead of example.com ?
acoulton
left a comment
acoulton
left a comment
There was a problem hiding this comment.
This is brilliant, thanks for all the work on this @carlos-granados
A few small comments / questions that I spotted when going through it, but overall it's in great shape.
src/Behat/Behat/Output/Node/EventListener/JSON/JSONElementListener.php
Outdated
Show resolved
Hide resolved
src/Behat/Behat/Output/Node/Printer/JSON/JSONScenarioPrinter.php
Outdated
Show resolved
Hide resolved
src/Behat/Testwork/Output/Printer/Factory/FileOutputFactory.php
Outdated
Show resolved
Hide resolved
|
@acoulton the time fields have been changed to be proper floats rounded to 3 decimals and I updated the field description in the json schema, let me know if it looks ok now |
|
@acoulton updated, thanks for all your suggestions. I also changed a few properties which I saw had been defined as unnecessarily nullable. Please let me know if it looks ok now |
There was a problem hiding this comment.
Great @carlos-granados just one small naming thing in the updated schema and (the discussion about paths. now resolved)
acoulton
left a comment
acoulton
left a comment
There was a problem hiding this comment.
Brilliant, great work @carlos-granados thank you :)
3 participants
Lately JSON has become the lingua franca of tools, which use this format to exchange information everywhere. This PR introduces a JSON formatter which is able to save Behat test results as a JSON file. One possible application for this format is to be able to create an MCP server for Behat, using the generated JSON to return information to the client
This formatter is heavily inspired in the existing JUnit formatter and the exported output is similar but, since it is not constrained by the existing JUnit specification, it follows more closely the structure of our project. The generated JSON basically will contain a list of Suite nodes, which will contain Feature nodes, which will contain Scenario nodes. Each node will have properties indicating some info like the status, the number of tests for each status, file and line and execution time. Also, the JUnit formatter produced one file per suite, this formatter produces just a single JSON file for all suites so the
output_pathneeds to be a file name, not a folder nameThe PR also defines a JSON schema which can be used by the tools that will use this information