-
-
Notifications
You must be signed in to change notification settings - Fork 3.1k
PR 1091 generates changes in junitxml output which cause incompatibility with Jenkins Junit plugin #2228
Copy link
Copy link
Closed
Labels
plugin: junitxmlrelated to the junitxml builtin pluginrelated to the junitxml builtin plugintype: backward compatibilitymight present some backward compatibility issues which should be carefully noted in the changelogmight present some backward compatibility issues which should be carefully noted in the changelogtype: enhancementnew feature or API change, should be merged into features branchnew feature or API change, should be merged into features branchtype: feature-branchnew feature or API change, should be merged into features branchnew feature or API change, should be merged into features branchtype: regressionindicates a problem that was introduced in a release which was working previouslyindicates a problem that was introduced in a release which was working previously
Metadata
Metadata
Assignees
Labels
plugin: junitxmlrelated to the junitxml builtin pluginrelated to the junitxml builtin plugintype: backward compatibilitymight present some backward compatibility issues which should be carefully noted in the changelogmight present some backward compatibility issues which should be carefully noted in the changelogtype: enhancementnew feature or API change, should be merged into features branchnew feature or API change, should be merged into features branchtype: feature-branchnew feature or API change, should be merged into features branchnew feature or API change, should be merged into features branchtype: regressionindicates a problem that was introduced in a release which was working previouslyindicates a problem that was introduced in a release which was working previously
Let's consider testcase that fails in the call thus resulting in 1 failure
and then when it's teardown fixture is executed it fails there as well
thus resulting in 1 error. So we have 1 failure and 1 error for the
same test. This results in the junitxml XML report, as two different
attributes, one error and one failure for the same testcase.
However, until pytest 2.8.4, in such tests the XML file had two
testcase elements, one containing the failure in the call of the test,
and a second testcase element containing the error in the teardown.
From pytest 2.8.5 and after the XML file has only one testcase element
for such test, and this contains both fail and error sections.
The problem with the new XML files is that they cannot
be correctly parsed by Jenkins Junit plugin. The reason behind that is
that they violate the JUnit XML format, which is built so that in
every testcase element you can have as a child only one of {skipped,
error, failure, system-out, system-err} elements.
I would like your opinion on that, and if you agree on reverting the old style
XML reports I would be happy to send a PR.
@RonnyPfannschmidt
EXAMPLE:
The following code will have difference in the XML report between
pytest versions.
pytest version >= 2.4.5
pytest version < 2.4.5