Skip to content

Commit c29e198

Browse files
horenmarhorenmar
committed
Improve performance of JUnit reporter when handling passing assertions
Given `TEST_CASE` that just runs many passing assertions, e.g. ``` TEST_CASE( "PERFORMANCE_TEST_CASE", "[.]" ) { for (size_t i = 0; i < 1000'000'000; ++i) { REQUIRE( i == i ); } } ``` the new JUnit reporter will finish in 5:47, using up ~7.7 MB of RAM. The old JUnit reporter would finish in 0:30, due to bad_alloc after using up 14.5 GB of RAM (the system has 16 GB total). If the total number of assertions is lowered to 10M, the old implementation uses up ~ 7.1 GB of RAM and finishes in 12 minutes. The new implementation still needs ~7.7 MB of RAM, and finishes in 4 minutes. There is a slight downside in that the output is slightly different; the new implementation will include the `TEST_CASE` level section in output, even if it does not have its own assertion. In other words, given a `TEST_CASE` like this ``` TEST_CASE( "JsonWriter", "[JSON][JsonWriter]" ) { std::stringstream stream; SECTION( "Newly constructed JsonWriter does nothing" ) { Catch::JsonValueWriter writer{ stream }; REQUIRE( stream.str() == "" ); } SECTION( "Calling writeObject will create an empty pair of braces" ) { { auto writer = Catch::JsonValueWriter{ stream }.writeObject(); } REQUIRE( stream.str() == "{\n}" ); } } ``` the new implementation will output 3 `testcase` tags, 2 for the explicit `SECTION`s with tests, and 1 for the top level section. However, this can be worked-around if required, and the performance improvement is such that it is worth changing the current output, even if it needs to be fixed in the future. Closes #2897
1 parent 8898cc6 commit c29e198

File tree

3 files changed

+326
-2
lines changed

3 files changed

+326
-2
lines changed

‎src/catch2/reporters/catch_reporter_junit.cpp‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -88,7 +88,7 @@ namespace Catch {
8888
xml( m_stream )
8989
{
9090
m_preferences.shouldRedirectStdOut = true;
91-
m_preferences.shouldReportAllAssertions = true;
91+
m_preferences.shouldReportAllAssertions = false;
9292
m_shouldStoreSuccesfulAssertions = false;
9393
}
9494

@@ -198,7 +198,7 @@ namespace Catch {
198198
if( !rootName.empty() )
199199
name = rootName + '/' + name;
200200

201-
if( sectionNode.hasAnyAssertions()
201+
if ( sectionNode.stats.assertions.total() > 0
202202
|| !sectionNode.stdOut.empty()
203203
|| !sectionNode.stdErr.empty() ) {
204204
XmlWriter::ScopedElement e = xml.scopedElement( "testcase" );

0 commit comments

Comments
 (0)