Describe the bug
An uninitialized pointer is dereferenced when calculating statistics for a repeated benchmark that fails a certain number of times, which causes a segv/access violation/other mayhem.
This doesn't seem to happen for benchmarks that always fail, nor if you run without --benchmark_repetitions=N.
System
Which OS, compiler, and compiler version are you using:
- OS: Windows 10 Pro 21H2 (x64)
- Compiler and version: MSVC 19.29.30141.0
In my experience this happens on various compiler (MSVC, GCC) and execution environments (Windows x64, Linux arm64), as well as in both Debug and Release builds.
To reproduce
#include <benchmark/benchmark.h>
static int count = 0;
static void BM_StringCreation(benchmark::State &state)
{
if (++count == 3)
{
state.SkipWithError("Simulate a flaky benchmark");
return;
}
for (auto _ : state)
std::string empty_string;
}
BENCHMARK(BM_StringCreation);
int main(int argc, char *argv[])
{
benchmark::Initialize(&argc, argv);
benchmark::RunSpecifiedBenchmarks();
benchmark::Shutdown();
return 0;
}
Steps to reproduce the behavior:
Compile the given code above on MSVC 19 with benchmark 1.6.1, and run using:
./example.exe --benchmark_repetitions=5
Expected behavior
Benchmark does not crash. Statistics are calculated for only the passing runs.
(If this is incorrect usage of the library, it should be documented somewhere.)
Screenshots

Additional context
Looks like the uninitialized pointer is reports[0].statistics:
|
for (const auto& Stat : *reports[0].statistics) { |
It seems to not be initialized here (on line 98) when a run has an error:
|
if (!report.error_occurred) { |
|
if (b.use_manual_time()) { |
|
report.real_accumulated_time = results.manual_time_used; |
|
} else { |
|
report.real_accumulated_time = results.real_time_used; |
|
} |
|
report.cpu_accumulated_time = results.cpu_time_used; |
|
report.complexity_n = results.complexity_n; |
|
report.complexity = b.complexity(); |
|
report.complexity_lambda = b.complexity_lambda(); |
|
report.statistics = &b.statistics(); |
|
report.counters = results.counters; |
Describe the bug
An uninitialized pointer is dereferenced when calculating statistics for a repeated benchmark that fails a certain number of times, which causes a segv/access violation/other mayhem.
This doesn't seem to happen for benchmarks that always fail, nor if you run without
--benchmark_repetitions=N.System
Which OS, compiler, and compiler version are you using:
In my experience this happens on various compiler (MSVC, GCC) and execution environments (Windows x64, Linux arm64), as well as in both Debug and Release builds.
To reproduce
Steps to reproduce the behavior:
Compile the given code above on MSVC 19 with benchmark 1.6.1, and run using:
Expected behavior
Benchmark does not crash. Statistics are calculated for only the passing runs.
(If this is incorrect usage of the library, it should be documented somewhere.)
Screenshots

Additional context
Looks like the uninitialized pointer is
reports[0].statistics:benchmark/src/statistics.cc
Line 159 in 6d50251
It seems to not be initialized here (on line 98) when a run has an error:
benchmark/src/benchmark_runner.cc
Lines 88 to 99 in 6d50251