Skip to content

[BUG] Uninitialized pointer access when using --benchmark_repetitions in some scenarios #1406

Description

@Sparta142

Describe the bug
An uninitialized pointer is dereferenced when calculating statistics for a repeated benchmark that fails a certain number of times, which causes a segv/access violation/other mayhem.

This doesn't seem to happen for benchmarks that always fail, nor if you run without --benchmark_repetitions=N.

System
Which OS, compiler, and compiler version are you using:

  • OS: Windows 10 Pro 21H2 (x64)
  • Compiler and version: MSVC 19.29.30141.0

In my experience this happens on various compiler (MSVC, GCC) and execution environments (Windows x64, Linux arm64), as well as in both Debug and Release builds.

To reproduce

#include <benchmark/benchmark.h>

static int count = 0;

static void BM_StringCreation(benchmark::State &state)
{
    if (++count == 3)
    {
        state.SkipWithError("Simulate a flaky benchmark");
        return;
    }

    for (auto _ : state)
        std::string empty_string;
}
BENCHMARK(BM_StringCreation);

int main(int argc, char *argv[])
{
    benchmark::Initialize(&argc, argv);
    benchmark::RunSpecifiedBenchmarks();
    benchmark::Shutdown();
    return 0;
}

Steps to reproduce the behavior:
Compile the given code above on MSVC 19 with benchmark 1.6.1, and run using:

./example.exe --benchmark_repetitions=5

Expected behavior
Benchmark does not crash. Statistics are calculated for only the passing runs.
(If this is incorrect usage of the library, it should be documented somewhere.)

Screenshots
image

Additional context
Looks like the uninitialized pointer is reports[0].statistics:

for (const auto& Stat : *reports[0].statistics) {

It seems to not be initialized here (on line 98) when a run has an error:

if (!report.error_occurred) {
if (b.use_manual_time()) {
report.real_accumulated_time = results.manual_time_used;
} else {
report.real_accumulated_time = results.real_time_used;
}
report.cpu_accumulated_time = results.cpu_time_used;
report.complexity_n = results.complexity_n;
report.complexity = b.complexity();
report.complexity_lambda = b.complexity_lambda();
report.statistics = &b.statistics();
report.counters = results.counters;

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions