-
-
Notifications
You must be signed in to change notification settings - Fork 95
Description
We're using nanobench in Bitcoin Core to track performance regressions.
One of the features I'm missing most is the ability to do untracked pre-calculations or post-calculations for benchmarks that mutate the data (and cannot be trivially repeated).
Preconditions:
- it would be preferable if the vector copying wouldn't be in the timed lambda https://github.com/bitcoin/bitcoin/blob/master/src/bench/merkle_root.cpp#L22.
- block deserialization should be measured separately from validation - https://github.com/bitcoin/bitcoin/blob/master/src/bench/checkblock.cpp#L52-L55
- setting up the conditions to be able to call
ConnectBlockisn't trivial: https://github.com/bitcoin/bitcoin/blob/master/src/bench/connectblock.cpp#L100
Postcondition:
- vector has to be reset at the end: https://github.com/bitcoin/bitcoin/blob/master/src/bench/coin_selection.cpp#L136
- measurements include setting up the next value: https://github.com/bitcoin/bitcoin/blob/master/src/bench/ellswift.cpp#L28
- we have to reset the state to make it measurable again - https://github.com/bitcoin/bitcoin/blob/master/src/bench/checkblock.cpp#L37
Google Benchmark has an explicit PauseTiming / ResumeTiming (https://github.com/google/benchmark/blob/main/docs/user_guide.md#controlling-timers), but that would likely introduce needless complexity.
What would probably suffice in our case would be to have setup and cleanup lambdas as well, in addition to the timed run. Probably either of those could be enough as well, e.g. a setup. I don't mind implementing this myself if you think it's a good idea.