Problem

While mining SHA256D, the wallet periodically gets stuck, failing to stay caught up with the chain, even with #915. (At the same time, the mining significantly slows down.)

Root Cause

Lock contention; the cs_main lock is held throughout critical parts of block template creation, done independently per mining thread. The intermittent stuckness appears correlated with large numbers of txins during zerocoin spend validation. This requires determining the height of an accumulator hash, which is done by searching through all the blocks in the chain starting from genesis, for every txin. My debug logs (with -debug=bench) show very consistent examples like - Verify 646 txins: 17392.00ms (26.923ms/txin).

Solution

Since the height of a particular hash-denomination pair is not going to change, we can cache it. One mining thread will still have to find the height to begin with, but all the other threads will not need to. (Also, hashes may persist to be reused in later blocks; there are likely multiple coins of the same denom with the same accumulator hash.)

Adds a SimpleLRUCache template based on 7c9e97a and uses it to store checksum heights. I chose a somewhat arbitrary size for it that should be large enough for significant spends.

This results in a significant speedup of zerocoin spend validation (alternately got - Verify 101 txins: 1.00ms (0.010ms/txin) , - Verify 101 txins: 0.00ms (0.000ms/txin) on one block). Vastly improves #862 alongside #915--I can confirm it still gets stuck without #915.

Bounty PR

#862

Bounty Payment Address

sv1qqpswvjy7s9yrpcmrt3fu0kd8rutrdlq675ntyjxjzn09f965z9dutqpqgg85esvg8mhmyka5kq5vae0qnuw4428vs9d2gu4nz643jv5a72wkqqq73mnxr

Testing

testnet: built with -with-sanitizers=thread -O0 -g and run with 3/4 threads mining sha256d, and spending tons of 10s at once to stress-test this section.
mainnet: running 12 threads sha256d, built with #915 and -O3 and

diff --git a/src/validation.cpp b/src/validation.cpp
index f92427109..a2a2c9ef5 100644
--- a/src/validation.cpp
+++ b/src/validation.cpp              
@@ -2863,0 +2895,2 @@ bool CChainState::ConnectBlock(const CBlock& block, CValidationState& state, CBl
         return state.DoS(100, error("%s: CheckQueue failed", __func__), REJECT_INVALID, "block-validation-failed");

     int64_t nTime4 = GetTimeMicros(); nTimeVerify += nTime4 - nTime2;
+    if (nInputs > 20)
+        LogPrintf("    - Verify %u txins: %.2fms (%.3fms/txin) [%.2fs (%.2fms/blk)]\n", nInputs - 1, MILLI * (nTime4 - nTime2), nInputs <= 1 ? 0 : MILLI * (nTime4 - nTime2) / (nInputs-1), nTimeVerify * MICRO, nTimeVerify * MILLI / nBlocksTotal);
     LogPrint(BCLog::BENCH, "    - Verify %u txins: %.2fms (%.3fms/txin) [%.2fs (%.2fms/blk)]\n", nInputs - 1, MILLI * (nTime4 - nTime2), nInputs <= 1 ? 0 : MILLI * (nTime4 - nTime2) / (nInputs-1), nTimeVerify * MICRO, nTimeVerify * MILLI / nBlocksTotal); 

for simple performance monitoring.