Originally posted by @plebhash in #33899

we are getting reports of out-of-memory crashes on stratum-mining/sv2-apps#59 (comment)

initially I suspected there could be thread-related issues similar to what I reported on #33923, but it turns out it was the VPS running out of it's 2GB available RAM, which only happened after long (12h+) running sessions

so I leveraged psrecord to observe how Bitcoin Core (a14e7b9 from 30.x branch) was consuming RAM across time (connected to mainnet for high mempool activity, for about ~1h to have enough chain tip updates)

this is RAM consumption of Bitcoin Core with the Rust code from stratum-mining/sv2-apps#59 (more specifically SRI Pool at stratum-mining/sv2-apps@57576e0) connected to it:

there's a clear upwards trend in RAM consumption, which made me wonder if we were doing something wrong on the Rust code

so then I ran it alongside sv2-tp for a comparison:

which shows a similar upwards trend that never gets throttled down

---

I don't know whether the root cause for this is related to template memory management, but there's a chance it is, so I'm reporting it here rather than opening a new issue

Originally posted by @Sjors in #33899

I'm planning to make similar plots for #33922 (without CPU and ideally with marks to indicate where blocks were found).

If you're measuring the process memory instead of only the template memory (which #33922 enables), you'll want to hold the mempool itself constant. E.g. by picking some value for -maxmempool and waiting for it to fill before starting the measurement. You also want to set -dbcache to its minimum, because that's also accuring

Originally posted by @Sjors in #33899

@plebhash it looks like you found a real bug. Because BlockTemplate::createNewBlock doesn't have a context param, it looks like its destroy method is not invoked until sv2-tp disconnects. So the node keeps holding on to templates even though the Template Provider already pruned them.

I'll open a PR to fix that.

Originally posted by @plebhash in #33899

that's good to know!

but do we have to explicitly call destroy or is it sufficient to drop the reference from memory on the client side?

from my understanding of capnp, I believe it should be sufficient to drop it from memory, but on the other hand there must be a reason for destroy to exist?

Originally posted by @Sjors in #33899

@plebhash IIUC libmultiprocess does this automatically, but only sv2-tp uses that library. So it probably depends on how the rust capnp library is implemented. It might be worth testing how that library behaves out of the box, with and without the fix here. Just look for the IPC server destroy messages on the Bitcoin Core side (preferably tested against master).