-
Notifications
You must be signed in to change notification settings - Fork 1.2k
perf: temporary cache for consequent calls of CDeterministicMNList for historical blocks #6734
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
WalkthroughAdded #include <node/blockstorage.h> and an in-memory mini-snapshot caching step inside CDeterministicMNManager::GetListForBlockInternal (src/evo/deterministicmns.cpp). After applying each cached diff to the snapshot, when not reindexing (fReindex == false) and the snapshot height is a multiple of 32 (MINI_SNAPSHOT_INTERVAL = 32), the code inserts the current snapshot into mnListsCache keyed by snapshot.GetBlockHash(). No changes to exported/public declarations; existing snapshot/diff loading and other caching logic remain unchanged. Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Tip 🔌 Remote MCP (Model Context Protocol) integration is now available!Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats. 📜 Recent review detailsConfiguration used: CodeRabbit UI 💡 Knowledge Base configuration:
You can enable these sources in your CodeRabbit configuration. 📥 CommitsReviewing files that changed from the base of the PR and between 2896e2b507b61ffde410c263e50ea4e40a1317bd and 366cd2b. 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
|
at first, I haven't planned to get this one merged, because I assumed that it improves only |
4de63d1 to
30a9b2e
Compare
src/evo/deterministicmns.cpp
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Consider Making Cache Interval Configurable
- The hardcoded value 32 could be a named constant or configurable parameter
static constexpr int MINI_SNAPSHOT_INTERVAL = 32;
if (snapshot.GetHeight() % MINI_SNAPSHOT_INTERVAL == 0) {
|
Consider:
|
diff --git a/src/evo/deterministicmns.cpp b/src/evo/deterministicmns.cpp
index 921a119642..3f2dab1d3e 100644
--- a/src/evo/deterministicmns.cpp
+++ b/src/evo/deterministicmns.cpp
@@ -16,6 +16,7 @@
#include <consensus/validation.h>
#include <deploymentstatus.h>
#include <messagesigner.h>
+#include <node/blockstorage.h>
#include <script/standard.h>
#include <stats/client.h>
#include <uint256.h>
@@ -26,6 +27,8 @@
#include <optional>
#include <memory>
+using node::fReindex;
+
static const std::string DB_LIST_SNAPSHOT = "dmn_S3";
static const std::string DB_LIST_DIFF = "dmn_D3";
@@ -780,7 +783,7 @@ CDeterministicMNList CDeterministicMNManager::GetListForBlockInternal(gsl::not_n
for (const auto& diffIndex : listDiffIndexes) {
const auto& diff = mnListDiffsCache.at(diffIndex->GetBlockHash());
snapshot.ApplyDiff(diffIndex, diff);
- if (snapshot.GetHeight() % 32 == 0) {
+ if (!fReindex && (snapshot.GetHeight() % 32 == 0)) {
// Add this temporary mini-snapshot to cache.
// This extra cached mn-list helps to improve performance of GetListForBlock
// for close blocks, because in the worst cases each of them requires to retrieve
Agree
Adding unit tests here feels like an overkill tbh |
✅ No Merge Conflicts DetectedThis PR currently has no conflicts with other open PRs. |
I declared named const
It's a bit overkill IMO. considerable, possible to implement
Implementation of cache cleanup is intact; every cleanup() call called by scheduler will remove this mini-snapshots. They help to improve performance for 50% of calculation
24 is a theoretically calculated number for The best theoretical performance will have |
Nevermind, I already solved it by #6728, which is merged. |
…r historical blocks It speeds ups blocks's Undo up to 10x. It speeds up RPC `protx diff` up to 9x
589587c to
2896e2b
Compare
2896e2b to
366cd2b
Compare
UdjinM6
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, utACK 366cd2b
Issue being fixed or feature implemented
Each call of
CDeterministicMNManager::GetListForBlockfor historical block can require up to 575 diff applying from snapshot.Consequent calls for close blocks repeat this calculation twice.
What was done?
This PR abuses
mnListsCacheby adding mini-snapshot each 32 blocks between snapshots in database (each 576 blocks).Downside of this solution: extra RAM usage.
Though, this cache is cleaned every 10 seconds by
CDeterministicMNManager::DoMaintenance, so, the RAM usage is temporary.How Has This Been Tested?
It speeds up RPC
protx diffup to 8x.develop:PR:
It speeds ups blocks's Undo up to 10x; measured by calling
invalidateblock blockhashwhere blockhash is distant block, far from the tip (500+).Breaking Changes
N/A
Checklist: