-
Notifications
You must be signed in to change notification settings - Fork 0
Merge bitcoin/bitcoin#29434: rpc: Fixed signed integer overflow for large feerates #701
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merge bitcoin/bitcoin#29434: rpc: Fixed signed integer overflow for large feerates #701
Conversation
Going through the vin and then adding the vin index as a vout position shouldn't work, especially if there are an unequal number of inputs and outputs but the test transactions tend to have spend patterns that allow the code to inadvertently work, evading detection. A new test has been added that uses a spend pattern that would've netted unexpected behavior in the previous logic. Also, we can just use `AddChildrenToWorkSet` for this because it effectively implements the same logic (going through vout, finding and inserting).
Both functions that manipulate it, `AddTx()` and `LimitOrphans()` hold the mutex.
This commit will not compile as-is as block based orphan reprocessing, a Dash-ism, has not been adapted to work with this refactor. That has been segmented out into a separate commit to aid in review.
|
Important Review skippedMore than 25% of the files skipped due to max files limit. The review is being skipped to prevent a low-quality review. 126 files out of 300 files are above the max files limit of 100. Please upgrade to Pro plan to get higher limits. You can disable this status message by setting the ✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
✅ Backport Verification SuccessfulNo issues found! This backport is complete and faithful to the original Bitcoin change. Original Bitcoin commit: Analysis SummaryThis is an excellent backport that correctly adapts Bitcoin's fee rate overflow fix to Dash's architecture: ✅ ParseFeeRate function added - Properly implemented in Technical Notes
✅ Range-diff1: fa0ff66109 < -: ---------- rpc: Implement RPCHelpMan::ArgValue<> for UniValue
2: fade94d11a < -: ---------- rpc: Add ParseFeeRate helper
3: fa2a4fdef7 \! 1: 05b09bfede rpc: Fixed signed integer overflow for large feerates
@@
## Metadata ##
-Author: MarcoFalke <*~=`'#}+{/-|&$^[email protected]>
+Author: Claude Code <[email protected]>
## Commit message ##
- rpc: Fixed signed integer overflow for large feerates
+ Merge bitcoin/bitcoin#29434: rpc: Fixed signed integer overflow for large feerates
- ## src/rpc/mempool.cpp ## [Bitcoin location]
+ ## src/rpc/rawtransaction.cpp ## [Dash location]
@@ Correctly adapted function locations
## ParseFeeRate implementation added to util.cpp/util.h ##
+CFeeRate ParseFeeRate(const UniValue& json)
+{
+ CAmount val{AmountFromValue(json)};
+ if (val >= COIN) throw JSONRPCError(RPC_INVALID_PARAMETER, "Fee rates larger than or equal to 1BTC/kvB are not accepted");
+ return CFeeRate{val};
+}This PR is ready for merge. ✅ |
…n `interface_rest`
…et_createwallet
…erheaders` (REST)
…ot calculation eeb577b perf: swap arrays instead assigments to save allocation (Konstantin Akimov) 8254e06 refactor: resolve freshly introduced circular dependency (Konstantin Akimov) 8958152 perf: cache mined commitment for quorum merkle root calculation (Konstantin Akimov) f2b2479 refactor: avoid code duplication due to 2 implementation of GetMinedCommitment (Konstantin Akimov) d78fb02 perf: replace re-creation of CFinalCommitment and extra unique_ptr to RVO (Konstantin Akimov) 6d90189 refactor: futher simplification of GetMinedAndActiveCommitmentsUntilBlock and related code (Konstantin Akimov) 40f3d98 refactor: simplify GetLastMinedCommitmentsPerQuorumIndexUntilBlock interface (Konstantin Akimov) Pull request description: ## Issue being fixed or feature implemented Every time when list of quorum is changed, the quorum final commitments for multiple active quorums are re-read from database. It's a heavy operation in a current implementation due to heavy database reading, heavy BLS deserialization and multiple in-memory copies. ## What was done? - add cache for active quorums, see `llmq::utils::InitQuorumsCache(qc_hashes_cached)` - use RVO for GetMinedCommitment, avoid extra allocation by unique_ptr - adjust return value of GetLastMinedCommitmentsPerQuorumIndexUntilBlock to avoid extra data copies and transformations ## How Has This Been Tested? Time of calculation `CheckCbTxMerkleRoots` is shortened for 40%, and this step is a significant part of block validation. Expected impact to total time for block validation and re-index is 3-4%. `develop`: ``` [bench] - CachedGetQcHashesQcIndexedHashes: 1.16ms [5.22s] [bench] - CheckCbTxMerkleRoots: 1.30ms [6.04s] ``` vs PR: ``` [bench] - CachedGetQcHashesQcIndexedHashes: 1.20ms [2.65s] [bench] - CheckCbTxMerkleRoots: 1.34ms [3.44s] ``` ## Breaking Changes N/A ## Checklist: - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have added or updated relevant unit/integration/functional/e2e tests - [ ] I have made corresponding changes to the documentation - [x] I have assigned this pull request to a milestone ACKs for top commit: UdjinM6: utACK eeb577b Tree-SHA512: f55dc835dcd536d37e6b3b4fa4854902c0dd49044f60f8ec0aca95cd497a9eb52c9664eb713957b5bba2c82b69a4993e7c357ae799d36b923cb55fe735ce3c9f
dash#6425 moved AskNodesForLockedTx to PeerManager as AskPeersForTransaction, which removed the only usage of IsInvInFilter. We can safely remove it from the PeerManager interface as its usage is now purely internal.
…w format with `nVersion` being the first bit to read 4d9d8f7 chore: update release notes (UdjinM6) 1c50949 refactor: `uint256()` -> `uint256{}` (UdjinM6) f369673 fix: adjust ser/deser condition for netInfo (UdjinM6) efea931 fix: do not trust legacy diffs, detect correct `nVersion` on migration (UdjinM6) 80906fa fix: apply review suggestions (UdjinM6) 8832a3b docs: add release notes (UdjinM6) f6a4f01 test: add test cases for bit mapping and migration deser logic (UdjinM6) ac6ea47 feat: enforce `nVersion` bit for `pubKeyOperator` and `netInfo` bits in new format (UdjinM6) f7963d4 feat: implement EvoDB migration logic (UdjinM6) bdb0d00 feat: move `nVersion` from `0x40000` to `0x0001` (first position) in `CDeterministicMNStateDiff` (UdjinM6) 733cdbc feat: introduce `CDeterministicMNStateDiffLegacy` class with original field positions and conversion to new format (UdjinM6) Pull request description: ## Issue being fixed or feature implemented `nVersion` is needed for `pubKeyOperator` and `netInfo` diffs. Let's fix bit order and streamline the logic instead of adding workarounds and magic numbers (dashpay#6666). Migration takes ~1m30s~ 3-5 minutes on mainnet on my machine. ## What was done? Changes: - Introduced `CDeterministicMNStateDiffLegacy` class to maintain backward compatibility with existing data while enabling migration to the optimized format - Moved `nVersion` field from position `0x40000` to `0x0001` (first position) in `CDeterministicMNStateDiff` - Implemented migration logic to handle the transition from legacy to new masternode state format - Implemented stricter validation for `pubKeyOperator` and `netInfo` bits in the new serialization format - Added new test cases covering legacy-to-new bit mapping and migration deserialization logic ## How Has This Been Tested? - Run tests. - Backup datadir, run a node, check logs ## Breaking Changes Needs reindex on downgrades ## Checklist: - [ ] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have added or updated relevant unit/integration/functional/e2e tests - [ ] I have made corresponding changes to the documentation - [ ] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_ ACKs for top commit: PastaPastaPasta: utACK 4d9d8f7 kwvg: utACK 4d9d8f7 Tree-SHA512: f62a4741ccebe9725caa89041306d48d6dbeaffa2df2e3ea9d9476106cb131678a70fe3189e0c2a6086ab5074de8e25c220378961fc6c4dec8c98cd91d758f10
…ime logging f9536e8 fmt: applyed clang format for blockprocessor (Konstantin Akimov) a04e831 feat: use seconds since 1970 for mocktime in logs (Konstantin Akimov) 3458ceb feat: remove useless checks after UpdateMN (Konstantin Akimov) 1abe753 refactor: remove useless logs from CQuorumBlockProcessor and make its member `IsMiningPhase` be anonymous function (Konstantin Akimov) 6a53aa2 feat: stop logging 'remove mapSendableNodes' and 'remove mapReceivableNodes' which triggers on each send / receive bytes (Konstantin Akimov) Pull request description: ## Issue being fixed or feature implemented I have been lately watching to debug logs really a lot while investigating various flackiness in functional tests, and there are some adjustment that I found as useful. ## What was done? - removed `remove mapSendableNodes` an `remove mapReceivableNodes` lines that are triggered for almost each received network package There's still plenty logs about network connection, but that particular line is useless. ``` node0 2025-06-03T06:50:11.185505Z (mocktime: 2014-12-04T17:15:48Z) [ net] [net.cpp:2029] [CreateNodeFromAcceptedSocket] [net|netconn] connection accepted, sock=46, peer=0 [REMOVED] node0 2025-06-03T06:50:11.185760Z (mocktime: 2014-12-04T17:15:48Z) [ net] [net.cpp:2798] [SocketHandlerConnected] [net] SocketHandlerConnected -- remove mapReceivableNodes, peer=0 node0 2025-06-03T06:50:11.185877Z (mocktime: 2014-12-04T17:15:48Z) [ msghand] [net_processing.cpp:3518] [ProcessMessage] [net] received: version (148 bytes) peer=0 node0 2025-06-03T06:50:11.185927Z (mocktime: 2014-12-04T17:15:48Z) [ msghand] [net.cpp:4967] [PushMessage] [net] sending version (148 bytes) peer=0 node0 2025-06-03T06:50:11.185959Z (mocktime: 2014-12-04T17:15:48Z) [ msghand] [net_processing.cpp:1468] [PushNodeVersion] [net] send version message: version 70237, blocks=113, txrelay=1, peer=0 node0 2025-06-03T06:50:11.185977Z (mocktime: 2014-12-04T17:15:48Z) [ msghand] [net.cpp:4967] [PushMessage] [net] sending sendaddrv2 (0 bytes) peer=0 node0 2025-06-03T06:50:11.186004Z (mocktime: 2014-12-04T17:15:48Z) [ msghand] [net.cpp:4967] [PushMessage] [net] sending verack (0 bytes) peer=0 ``` - removed multiple lines from CQuorumBlockProcessor which are logged for each block but doesn't really help to understand why any quorum is failed to form, not bringing any useful input. Also `CQuorumBlockProcessor::ProcessCommitment` is simplified to just ProcessCommitment ``` [REMOVED] node0 2025-06-03T06:50:08.720090Z (mocktime: 2014-12-04T17:15:38Z) [httpworker.1] [llmq/blockprocessor.cpp:412] [IsMiningPhase] [llmq] [IsMiningPhase] nHeight[2] llmqType[100] quorumCycleStartHeight[0] -- NOT mining[10-18] [REMOVED] node0 2025-06-03T06:50:08.720101Z (mocktime: 2014-12-04T17:15:38Z) [httpworker.1] [llmq/blockprocessor.cpp:412] [IsMiningPhase] [llmq] [IsMiningPhase] nHeight[2] llmqType[103] quorumCycleStartHeight[0] -- NOT mining[12-20] [REMOVED] node0 2025-06-03T06:50:08.720112Z (mocktime: 2014-12-04T17:15:38Z) [httpworker.1] [llmq/blockprocessor.cpp:412] [IsMiningPhase] [llmq] [IsMiningPhase] nHeight[2] llmqType[106] quorumCycleStartHeight[0] -- NOT mining[10-18] [OLD] node0 2025-06-03T06:50:13.768237Z (mocktime: 2014-12-04T17:15:56Z) [httpworker.1] [llmq/blockprocessor.cpp:236] [ProcessCommitment] [llmq] CQuorumBlockProcessor::ProcessCommitment height=114, type=100, quorumIndex=0, quorumHash=224adea5fe8b95219555319e09cc4509ff550343b306a74fcf57dac9c380c6f2, signers=0, validMembers=0, quorumPublicKey=000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 fJustCheck[1] processing commitment from block. [NEW] node0 2025-06-03T06:51:07.902742Z (mocktime: 1417713341) [httpworker.1] [llmq/blockprocessor.cpp:250] [ProcessCommitment] [llmq] ProcessCommitment -- height=34, type=100, quorumIndex=0, quorumHash=793bfcce397cc0ab4a685f4fe2d2a4b71ce06402813ea1f55e626ad2f1acbe9b, signers=0, validMembers=0, quorumPublicKey=000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 fJustCheck[1] processing commitment from block. ``` - removed logs for consequent masternode payments (which is always 0 since mn_rr activated) ``` [REMOVED] node0 2025-06-03T06:50:20.938625Z (mocktime: 2014-12-04T17:16:09Z) [httpworker.0] [evo/deterministicmns.cpp:977] [BuildNewListFromBlock] [mnpayments] CDeterministicMNManager::BuildNewListFromBlock -- MN 8e26622b3336527f126324c71fb169b9be741ed5351832c28007571cbe864b28, nConsecutivePayments=0 ``` - mocktime is shown now in seconds since 1970. Comparing strings `2014-12-04T17:15:37Z` and `2014-12-04T17:15:38Z` I find difficult to do, especially if mocktime bumped for more than 1 second or multiple times; it requires to do some math also over module 60 (60 seconds in minute). ``` [OLD] node0 2025-06-03T06:47:49.402583Z (mocktime: 2014-12-04T17:15:37Z) [httpworker.3] [rpc/request.cpp:180] [parse] [rpc] ThreadRPCServer method=setmocktime user=__cookie__ [OLD] node0 2025-06-03T06:47:49.402632Z (mocktime: 2014-12-04T17:15:38Z) [httpworker.3] [httprpc.cpp:93] [~RpcHttpRequest] [bench] HTTP RPC request handled: user=__cookie__ command=setmocktime external=false status=200 elapsed_time_ms=0 [NEW] node0 2025-06-03T06:46:35.684642Z (mocktime: 1417713337) [httpworker.2] [rpc/request.cpp:180] [parse] [rpc] ThreadRPCServer method=setmocktime user=__cookie__ [NEW] node0 2025-06-03T06:46:35.684674Z (mocktime: 1417713338) [httpworker.2] [httprpc.cpp:93] [~RpcHttpRequest] [bench] HTTP RPC request handled: user=__cookie__ command=setmocktime external=false status=200 elapsed_time_ms=0 ``` ## How Has This Been Tested? It makes reading by eyes much more friendly because less items to read and lines a bit shorter, and less matches by block hash / protx hashes when you are looking for some particular records. As positive side effect, total size of all logs of all functional tests is decreased for ~200Mb (1.7Gb -> 1.5Gb). ## Breaking Changes N/A ## Checklist: - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have added or updated relevant unit/integration/functional/e2e tests - [ ] I have made corresponding changes to the documentation - [x] I have assigned this pull request to a milestone ACKs for top commit: PastaPastaPasta: utACK f9536e8 UdjinM6: utACK f9536e8 Tree-SHA512: 3f48a2f00e8b120e41301e258cda153cf2e45c658841ac17f48f10826d6ffbf3a3009abeced7fc2085eb9714976496e1c5bdc5c754cfd22378937393825c5abc
…uggestions, ChainLock and InstantSend refactoring 34a90e6 chore: remove unused `IsInvInFilter` from `PeerManager` interface (Kittywhiskers Van Gogh) f3224ae refactor: consolidate `INPUTLOCK_REQUESTID_PREFIX` usage to `lock.cpp` (Kittywhiskers Van Gogh) 7b4ee6b trivial: document transaction confirmation safety threshold (Kittywhiskers Van Gogh) 25f05c1 refactor: make unknown block clsig flow easier to follow (Kittywhiskers Van Gogh) 9578146 refactor: document `pindex` assumptions in chainlocks code (Kittywhiskers Van Gogh) 4a744c7 refactor: use `std::chrono` for time variables, reduce resolution (Kittywhiskers Van Gogh) b051c22 refactor: consolidate `CLSIG_REQUESTID_PREFIX` usage to `clsig.cpp` (Kittywhiskers Van Gogh) 024b466 chore: move lock annotations in `chainlock.h` to the next line (Kittywhiskers Van Gogh) c6e99fb chore: apply most `clang-format` suggestions (Kittywhiskers Van Gogh) Pull request description: ## Additional Information * Depends on dashpay#6761 * Dependency for dashpay#6821 * Assumptions surrounding `pindex` usage have been documented in response to reviewer comments in [dash#6761](dashpay#6761) ([comment](dashpay#6761 (comment)), [comment](dashpay#6761 (comment))). * The internal structures of `CChainLocksHandler` (`txFirstSeenTime`, `seenChainLocks`, `lastCleanupTime`) have their time resolution reduced from milliseconds to seconds while migrating to `std::chrono`. * `CInstantSendManager::AskNodesForLockedTx()` was moved as `PeerManagerImpl::AskPeersForTransaction()` in [dash#6425](dashpay#6425) and with that, the sole external usage of `PeerManagerImpl::IsInvInFilter()` was moved internally, we can therefore safely remove it from the `PeerManager` interface. ## Breaking Changes None expected. ## Checklist - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have added or updated relevant unit/integration/functional/e2e tests **(note: N/A)** - [x] I have made corresponding changes to the documentation **(note: N/A)** - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_ ACKs for top commit: UdjinM6: utACK 34a90e6 Tree-SHA512: 28b532cb5b8b5e6d7c1331c7c6c0ecd4d45b186922c279db6d2d3e8974d422ec8b67d75aeadce77986d409a8ed071e85359ee08609e0c2dde657e4520c546817
…terministicMNList for historical blocks 366cd2b refactor: clarify logic and add a named constant MINI_SNAPSHOT_INTERVAL (Konstantin Akimov) bfca726 perf: do not create mini-snapshots during re-index (Konstantin Akimov) 38513b6 perf: temporary cache for consequent calls of CDeterministicMNList for historical blocks (Konstantin Akimov) Pull request description: ## Issue being fixed or feature implemented Each call of `CDeterministicMNManager::GetListForBlock` for historical block can require up to 575 diff applying from snapshot. Consequent calls for close blocks repeat this calculation twice. ## What was done? This PR abuses `mnListsCache` by adding mini-snapshot each 32 blocks between snapshots in database (each 576 blocks). Downside of this solution: extra RAM usage. Though, this cache is cleaned every 10 seconds by `CDeterministicMNManager::DoMaintenance`, so, the RAM usage is temporary. ## How Has This Been Tested? It speeds up RPC `protx diff` up to 8x. `develop`: ``` $ time ( for j in $(seq 500) ; do src/dash-cli protx diff $((2121000+$j)) $((2121000+$j+1)) ; done ) > /dev/null real 0m47,743s user 0m0,472s sys 0m1,467s ``` PR: ``` $ time ( for j in $(seq 500) ; do src/dash-cli protx diff $((2121000+$j)) $((2121000+$j+1)) ; done ) > /dev/null real 0m6,032s user 0m0,423s sys 0m1,300s ``` It speeds ups blocks's Undo up to 10x; measured by calling `invalidateblock blockhash` where blockhash is distant block, far from the tip (500+). ## Breaking Changes N/A ## Checklist: - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have added or updated relevant unit/integration/functional/e2e tests - [ ] I have made corresponding changes to the documentation - [x] I have assigned this pull request to a milestone ACKs for top commit: UdjinM6: LGTM, utACK 366cd2b Tree-SHA512: 1880783916394b6f45a0e71dc59c454766bff498df62cc4642dfa4e8a53736de2ac7c180397584572b4c21a651f65b26cb832c7e4cf2d4b9599556f58bda621e
It introduces new commandline argument -parbls to set up amount of parallel threads for BLS validation New parallel BlsChecker validates asynchonously quorumSig and membersSig in Quorum Commitment
…g 2 quorums only and related test changes 56711d9 test: flip request-id for asset-unlock-tx instead generation new quorums to get a favour situation (Konstantin Akimov) 4a7937d test: re-order logic related to IS in feature_asset_locks.py (Konstantin Akimov) 825dfdd test: activate v23 earlier (block 750 instead 1050) (Konstantin Akimov) a039345 test: activate mn_rr in feature_asset_locks.py for better performance (Konstantin Akimov) 75de576 feat: remove pre-withdrawals fork logic for quorum expiration (Konstantin Akimov) Pull request description: ## Issue being fixed or feature implemented Fork `withdrawals` is activated long time ago. No more need to test functionality for pre-'withdrawals' fork when only 2 quorum could be used for signing messages. ## What was done? Removed pre-'withdrawals' logic to enforce using only 2 quorums for signing of withdrawals from code and from functional test `feature_asset_locks.py` Functional test `feature_asset_locks.py` generate less quorums and less blocks which significantly shortened runtime of this test, 166->139seconds on median; up to 40 seconds if quorum has been generated non-favourable way. ## How Has This Been Tested? Run in parallel 20 jobs - PR: ```TEST | STATUS | DURATION feature_asset_locks.py | ✓ Passed | 133 s feature_asset_locks.py | ✓ Passed | 134 s feature_asset_locks.py | ✓ Passed | 134 s feature_asset_locks.py | ✓ Passed | 134 s feature_asset_locks.py | ✓ Passed | 135 s feature_asset_locks.py | ✓ Passed | 135 s feature_asset_locks.py | ✓ Passed | 137 s feature_asset_locks.py | ✓ Passed | 138 s feature_asset_locks.py | ✓ Passed | 138 s feature_asset_locks.py | ✓ Passed | 139 s feature_asset_locks.py | ✓ Passed | 139 s feature_asset_locks.py | ✓ Passed | 140 s feature_asset_locks.py | ✓ Passed | 140 s feature_asset_locks.py | ✓ Passed | 140 s feature_asset_locks.py | ✓ Passed | 141 s feature_asset_locks.py | ✓ Passed | 141 s feature_asset_locks.py | ✓ Passed | 142 s feature_asset_locks.py | ✓ Passed | 142 s feature_asset_locks.py | ✓ Passed | 143 s feature_asset_locks.py | ✓ Passed | 143 s ALL | ✓ Passed | 2768 s (accumulated) Runtime: 143 s ``` - develop ``` TEST | STATUS | DURATION feature_asset_locks.py | ✓ Passed | 151 s feature_asset_locks.py | ✓ Passed | 157 s feature_asset_locks.py | ✓ Passed | 160 s feature_asset_locks.py | ✓ Passed | 161 s feature_asset_locks.py | ✓ Passed | 161 s feature_asset_locks.py | ✓ Passed | 163 s feature_asset_locks.py | ✓ Passed | 164 s feature_asset_locks.py | ✓ Passed | 165 s feature_asset_locks.py | ✓ Passed | 166 s feature_asset_locks.py | ✓ Passed | 166 s feature_asset_locks.py | ✓ Passed | 167 s feature_asset_locks.py | ✓ Passed | 167 s feature_asset_locks.py | ✓ Passed | 170 s feature_asset_locks.py | ✓ Passed | 171 s feature_asset_locks.py | ✓ Passed | 172 s feature_asset_locks.py | ✓ Passed | 174 s feature_asset_locks.py | ✓ Passed | 181 s feature_asset_locks.py | ✓ Passed | 185 s feature_asset_locks.py | ✓ Passed | 197 s feature_asset_locks.py | ✖ Failed | 49 s ALL | ✖ Failed | 3247 s (accumulated) Runtime: 197 s ``` ## Breaking Changes Removed pre-'withdrawals' requirement of using only 2 last quorums ## Checklist: _Go over all the following points, and put an `x` in all the boxes that apply._ - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have added or updated relevant unit/integration/functional/e2e tests - [ ] I have made corresponding changes to the documentation - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_ ACKs for top commit: UdjinM6: light ACK 56711d9 Tree-SHA512: 779194997cff3a00626b0aba8f7026c8c4528f9327f49aa1c8cfb1cad8fa285c881517545951a2cd36d01fdcf928e2d5be2fc65fa8a915678c06f2b9f1fed2c3
Co-authored-by: UdjinM6 <[email protected]>
Blocks could have 0, 2 or 32 commitments currently; further benchmarking is needed to find a point where is a balance point, but likely it's somewhere between 32 and 64; because each quorum commitment have 2 BLS signatures
…ignatures in quorum commitments 7733740 fix: add missing log for case of no public key is provided (Konstantin Akimov) 43ed80d fix: increased "max amount of threads" for bls check from 31 to 33 (Konstantin Akimov) 2d4fe1b fix: set proper state after async signatures validation in block processor (Konstantin Akimov) 8530c92 fix: replace strings Bls to BLS in command line help (Konstantin Akimov) 7a86e19 refactor: drop unused flag from ProcessCommitment (Konstantin Akimov) 0f3a8e4 feat: add -parbls and validate async quorumSig and membersSig in QC (Konstantin Akimov) 76e4e72 refactor: split quorum commitment verification and its signatures (Konstantin Akimov) Pull request description: ## Issue being fixed or feature implemented During blocks validation, quorum commitments are processed in single thread. While some blocks have up to 32 commitments (blocks which have rotation quorums commitment), each quorum commitment have hundreds public keys to validate and 2 signature (quorum signature and public key signature). It takes up to 30% in total indexing time and up to 1 second for heavy blocks. ## What was done? `CCheckQueue` which is used for validation ECDSA signatures is used now for validation of BLS signatures in quorum commitments. Quorum signature and members signatures are validated simultaneously now which makes performance improvement even for blocks which has only 1 commitment. ## How Has This Been Tested? Invalidated + reconsidered 15k blocks (~1 months worth) This PR makes validation of Quorum Commitment 3.5x times faster; overall indexing 25% faster on my 12 cores environment. PR: <img width="770" alt="image" src="https://github.com/user-attachments/assets/6f1d9815-e8f1-4294-a376-ebbfd7a312c9" /> ``` 2025-05-28T10:17:56Z [bench] - m_qblockman: 0.03ms [28.90s] 2025-05-28T10:17:56Z [bench] - Connect total: 9.01ms [184.16s (11.86ms/blk)] 2025-05-28T10:17:56Z [bench] - Connect block: 9.21ms [190.33s (12.25ms/blk)] ``` develop: <img width="770" alt="image" src="https://github.com/user-attachments/assets/af7e5379-b3ce-4df0-ab90-f34649011f46" /> ``` 2025-05-22T18:39:44Z [bench] - m_qblockman: 0.03ms [96.90s] 2025-05-22T18:39:44Z [bench] - Connect total: 9.31ms [252.80s (16.28ms/blk)] 2025-05-22T18:39:44Z [bench] - Connect block: 9.50ms [258.90s (16.67ms/blk)] ``` ## Breaking Changes N/A ## Checklist: - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have added or updated relevant unit/integration/functional/e2e tests - [x] I have made corresponding changes to the documentation - [x] I have assigned this pull request to a milestone ACKs for top commit: UdjinM6: utACK 7733740 Tree-SHA512: 59de70d645fa7be2fcf9978d46a36bf4cea83654152c76671014b0508271f8adf11a0610aa7f2eda7f3937d932075eef45ecd06887d8d4d751d2637a52c7062e
… for constituent hashes, apply stronger optimizations 44eebe7 chore: resolve `constVariable` linter error (Kittywhiskers Van Gogh) 30bf19d crypto: isolate sphlib sources and apply stronger optimizations to it (Kittywhiskers Van Gogh) 3ec483e bench: more final proof of work benchmark to `pow_hash` (Kittywhiskers Van Gogh) fb54ce0 bench: add benchmarks for constituent hash algorithms for proof of work (Kittywhiskers Van Gogh) ee6efaa refactor: remove unused macros and function definitions (Kittywhiskers Van Gogh) 32156b2 build: assume true 64-bit target, assume `SPH_64` and related macros (Kittywhiskers Van Gogh) Pull request description: ## Additional Information * sphlib requires platforms to be `SPH_64` in order to build variants like Blake512 ([source](https://github.com/dashpay/dash/blob/489c5f0127891c2d058b65e00750ff3100d6d7b8/src/crypto/x11/blake.c#L632-L663)) and Bmw512 ([source](https://github.com/dashpay/dash/blob/489c5f0127891c2d058b65e00750ff3100d6d7b8/src/crypto/x11/bmw.c#L545-L576)). While other variants have alternate implementations for non-`SPH_64` platforms like Groestl ([source](https://github.com/dashpay/dash/blob/489c5f0127891c2d058b65e00750ff3100d6d7b8/src/crypto/x11/groestl.c#L46-L56)), non-`SPH_64` platforms _cannot_ build Blake512 or Bmw512. As X11 (and by extension, Dash) requires both, we can safely assume that Dash Core doesn't support non-`SPH_64` targets and can remove fallback code. * To inform decisions when optimizing X11, this pull request introduces benchmarks for all constituent hash algorithms across the same set of data sizes (32b, 80b, 128b, 512b, 1024b, 2048b and 1M), all tests can be run with `bench_dash --filter="Pow(.*)"` and can be filtered by data size with `bench_dash --filter="Pow(.*)1024b(.*)"` (see below) <details> <summary>Benchmarks:</summary> ``` | ns/byte | byte/s | err% | ins/byte | cyc/byte | IPC | bra/byte | miss% | total | benchmark |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:---------- | 1.60 | 625,125,455.46 | 0.3% | 25.60 | 7.10 | 3.603 | 0.13 | 0.0% | 0.01 | `Pow_Blake512_1024b` | 0.75 | 1,333,435,941.89 | 0.2% | 17.06 | 3.33 | 5.124 | 0.12 | 0.0% | 0.01 | `Pow_Bmw512_1024b` | 6.24 | 160,132,129.57 | 0.0% | 118.36 | 27.74 | 4.267 | 0.25 | 0.4% | 0.01 | `Pow_Cubehash512_1024b` | 14.64 | 68,308,839.95 | 0.0% | 261.17 | 65.02 | 4.017 | 1.63 | 0.6% | 0.01 | `Pow_Echo512_1024b` | 8.72 | 114,678,066.51 | 0.1% | 169.16 | 38.73 | 4.368 | 0.25 | 0.4% | 0.01 | `Pow_Groestl512_1024b` | 8.87 | 112,774,410.39 | 0.1% | 153.89 | 39.38 | 3.907 | 0.15 | 0.6% | 0.01 | `Pow_Jh512_1024b` | 3.79 | 264,078,320.88 | 0.1% | 81.14 | 16.81 | 4.826 | 0.23 | 0.0% | 0.01 | `Pow_Keccak512_1024b` | 6.87 | 145,603,041.03 | 0.1% | 131.55 | 30.46 | 4.319 | 1.11 | 0.1% | 0.01 | `Pow_Luffa512_1024b` | 7.54 | 132,578,659.24 | 0.1% | 119.56 | 33.48 | 3.572 | 0.26 | 0.4% | 0.01 | `Pow_Shavite512_1024b` | 7.24 | 138,072,734.74 | 0.5% | 129.58 | 32.09 | 4.037 | 2.38 | 0.0% | 0.01 | `Pow_Simd512_1024b` | 1.17 | 857,609,484.77 | 0.0% | 20.67 | 5.18 | 3.992 | 0.15 | 0.0% | 0.01 | `Pow_Skein512_1024b` | 11.45 | 87,351,960.89 | 0.0% | 195.23 | 50.85 | 3.840 | 1.30 | 0.1% | 0.01 | `Pow_X11_1024b` ``` </details> * This pull request also makes changes that result in performance impact, the baseline measurement is given below Build parameters: `LDFLAGS="-Wl,--as-needed -Wl,-O2" CC=clang-16 CXX=clang++-16 ./configure --prefix=$(pwd)/depends/x86_64-linux-gnu --enable-reduce-exports --disable-tests --disable-gui-tests --disable-fuzz --disable-fuzz-binary --disable-ccache --disable-maintainer-mode --disable-dependency-tracking --without-gui` <details> <summary>Baseline benchmarks (489c5f0):</summary> ``` | ns/byte | byte/s | err% | ins/byte | cyc/byte | IPC | bra/byte | miss% | total | benchmark |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:---------- | 321.49 | 3,110,502.97 | 0.1% | 5,527.82 | 1,427.78 | 3.872 | 38.97 | 0.1% | 0.01 | `X11_0032b_single` | 128.63 | 7,774,397.65 | 0.1% | 2,211.03 | 571.22 | 3.871 | 15.56 | 0.1% | 0.01 | `X11_0080b_single` | 81.97 | 12,199,203.11 | 0.1% | 1,404.79 | 364.00 | 3.859 | 9.80 | 0.1% | 0.01 | `X11_0128b_single` | 21.55 | 46,394,313.29 | 0.1% | 368.03 | 95.73 | 3.844 | 2.51 | 0.1% | 0.01 | `X11_0512b_single` | 11.48 | 87,085,396.93 | 0.1% | 195.23 | 51.00 | 3.828 | 1.30 | 0.1% | 0.01 | `X11_1024b_single` | 1.43 | 700,791,473.89 | 0.1% | 22.61 | 6.33 | 3.572 | 0.09 | 0.0% | 0.02 | `X11_1M` | 6.45 | 155,097,633.83 | 0.0% | 108.83 | 28.64 | 3.801 | 0.69 | 0.1% | 0.01 | `X11_2048b_single` ``` </details> Enablement of small footprint for JH, CubeHash (~10% improvement) <details> <summary>Benchmarks (30bf19d):</summary> ``` | ns/byte | byte/s | err% | ins/byte | cyc/byte | IPC | bra/byte | miss% | total | benchmark |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:---------- | 291.68 | 3,428,360.28 | 0.1% | 5,135.01 | 1,294.73 | 3.966 | 41.91 | 0.1% | 0.01 | `Pow_X11_0032b` | 117.46 | 8,513,845.07 | 0.1% | 2,053.90 | 518.66 | 3.960 | 16.74 | 0.1% | 0.01 | `Pow_X11_0080b` | 75.22 | 13,293,803.11 | 0.1% | 1,305.23 | 332.11 | 3.930 | 10.53 | 0.1% | 0.01 | `Pow_X11_0128b` | 19.65 | 50,880,118.90 | 0.0% | 342.12 | 86.79 | 3.942 | 2.70 | 0.1% | 0.01 | `Pow_X11_0512b` | 10.46 | 95,567,763.92 | 0.1% | 181.60 | 46.21 | 3.930 | 1.39 | 0.1% | 0.01 | `Pow_X11_1024b` | 1.20 | 835,966,628.21 | 0.4% | 21.24 | 5.28 | 4.023 | 0.09 | 0.0% | 0.01 | `Pow_X11_1M` | 5.83 | 171,499,428.31 | 0.2% | 101.34 | 25.75 | 3.936 | 0.74 | 0.1% | 0.01 | `Pow_X11_2048b` ``` </details> ## Breaking Changes None expected. ## Checklist: - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have added or updated relevant unit/integration/functional/e2e tests - [x] I have made corresponding changes to the documentation **(note: N/A)** - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_ ACKs for top commit: UdjinM6: utACK 44eebe7 Tree-SHA512: 26afe6b005b76285f8d7032e516d70daaa257e64066d77a4626f9da17f5b7208d5ca345975074fb539bcd7984ba2a923d02c73c2e9cdbe440d2fb7175df8b542
…nodes only 2b17480 fix(tests): add feature_masternode_params.py to test runner (pasta) bcff97a test: remove unused import from feature_masternode_params (pasta) fcd1b8c doc: add disk space requirement note to release notes (pasta) cec4ae0 test: address code review feedback for feature_masternode_params (pasta) 4e33c24 fix: specify utf8 (pasta) ac5c499 chore: trailing whitespace (pasta) 2fabaa9 fix: Address code review feedback for masternode-only BIP157 (pasta) 90ef2d5 refactor: Enable BIP157 block filters only for masternodes (pasta) d40ae45 test: fix tests incompatible with default peerblockfilters=1 (pasta) d6c11f0 test: fix feature_index_prune.py for new peerblockfilters default (pasta) 2a88db9 test: Fix feature_index_prune.py to explicitly disable indices (pasta) 4412ced test: Fix feature_index_prune.py to work with new blockfilterindex defaults (pasta) e6e9ff8 test: Fix functional tests to work with new blockfilterindex defaults (pasta) ef2a1e3 Fix initialization error when using default blockfilterindex value (pasta) fbde721 docs: Fix PR number in release notes - rename to 6711 (pasta) 0274cd2 docs: Add release notes for PR dashpay#6708 (pasta) ac7e03e Enable BIP157 block filters index and serving by default (pasta) Pull request description: ## Issue being fixed or feature implemented This pull request introduces automatic enabling of BIP157 compact block filters specifically for masternodes to improve privacy and functionality for light clients connecting to them. Regular nodes retain the existing default settings. ## What was done? - **Automatic Enablement for Masternodes**: When a node is configured as a masternode (via `-masternodeblsprivkey`), both `-peerblockfilters` and `-blockfilterindex=basic` are now automatically enabled. - **Regular Nodes Unchanged**: Regular (non-masternode) nodes keep the previous defaults with both options disabled (`-peerblockfilters=false` and `-blockfilterindex=0`). - **Opt-out Available**: Masternodes can still explicitly disable these features if needed by setting `-peerblockfilters=0` or `-blockfilterindex=0`. - **Release Notes**: Updated to clearly document this masternode-specific behavior and the ~1GB disk space requirement for the block filter index. - **Tests**: Added functional test to verify the parameter interactions and that the settings are correctly applied only to masternodes. ## How Has This Been Tested? - Built locally - Added new functional test `feature_masternode_params.py` that verifies: - Regular nodes have filters disabled by default - Masternodes have filters auto-enabled - Masternodes can explicitly disable filters - Parameter interaction logging works correctly ## Breaking Changes None - this only affects masternodes and they can opt out if needed. ## Checklist: _Go over all the following points, and put an `x` in all the boxes that apply._ - [x] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [x] I have added or updated relevant unit/integration/functional/e2e tests - [x] I have made corresponding changes to the documentation - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_ ACKs for top commit: UdjinM6: utACK 2b17480 Tree-SHA512: 6a11305cf54c7584de4ca8dbe520e022e23f211ec6b0e2ee7679073f75a465ead927814f3f01f77e8576f96a187167511204d1906d380937f98310948933dc83
…coin#25656, bitcoin#25616, bitcoin#26005, bitcoin#24855, bitcoin#18554, bitcoin#17204, bitcoin#20562, bitcoin#21166, bitcoin#25044, bitcoin#25364, bitcoin#25525, bitcoin#25512, bitcoin#24678, bitcoin#26747, partial bitcoin#22154, bitcoin#24584 (wallet backports: part 8) c0f9225 merge bitcoin#26747: fix confusing error / GUI crash on cross-chain legacy wallet restore (Kittywhiskers Van Gogh) d63ca8a merge bitcoin#24678: Prevent wallet unload on GetWalletForJSONRPCRequest (Kittywhiskers Van Gogh) 2a880d8 merge bitcoin#25512: remove wallet dependency and refactor rpc_signrawtransaction.py (Kittywhiskers Van Gogh) f405790 merge bitcoin#25525: remove wallet dependency from mempool_updatefromblock.py (Kittywhiskers Van Gogh) c24c9ea merge bitcoin#25364: remove wallet dependency from feature_nulldummy.py (Kittywhiskers Van Gogh) a118fe1 test: reduce `num_nodes` in `feature_nulldummy.py` to 1 (Kittywhiskers Van Gogh) c6eabc1 merge bitcoin#25044: Use MiniWallet in rpc_rawtransaction.py (Kittywhiskers Van Gogh) 4e0725e merge bitcoin#21166: Introduce DeferredSignatureChecker and have SignatureExtractorClass subclass it (Kittywhiskers Van Gogh) f883122 merge bitcoin#20562: Test that a fully signed tx given to signrawtx is unchanged (Kittywhiskers Van Gogh) c349dad merge bitcoin#17204: Do not turn OP_1NEGATE in scriptSig into 0x0181 in signing code (Kittywhiskers Van Gogh) c5e8bd6 merge bitcoin#18554: ensure wallet files are not reused across chains (Kittywhiskers Van Gogh) 0fca914 merge bitcoin#24855: Fix `setwalletflag` disabling of flags (Kittywhiskers Van Gogh) f15a1b9 merge bitcoin#26005: Fix error handling (copy_file failure in RestoreWallet, and in general via interfaces) (Kittywhiskers Van Gogh) 240765e merge bitcoin#25616: Return `util::Result` from WalletLoader methods (Kittywhiskers Van Gogh) bd897fa merge bitcoin#25656: return util::Result from `GetReservedDestination` methods (Kittywhiskers Van Gogh) 56accfe merge bitcoin#25721: Replace BResult with util::Result (Kittywhiskers Van Gogh) 03939f2 partial bitcoin#24584: avoid mixing different `OutputTypes` during coin selection (Kittywhiskers Van Gogh) 50ca8cf refactor: remove `CKey` overload for `Create{,AndProcess}Block()` (Kittywhiskers Van Gogh) 9e23b11 merge bitcoin#25218: introduce generic 'Result' class and connect it to CreateTransaction and GetNewDestination (Kittywhiskers Van Gogh) ec764fd partial bitcoin#22154: Add OutputType::BECH32M and related wallet support for fetching bech32m addresses (Kittywhiskers Van Gogh) Pull request description: ## Additional Information * The `CKey` overload for `Create{,AndProcess}Block()` was introduced as a convenience function in [dash#2264](dashpay#2264). To allow for the easier backport of [bitcoin#24584](bitcoin#24584) (by resolving an ambiguous overload error) and because `GetScriptForRawPubKey` makes the same construction, it was opted to realign with upstream and drop the overload. * Even though we don't actively support any address types that utilize Bech32m, [bitcoin#25656](bitcoin#25656) assumes that `GetReservedDestination` modifies a `bilingual_str` for its error case in order to encapsulate it in a `util::Result`. In order to complete that backport, [bitcoin#22154](bitcoin#22154) has been partially backported. * `CKeyHolder` has always assumed that `GetReservedDestination` will succeed without validation of this assumption ([source](https://github.com/dashpay/dash/blob/develop/src/coinjoin/util.cpp#L29-L33)). As [bitcoin#25656](bitcoin#25656) forces us to unwrap the returned value, this assumption has been documented with an `assert` ([source](https://github.com/dashpay/dash/blob/bd897fa7097d3d68d19b8a9af14b23f4007645ca/src/coinjoin/util.cpp#L32-L34)) * When backporting [bitcoin#21166](bitcoin#21166), we don't have the luxury of a `scriptWitness` to include the redemption script, so `rpc_signrawtransaction.py` must import the script in order to spend it. This creates an additional complication as descriptor wallets do not permit wallets to mix descriptors with and without private keys ([source](https://github.com/dashpay/dash/blob/cc9da5d9e28bf9a7b411dc390523ac6d2dd09de6/src/wallet/rpc/backup.cpp#L1758)) and as P2SH descriptors don't involve a private key, those specific subtests need to be skipped for descriptor wallets. * `OP_TRUE` has been appended to the script, this mirrors the `OP_TRUE` in the `scriptWitness` ([source](https://github.com/bitcoin/bitcoin/blob/a97a9298cea085858e1a65a5e9b20d7a9e0f7303/test/functional/rpc_signrawtransaction.py#L270)) * The subtest `test_signing_with_missing_prevtx_info()` introduced in [bitcoin#25044](bitcoin#25044) requires an extra `walletpassphrase` as the preceding tests that unlock the wallet are not called for descriptor wallets, making the call necessary. It is redundant but harmless for legacy wallets. * With the backport of [bitcoin#19937](bitcoin#19937) in [dash#6726](dashpay#6726) (as ae7e4cb), we can remove a workaround in `feature_nulldummy.py` where we used two nodes to satisfy `getblocktemplate`'s requirement of a connected node as that requirement was lifted on test chains. ## Breaking Changes None expected. ## How Has This Been Tested? A full CoinJoin session run on c0f9225  ## Checklist - [x] I have performed a self-review of my own code - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)** - [x] I have added or updated relevant unit/integration/functional/e2e tests - [x] I have made corresponding changes to the documentation **(note: N/A)** - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_ ACKs for top commit: PastaPastaPasta: utACK c0f9225 UdjinM6: utACK c0f9225 Tree-SHA512: 2217281ea75afe3eb6061e1acbd42afb66ff2ab28c15a3ad03bdb528ef4e40ea30bf1adad8d0ba93310a3d561d6c3f2ad95780b0b972af238523c6f2950a39c7
…transaction and testmempoolaccept Applied the ParseFeeRate fix from Bitcoin commit c265aad to the correct location in Dash. In Dash, sendrawtransaction and testmempoolaccept functions are located in src/rpc/rawtransaction.cpp instead of src/rpc/mempool.cpp. This fix prevents signed integer overflow when large fee rates are passed to these RPC methods by using the new ParseFeeRate helper that validates and rejects fee rates >= 1BTC/kvB. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>
05b09bf to
f634270
Compare
|
🔧 Validation FIXED: Applied 1 critical fix to ensure faithful Bitcoin backport Issue Found & Fixed:
Fix Applied:
Validation Summary:
The backport is now complete and faithful to the original Bitcoin changes. |
Backports bitcoin#29434
Original commit: c265aad
Backported from Bitcoin Core v0.27