-
Notifications
You must be signed in to change notification settings - Fork 725
[Consensus] Compatibility code for MN payments + budget voting #2308
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Consensus] Compatibility code for MN payments + budget voting #2308
Conversation
28054b6 to
ec15be0
Compare
77a8e89 to
d479681
Compare
|
Rebased on master. Ready for review |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Started the long road here, have reviewed 1/4 of it. Good stuff 🥃 .
Found few problems (aside from the comments) that have gone deeper on a personal branch repo link.
Essentially, was validating that votes for regular masternodes and deterministic masternodes were expiring properly (added functional test coverage for it in ef1f9afbc942bcdb58f69175435c7900ae6eace3) and found out that the COutPoint returned by the DMN setup in the test framework suite had the index hardcoded to 0, which was making the test fail sometimes as the collateral output index isn't fixed to any position in the tx creation process (fixes bfa6fa2b18d94e1b0a063497eaf601bc4f8dcd70 and ace871f276e2c193c8714f2665c073effb75f09d).
-- All yours to cherry-pick the three commits that did into this branch ☕ ☕ --
Aside from that, noticed that for some reason that haven't found yet, DMNs aren't removed from the list right away after a collateral spending. It takes a bit of time. Idk if you already have done it in a future PR but will be good to create a test similar to the tiertwo_masternode_activation.py but for DMN and verify every possible status and activation/deactivation/removal possible errors (ofc that this work is for another PR).
d479681 to
b5ac0b4
Compare
random-zebra
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot for the review @furszy. 🙏
I know that this one is probably the hardest PR of the list, and we should take our time, as it's easy to introduce bugs here.
So kudos for the test-driven approach 🥃
I think that what you suggest, though, goes in the wrong direction, due to a confusion between ProTx and collateral.
found out that the COutPoint returned by the DMN setup in the test framework suite had the index hardcoded to 0
Yes. It's on purpose.
That is not the collateral outpoint. It is just the hash of the proRegTx (collateral and proTx are different things, their txid is the same only in the "fund" case, not in "internal" or "external" case). We are returning a COutPoint in setupDMN just for compatibility with the old system (setupMasternode), but we only ever use its hash in the test (self.proRegTx1.hash, self.proRegTx2.hash, self.proRegTx3.hash).
I realize now that this is probably confusing, so have reworked setupDMN to return only the (proTx) hash -as it actually was already doing before 64d09e55aaa621045e93b3d26405cbcd632111b7-. Now it should be clearer.
This setup function is used exclusively in the compatibility test (and, as the comment states, it will be removed with it).
In the new tiertwo_deterministicmns.py (#2345) we use a different initialization, returning objects of the Masternode class introduced at the end of this PR, which encapsulates (among other things) the collateral out.
Aside from this, inserting an "index" in all the ProTx RPC outputs, as you did, is conceptually wrong imo, and it would be confusing for the user, because, in all cases, except for protx_register_fund:
- the index refers to another transaction, not the one identified by the returned "txid"
- the user already knows the collateral vout, as it is one of the RPC input parameters.
The reason why your test was failing is because this is wrong:
self.log.info("expiring DMN..")
self.spend_collateral(self.ownerOne, self.proRegTx1, self.miner)
self.wait_until_mn_vinspent(self.proRegTx1.hash, 30, [self.remoteTwo])You shouldn't pass self.proRegTx as collateral outpoint to spend (even though the first DMN incidentally was registered with "fund").
As explained, the output returned by setupDMN is not the collateral.
In the deterministic tests the collateral outpoint can be accessed from the Masternode object (dmn.collateral). Here we can just retrieve it with the RPC before spending.
Another side-note for the test is with this:
self.stake(12, [self.remoteTwo]) # create blocks to remove staled votesNewBlock is called every 14 blocks, not 12.
Technically we could even not stake, after spending the collateral, and simply force RemoveStaleVotesOnProposal by calling GetBudget() via RPC, but still... better to cover NewBlock.
Have picked and fixed the first commit from your branch.
The test is passing fine now.
Idk if you already have done it in a future PR
#2345 introduces lots more testing (e.g. check b28fc43445b0556af2f9b079f9a670b160a30bf3 for collateral spending)
After this first milestone, I think we can start working on a separate functional test for the governance, based only on DMNs.
This way, when the legacy system is obsolete, we can just remove tiertwo_governance_sync_basic.py (as well as tiertwo_mn_compatibility.py), and keep only the new tests.
Fuzzbawls
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
still digging through this one, quite a lot to digest, but wanted to leave some quick comments
3b3c44f to
42737b9
Compare
|
Thanks for the detailed answer zebra 👌. |
furszy
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A bit more feedback.
42737b9 to
3e46399
Compare
furszy
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Another day here, getting closer to finish it ☕ ☕ . Left some more feedback.
7807357 to
19550f7
Compare
also add finalized budget voting for DMNs
Which adds missing Relay and AddSeen to budget votes
Legacy manager should never call the deterministic one, while holding its own mutex (e.g. LegacyMNObsolete) fixing POTENTIAL DEADLOCK DETECTED Previous lock order was: cs_main blockassembler.cpp:236 (in thread bitcoin-httpworker) (1) cs evo/deterministicmns.cpp:517 (in thread bitcoin-httpworker) (2) cs masternodeman.cpp:456 (in thread bitcoin-httpworker) Current lock order is: (2) cs masternodeman.cpp:244 (in thread pivx-masternodeman) (1) cs evo/deterministicmns.cpp:855 (in thread pivx-masternodeman)
Before LegacyMNObsolete we need to use the same payment logic for DMNs too. In order to do so, we initialize a temporary MastrnodeRef object for each valid deterministic masternode and apply the masternode winner logic to it.
Also don't get the keys if the node is still waiting
we know that the masternode who signed the mnw message has positive rank, thus it exists, it's enabled, and it has a valid active protocol. Also change return value of CMasternodePayments::ProcessBlock to void.
Also pass the genesis hash instead of a null hash (as it was the original behavior before bfa628e) when no-masternode-to-pay is found in GetLegacyMasternodeTxOut (e.g. when there are less than 9 masternodes enabled on the network, and they are all scheduled already).
Instead of using always the genesis hash (which is constant)
19550f7 to
04cd17e
Compare
furszy
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK 04cd17e ☕.
Future work: add test cases for the invalid mnw and mnv broadcast paths.
Fuzzbawls
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK 04cd17e
4fd564c [Tests] Add EvoNotificationInterface in the TestingSetup fixture (random-zebra) f6aa6ad [BUG] Check masternode/budget payments during block connection (random-zebra) 57fa99f [Tests] Introduce tiertwo_reorg_mempool test (random-zebra) 9a1bf27 [BUG] Add special tx processing to RollforwardBlock (random-zebra) 5d6bcef [RPC][Tests] Include proTx data in json formatted transactions (random-zebra) 93e8453 [Tests] Update deterministicmns test checking collateral locking (random-zebra) cc132a8 [Wallet] Implement auto-locking of masternode collaterals (random-zebra) c857636 [Tests] Introduce new functional test tiertwo_deterministicmns.py (random-zebra) Pull request description: Another one coming from #2267 This PR introduces a "auto-locking" feature for masternode collaterals: - at startup, the wallet is scanned, and any masternode collateral coin found gets locked. Can be disabled setting `-mnconflock=0` (although, there is no need for a `masternode.conf` anymore, so we might want to consider defining and using a new flag here, and removing `mnconflock` after 6.0). - during runtime, when a ProRegTx is processed by `CWallet::AddToWalletIfInvolvingMe`, the wallet checks for ownership of the collateral, and automatically locks it in case. Here, we also add two more functional tests (`tiertwo_deterministicmns.py`, `tiertwo_reorg_mempool.py`), update the other tests, checking the auto-locking feature, and fix a couple bugs: - Missing special tx processing in RollforwardBlock - Validity of tiertwo payments can (and must) be verified only during block connection, as we keep the deterministic state only for the active chain, not for all possible chaintips. Builds on top of: - [x] #2308 - [x] #2309 ACKs for top commit: furszy: re-ACK 4fd564c after the tiny fix and merging.. Tree-SHA512: b0f05d9265c02fe90baf48e5dcf66b34f078b90ffd0eef1488606013f84996b2ebaa7d70725c59db2f7456105a6eff2ba08da966858f81c4f19d014a72cd7692
0f5064f [Trivial] rename local variable in CMasternodeMan::CountEnabled (random-zebra) 4e89bb2 [Refactor] move IsMNValid/IsMNPoSeBanned to CDeterministicMN (random-zebra) 2730521 [BUG] Check budget list for CMasternodeSync::Reset after SPORK 21 (random-zebra) 5da28cc [BUG] Fix potential deadlock (random-zebra) 8cc9735 [QA] Check enabled masternode count in functional tests (random-zebra) a0830a7 [BUG] Consider DMN in CountEnabled and CountNetworks (random-zebra) bf995eb [Cleanup] Remove unused active protocol argument in CountEnabled() (random-zebra) 34e1375 [BUG] Add missing initialization for MNsInfo::total (random-zebra) Pull request description: Bug introduced in #2308. We are considering the total masternode count, inclusive of deterministic masternodes, only in `GetNextMasternodeInQueueForPayment`, but not in other places. Fix this by updating directly the function `CountEnabled`, using the legacy-only count exclusively for the masternode list sync. ACKs for top commit: furszy: Code review ACK 0f5064f with a topic. Fuzzbawls: Code/Tests ACK 0f5064f Tree-SHA512: 9f182deeea471f8ab267659d9938d103a1aa2cdf7309635b057b71626ea3f99705f6948e09a6debc7411eaf83b8289c50e9cdc286d708b2ee2c51726f8718312
Extrated from #2267.
This enables proposals and budget voting for DMNs.
It also implements the compatibility code for masternode payments, to be used between v6 enforcement and SPORK_21 activation (when both systems coexist).
Based on top of: