Skip to content

Conversation

@random-zebra
Copy link

@random-zebra random-zebra commented Apr 11, 2021

Extrated from #2267.
This enables proposals and budget voting for DMNs.
It also implements the compatibility code for masternode payments, to be used between v6 enforcement and SPORK_21 activation (when both systems coexist).

Based on top of:

@random-zebra
Copy link
Author

Rebased on master. Ready for review

Copy link

@furszy furszy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Started the long road here, have reviewed 1/4 of it. Good stuff 🥃 .

Found few problems (aside from the comments) that have gone deeper on a personal branch repo link.

Essentially, was validating that votes for regular masternodes and deterministic masternodes were expiring properly (added functional test coverage for it in ef1f9afbc942bcdb58f69175435c7900ae6eace3) and found out that the COutPoint returned by the DMN setup in the test framework suite had the index hardcoded to 0, which was making the test fail sometimes as the collateral output index isn't fixed to any position in the tx creation process (fixes bfa6fa2b18d94e1b0a063497eaf601bc4f8dcd70 and ace871f276e2c193c8714f2665c073effb75f09d).
-- All yours to cherry-pick the three commits that did into this branch ☕ ☕ --

Aside from that, noticed that for some reason that haven't found yet, DMNs aren't removed from the list right away after a collateral spending. It takes a bit of time. Idk if you already have done it in a future PR but will be good to create a test similar to the tiertwo_masternode_activation.py but for DMN and verify every possible status and activation/deactivation/removal possible errors (ofc that this work is for another PR).

@random-zebra random-zebra force-pushed the 202103-dmn-compatibility branch from d479681 to b5ac0b4 Compare May 13, 2021 22:22
Copy link
Author

@random-zebra random-zebra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the review @furszy. 🙏
I know that this one is probably the hardest PR of the list, and we should take our time, as it's easy to introduce bugs here.
So kudos for the test-driven approach 🥃

I think that what you suggest, though, goes in the wrong direction, due to a confusion between ProTx and collateral.

found out that the COutPoint returned by the DMN setup in the test framework suite had the index hardcoded to 0

Yes. It's on purpose.
That is not the collateral outpoint. It is just the hash of the proRegTx (collateral and proTx are different things, their txid is the same only in the "fund" case, not in "internal" or "external" case). We are returning a COutPoint in setupDMN just for compatibility with the old system (setupMasternode), but we only ever use its hash in the test (self.proRegTx1.hash, self.proRegTx2.hash, self.proRegTx3.hash).

I realize now that this is probably confusing, so have reworked setupDMN to return only the (proTx) hash -as it actually was already doing before 64d09e55aaa621045e93b3d26405cbcd632111b7-. Now it should be clearer.

This setup function is used exclusively in the compatibility test (and, as the comment states, it will be removed with it).
In the new tiertwo_deterministicmns.py (#2345) we use a different initialization, returning objects of the Masternode class introduced at the end of this PR, which encapsulates (among other things) the collateral out.

Aside from this, inserting an "index" in all the ProTx RPC outputs, as you did, is conceptually wrong imo, and it would be confusing for the user, because, in all cases, except for protx_register_fund:

  • the index refers to another transaction, not the one identified by the returned "txid"
  • the user already knows the collateral vout, as it is one of the RPC input parameters.

The reason why your test was failing is because this is wrong:

self.log.info("expiring DMN..")
self.spend_collateral(self.ownerOne, self.proRegTx1, self.miner)
self.wait_until_mn_vinspent(self.proRegTx1.hash, 30, [self.remoteTwo])

You shouldn't pass self.proRegTx as collateral outpoint to spend (even though the first DMN incidentally was registered with "fund").
As explained, the output returned by setupDMN is not the collateral.
In the deterministic tests the collateral outpoint can be accessed from the Masternode object (dmn.collateral). Here we can just retrieve it with the RPC before spending.

Another side-note for the test is with this:

self.stake(12, [self.remoteTwo]) # create blocks to remove staled votes

NewBlock is called every 14 blocks, not 12.
Technically we could even not stake, after spending the collateral, and simply force RemoveStaleVotesOnProposal by calling GetBudget() via RPC, but still... better to cover NewBlock.

Have picked and fixed the first commit from your branch.
The test is passing fine now.


Idk if you already have done it in a future PR

#2345 introduces lots more testing (e.g. check b28fc43445b0556af2f9b079f9a670b160a30bf3 for collateral spending)

After this first milestone, I think we can start working on a separate functional test for the governance, based only on DMNs.
This way, when the legacy system is obsolete, we can just remove tiertwo_governance_sync_basic.py (as well as tiertwo_mn_compatibility.py), and keep only the new tests.

Copy link
Collaborator

@Fuzzbawls Fuzzbawls left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still digging through this one, quite a lot to digest, but wanted to leave some quick comments

@random-zebra random-zebra force-pushed the 202103-dmn-compatibility branch 2 times, most recently from 3b3c44f to 42737b9 Compare May 18, 2021 02:05
@furszy
Copy link

furszy commented May 18, 2021

Thanks for the detailed answer zebra 👌.

Copy link

@furszy furszy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit more feedback.

@random-zebra random-zebra force-pushed the 202103-dmn-compatibility branch from 42737b9 to 3e46399 Compare May 19, 2021 10:32
Copy link

@furszy furszy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another day here, getting closer to finish it ☕ ☕ . Left some more feedback.

@random-zebra random-zebra force-pushed the 202103-dmn-compatibility branch 3 times, most recently from 7807357 to 19550f7 Compare May 22, 2021 15:59
random-zebra and others added 22 commits May 23, 2021 09:36
also add finalized budget voting for DMNs
Which adds missing Relay and AddSeen to budget votes
Legacy manager should never call the deterministic one, while holding
its own mutex (e.g. LegacyMNObsolete)

fixing
 POTENTIAL DEADLOCK DETECTED
 Previous lock order was:
  cs_main blockassembler.cpp:236 (in thread bitcoin-httpworker)
  (1) cs evo/deterministicmns.cpp:517 (in thread bitcoin-httpworker)
  (2) cs masternodeman.cpp:456 (in thread bitcoin-httpworker)
 Current lock order is:
  (2) cs masternodeman.cpp:244 (in thread pivx-masternodeman)
  (1) cs evo/deterministicmns.cpp:855 (in thread pivx-masternodeman)
Before LegacyMNObsolete we need to use the same payment logic for DMNs
too.
In order to do so, we initialize a temporary MastrnodeRef object for
each valid deterministic masternode and apply the masternode winner
logic to it.
Also don't get the keys if the node is still waiting
we know that the masternode who signed the mnw message has positive
rank, thus it exists, it's enabled, and it has a valid active protocol.

Also change return value of CMasternodePayments::ProcessBlock to void.
Also pass the genesis hash instead of a null hash (as it was the
original behavior before bfa628e) when
no-masternode-to-pay is found in GetLegacyMasternodeTxOut (e.g. when
there are less than 9 masternodes enabled on the network, and they are
all scheduled already).
Instead of using always the genesis hash (which is constant)
@random-zebra random-zebra force-pushed the 202103-dmn-compatibility branch from 19550f7 to 04cd17e Compare May 23, 2021 07:45
Copy link

@furszy furszy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 04cd17e ☕.

Future work: add test cases for the invalid mnw and mnv broadcast paths.

Copy link
Collaborator

@Fuzzbawls Fuzzbawls left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 04cd17e

@random-zebra random-zebra merged commit 6abc9b6 into PIVX-Project:master May 25, 2021
furszy added a commit that referenced this pull request May 31, 2021
4fd564c [Tests] Add EvoNotificationInterface in the TestingSetup fixture (random-zebra)
f6aa6ad [BUG] Check masternode/budget payments during block connection (random-zebra)
57fa99f [Tests] Introduce tiertwo_reorg_mempool test (random-zebra)
9a1bf27 [BUG] Add special tx processing to RollforwardBlock (random-zebra)
5d6bcef [RPC][Tests] Include proTx data in json formatted transactions (random-zebra)
93e8453 [Tests] Update deterministicmns test checking collateral locking (random-zebra)
cc132a8 [Wallet] Implement auto-locking of masternode collaterals (random-zebra)
c857636 [Tests] Introduce new functional test tiertwo_deterministicmns.py (random-zebra)

Pull request description:

  Another one coming from #2267
  This PR introduces a "auto-locking" feature for masternode collaterals:

  - at startup, the wallet is scanned, and any masternode collateral coin found gets locked. Can be disabled setting `-mnconflock=0` (although, there is no need for a `masternode.conf` anymore, so we might want to consider defining and using a new flag here, and removing `mnconflock` after 6.0).

  - during runtime, when a ProRegTx is processed by `CWallet::AddToWalletIfInvolvingMe`, the wallet checks for ownership of the collateral, and automatically locks it in case.

  Here, we also add two more functional tests (`tiertwo_deterministicmns.py`, `tiertwo_reorg_mempool.py`), update the other tests, checking the auto-locking feature, and fix a couple bugs:

  - Missing special tx processing in RollforwardBlock

  - Validity of tiertwo payments can (and must) be verified only during block connection, as we keep the deterministic state only for the active chain, not for all possible chaintips.

  Builds on top of:
  - [x] #2308
  - [x] #2309

ACKs for top commit:
  furszy:
    re-ACK 4fd564c after the tiny fix and merging..

Tree-SHA512: b0f05d9265c02fe90baf48e5dcf66b34f078b90ffd0eef1488606013f84996b2ebaa7d70725c59db2f7456105a6eff2ba08da966858f81c4f19d014a72cd7692
furszy added a commit that referenced this pull request Oct 1, 2021
0f5064f [Trivial] rename local variable in CMasternodeMan::CountEnabled (random-zebra)
4e89bb2 [Refactor] move IsMNValid/IsMNPoSeBanned to CDeterministicMN (random-zebra)
2730521 [BUG] Check budget list for CMasternodeSync::Reset after SPORK 21 (random-zebra)
5da28cc [BUG] Fix potential deadlock (random-zebra)
8cc9735 [QA] Check enabled masternode count in functional tests (random-zebra)
a0830a7 [BUG] Consider DMN in CountEnabled and CountNetworks (random-zebra)
bf995eb [Cleanup] Remove unused active protocol argument in CountEnabled() (random-zebra)
34e1375 [BUG] Add missing initialization for MNsInfo::total (random-zebra)

Pull request description:

  Bug introduced in #2308.
  We are considering the total masternode count, inclusive of deterministic masternodes, only in `GetNextMasternodeInQueueForPayment`, but not in other places.
  Fix this by updating directly the function `CountEnabled`, using the legacy-only count exclusively for the masternode list sync.

ACKs for top commit:
  furszy:
    Code review ACK 0f5064f with a topic.
  Fuzzbawls:
    Code/Tests ACK 0f5064f

Tree-SHA512: 9f182deeea471f8ab267659d9938d103a1aa2cdf7309635b057b71626ea3f99705f6948e09a6debc7411eaf83b8289c50e9cdc286d708b2ee2c51726f8718312
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants