Skip to content

Conversation

@random-zebra
Copy link

@random-zebra random-zebra commented Apr 27, 2021

Another one coming from #2267
This PR introduces a "auto-locking" feature for masternode collaterals:

  • at startup, the wallet is scanned, and any masternode collateral coin found gets locked. Can be disabled setting -mnconflock=0 (although, there is no need for a masternode.conf anymore, so we might want to consider defining and using a new flag here, and removing mnconflock after 6.0).

  • during runtime, when a ProRegTx is processed by CWallet::AddToWalletIfInvolvingMe, the wallet checks for ownership of the collateral, and automatically locks it in case.

Here, we also add two more functional tests (tiertwo_deterministicmns.py, tiertwo_reorg_mempool.py), update the other tests, checking the auto-locking feature, and fix a couple bugs:

  • Missing special tx processing in RollforwardBlock

  • Validity of tiertwo payments can (and must) be verified only during block connection, as we keep the deterministic state only for the active chain, not for all possible chaintips.

Builds on top of:

@random-zebra
Copy link
Author

Rebased on master. Next in line.

@random-zebra random-zebra force-pushed the 202103-dmn-test-fixes branch from dbb8352 to d88265b Compare May 25, 2021 12:23
@random-zebra random-zebra requested review from Fuzzbawls and furszy May 27, 2021 09:24
@random-zebra random-zebra force-pushed the 202103-dmn-test-fixes branch from d88265b to 62bb2d6 Compare May 28, 2021 09:23
Copy link

@furszy furszy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK 62bb2d6f581ac52e6d05e77f7f1f85839473a415, awesome testing work, we definitely need more ☕ ☕

Copy link
Collaborator

@Fuzzbawls Fuzzbawls left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 62bb2d6f581ac52e6d05e77f7f1f85839473a415

However, I noticed a race condition in the tiertwo_reorg_mempool.py test that required me to add some arbitrary sleeps to allow a newly generated block to finish being processed prior to checking the MN list:

Index: test/functional/tiertwo_reorg_mempool.py
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/test/functional/tiertwo_reorg_mempool.py b/test/functional/tiertwo_reorg_mempool.py
--- a/test/functional/tiertwo_reorg_mempool.py	(revision 62bb2d6f581ac52e6d05e77f7f1f85839473a415)
+++ b/test/functional/tiertwo_reorg_mempool.py	(date 1622194250450)
@@ -170,6 +170,7 @@
             self.register_masternode(nodeB, dmn, collateral_addr)
             mnsB.append(dmn)
             nodeB.generate(1)
+        time.sleep(1)
         self.check_mn_list_on_node(1, mnsB)
 
         # Pick the proReg for the first MN registered on chain A, and replay it on chain B
@@ -179,6 +180,7 @@
         mnsB.append(replay_mn)  # same proTx hash
         nodeB.sendrawtransaction(nodeA.getrawtransaction(replay_mn.proTx, False))
         nodeB.generate(1)
+        time.sleep(1)
         self.check_mn_list_on_node(1, mnsB)
 
         # Now pick a proReg for another MN registered on chain A, and re-register it on chain B
@@ -188,6 +190,7 @@
         self.register_masternode(nodeB, rereg_mn, collateral_addr)
         mnsB.append(rereg_mn)   # changed proTx hash
         nodeB.generate(1)
+        time.sleep(1)
         self.check_mn_list_on_node(1, mnsB)
 
         # Register 5 more masternodes. One per block.

without that, I couldn't get the test to pass at all in a few dozen attempts

@furszy
Copy link

furszy commented May 29, 2021

That is because the signal isn't being processed before calling the MN check on your machine and the retrieved MN list is from the previous chain tip.
To solve it, instead of the arbitrary sleep time, use the syncwithvalidationinterfacequeue command.

@random-zebra random-zebra force-pushed the 202103-dmn-test-fixes branch from 62bb2d6 to faa87a4 Compare May 29, 2021 10:44
As the tiertwo_reorg_mempool shows, the reorganization currently fails.
This is because, during a reorg, CheckBlock is called on blocks where
the prev is not connected to the chain yet.
Since the budget/mn payment is checked there, we call GeListForBlock
(requesting a list which has not been built yet) and so we erroneously
cache an empty list.

Fix this by moving IsBlockPayeeValid from CheckBlock to ConnectBlock
(as, to check budget/mn payments, the previous block deterministic list
must be fully built).

Also, to prevent sneaky bugs like this, in the future, only cache the
initial snapshot in GetListForBlock for the very last block before the
enforcement of DIP3.
After the enforcement block, at least the diff is always written to disk
in BuildListFromBlock.
@random-zebra random-zebra force-pushed the 202103-dmn-test-fixes branch from faa87a4 to f6f87a4 Compare May 29, 2021 10:52
@random-zebra
Copy link
Author

Yes, I agree syncwithvalidationinterfacequeue should be a better fix. I've added it inside check_mn_list_on_node.
@Fuzzbawls let me know if it fixes it for you.

@random-zebra random-zebra force-pushed the 202103-dmn-test-fixes branch from f6f87a4 to 6594af7 Compare May 29, 2021 12:18
@Fuzzbawls
Copy link
Collaborator

The previously failing (locally) functional test is now passing, however now the deterministicmns_tests::dip3_protx unit test is failing constantly

Entering test module "Pivx Test Suite"
test/evo_deterministicmns_tests.cpp(187): Entering test suite "deterministicmns_tests"
test/evo_deterministicmns_tests.cpp(189): Entering test case "dip3_protx"
test/evo_deterministicmns_tests.cpp(242): error: in "deterministicmns_tests/dip3_protx": check deterministicMNManager GetListAtChainTip().HasMN(txid) has failed
test/evo_deterministicmns_tests.cpp(242): error: in "deterministicmns_tests/dip3_protx": check deterministicMNManager->GetListAtChainTip().HasMN(txid) has failed
test/evo_deterministicmns_tests.cpp(242): error: in "deterministicmns_tests/dip3_protx": check deterministicMNManager->GetListAtChainTip().HasMN(txid) has failed
test/evo_deterministicmns_tests.cpp(242): error: in "deterministicmns_tests/dip3_protx": check deterministicMNManager->GetListAtChainTip().HasMN(txid) has failed
test/evo_deterministicmns_tests.cpp(242): error: in "deterministicmns_tests/dip3_protx": check deterministicMNManager->GetListAtChainTip().HasMN(txid) has failed
test/evo_deterministicmns_tests.cpp(264): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(264): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(264): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(264): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(264): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(264): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(264): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(264): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(264): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(264): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(264): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(264): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(264): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(264): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(264): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(264): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(264): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(183): error: in "deterministicmns_tests/dip3_protx": MN 5b9908de527448e817336c1851b116f38548c96052f9ed51986670462f19a942 didn't receive expected num of payments (2<3)
test/evo_deterministicmns_tests.cpp(366): error: in "deterministicmns_tests/dip3_protx": check mnList.HasMN(txns[j].GetHash()) has failed
test/evo_deterministicmns_tests.cpp(366): error: in "deterministicmns_tests/dip3_protx": check mnList.HasMN(txns[j].GetHash()) has failed
test/evo_deterministicmns_tests.cpp(366): error: in "deterministicmns_tests/dip3_protx": check mnList.HasMN(txns[j].GetHash()) has failed
test/evo_deterministicmns_tests.cpp(366): error: in "deterministicmns_tests/dip3_protx": check mnList.HasMN(txns[j].GetHash()) has failed
test/evo_deterministicmns_tests.cpp(366): error: in "deterministicmns_tests/dip3_protx": check mnList.HasMN(txns[j].GetHash()) has failed
test/evo_deterministicmns_tests.cpp(366): error: in "deterministicmns_tests/dip3_protx": check mnList.HasMN(txns[j].GetHash()) has failed
test/evo_deterministicmns_tests.cpp(366): error: in "deterministicmns_tests/dip3_protx": check mnList.HasMN(txns[j].GetHash()) has failed
test/evo_deterministicmns_tests.cpp(366): error: in "deterministicmns_tests/dip3_protx": check mnList.HasMN(txns[j].GetHash()) has failed
test/evo_deterministicmns_tests.cpp(366): error: in "deterministicmns_tests/dip3_protx": check mnList.HasMN(txns[j].GetHash()) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(379): error: in "deterministicmns_tests/dip3_protx": check IsMNPayeeInBlock(block, dmnExpectedPayee->pdmnState->scriptPayout) has failed
test/evo_deterministicmns_tests.cpp(179): error: in "deterministicmns_tests/dip3_protx": check mp.size() == mapSize has failed [14 != 15]
test/evo_deterministicmns_tests.cpp(183): error: in "deterministicmns_tests/dip3_protx": MN a84306328df55c6f796496a5c04aa986882fa7f24b5270f8ed3ba834609a9520 didn't receive expected num of payments (1<2)
test/evo_deterministicmns_tests.cpp(183): error: in "deterministicmns_tests/dip3_protx": MN c501d492ff749f0659f03f3a09899c419c1c19035afe8e013ea800568d316c44 didn't receive expected num of payments (1<2)
test/evo_deterministicmns_tests.cpp(183): error: in "deterministicmns_tests/dip3_protx": MN 2bfdf7e3ba1d67c9766131243c07c9f7988f1cb310b288a064911023e8c75747 didn't receive expected num of payments (1<2)
test/evo_deterministicmns_tests.cpp(183): error: in "deterministicmns_tests/dip3_protx": MN 37e0bd808319b87400f845e7b931a54790ebe625e209fe7a1fcb4b8550756b52 didn't receive expected num of payments (1<2)
test/evo_deterministicmns_tests.cpp(189): Leaving test case "dip3_protx"; testing time: 2937500us
test/evo_deterministicmns_tests.cpp(187): Leaving test suite "deterministicmns_tests"; testing time: 2937500us                                                                                                   Leaving test module "Pivx Test Suite"; testing time: 2937500us

@random-zebra random-zebra force-pushed the 202103-dmn-test-fixes branch from 6594af7 to 3ff41d2 Compare May 29, 2021 13:26
@random-zebra random-zebra force-pushed the 202103-dmn-test-fixes branch from 1a43512 to 2d83495 Compare May 29, 2021 13:53
furszy
furszy previously approved these changes May 30, 2021
Copy link

@furszy furszy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code ACK 2d83495fc75024b6aa3ca7b69d24f606a6b2262c, just found one extra line that can be removed.

Thus avoid direct calls to `UpdatedBlockTip` in the tests.
Copy link

@furszy furszy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re-ACK 4fd564c after the tiny fix and merging..
Two more PRs to go for 2267 long road completion.

@furszy furszy merged commit bc56903 into PIVX-Project:master May 31, 2021
furszy added a commit that referenced this pull request Jun 17, 2021
aacada8 [BUG] Fix mempool conflicts due to disconnected blocks (random-zebra)
91ad6eb [Tests] Functional testing for ProUpServ txes (random-zebra)
e2c83dd [RPC] Implement "protx_update_service" call (random-zebra)
fb2557f [Mempool] Conflict handling for ProUpServ txes (random-zebra)
281d460 [Tests] Add provider-update-service to dip3_protx unit-test (random-zebra)
e83e454 [TierTwo] Connect ProUpServ payload management in dmn manager (random-zebra)
a197fce [Tests] Add Get/Set payload unit-test for ProUpServ txes (random-zebra)
1f5f509 [Refactoring] Prepare mempool code for multiple payload types (random-zebra)
a83d87e [Core] Introduce ProUpServ payload and transaction type (random-zebra)

Pull request description:

  Exctracted from #2267
  This PR:
  - introduces the `PROUPSERV` payload and transaction type.
  This special transaction is submitted by the operator (the payload must be signed with the operator key) to update the IP/port fields.
  If the original ProReg transaction set a non-zero operatorReward, this payload can also be used to update the operator's payout address.

  - adds the new RPC `protx_update_service` to create ProUpServ transactions

  - adds unit and functional testing

  Builds on top of:
  - [x] #2345

ACKs for top commit:
  furszy:
    utACK aacada8 after rebase.
  Fuzzbawls:
    Code ACK aacada8

Tree-SHA512: 6a642dfa5c7d239de23adaacd2cba56621f4392b2d34a436e023df6452cf9d39c55fb58c62ade2705db4f55f47674fc936644d731e9f8eb526d6679815faa864
random-zebra added a commit to random-zebra/PIVX that referenced this pull request Jun 19, 2021
random-zebra added a commit that referenced this pull request Jun 21, 2021
4368613 [BUG] Check masternode/budget payments during block connection (random-zebra)
0374dbe [QA] Add tiertwo_governance_reorg functional test (random-zebra)

Pull request description:

  First commit backports the functional test of #2436.
  Second commit extracts from f6aa6ad (#2345) the fix to make it pass.

ACKs for top commit:
  Fuzzbawls:
    Code ACK 4368613

Tree-SHA512: d12cbcc59719c5a5328c04c567cfa387d98ef00565e21a2bd37a3de3d85cead7120267b04b8c121d0489be6d5c7cf726337228e4123048d1f285a2c35b826651
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants