Skip to content

Conversation

@random-zebra
Copy link

@random-zebra random-zebra commented Sep 9, 2020

Based on top of:

Current PR starts with [Refactor] Decouple CBudgetProposal::UpdateValid checks (6c6a97e3b27ffc3fe953d88e0f008c3fa9cdee4c).

This focuses on extracting context-independent checks and doing them only once in AddProposal/AddFinalizedBudget.
As for the confirmation/expiration check (which is the only thing left for UpdateValid) , it is done without locking cs_main (while holding cs_budgets/cs_proposals) by caching the height of the block with the collateral transaction and relying on it (todo: it needs to be addressed the case of it being reorganized from the chain).

This new field, called nBlockFeeTx is added directly to the serialization of CBudgetProposal and CFinalizedBudget, so the old budget.dat will be invalidated and recreated new at startup.

@random-zebra
Copy link
Author

Rebased on master. Ready for review.

@random-zebra random-zebra force-pushed the 202009_budget_UpdateValid branch from 6f9e943 to ae8e17c Compare October 1, 2020 01:27
Will refactor the name (so that it includes the proposals) in a
successive commit.
- Introduce a new variable nBlockFeeTx, set during CheckCollateral
  caching the height of the block holding the collateral tx

- Call CheckCollateral (setting proposals/budgets height/time) only from
  one place (AddProposal/AddFinalizedBudgets) without cs_main held.

- only check for confirmations inside UpdateValid (using the cached
  value of nBlockFeeTx).

- remove fCheckCollateral from UpdateValid
  In those two cases where we haven't submitted the collateral tx yet,
  don't add the proposal/budget to the map. Only check that it's well
  formed.

This ultimately removes the locking-order issue between cs_main and
cs_budgets/cs_proposals.

Changes to the serialization will invalidate the previous DB, and force
the node to recreate the map (calling AddProposal and setting the block
height).

TODO: Remove proposals/budgets from maps if the block is disconnected.
Since we no longer add immature/unconfirmed proposal/budgets there's no
reason to cache the collateral tx height.
Because all of them have the same name (object key).
Fix it by adding the nHash to the object key (removing it from the
object value).
@random-zebra random-zebra force-pushed the 202009_budget_UpdateValid branch from ae8e17c to 69bba50 Compare October 3, 2020 00:24
@random-zebra
Copy link
Author

Rebased on top of #1891 and #1894 .

Added another step in the cleanup:
Since we no longer add immature budget/proposals, we no longer need to keep checking the confirmations (and save the nBlockFeeTx in the object).

Also fixed two bugs that popped out during testing:
1- Finalized budgets remain in the map (and can be voted) for two budget cycles after they expire. This goes against IsTransactionValid (which checks the nBlockEnd of a finalized budget against the current height).
2- When there is no finalized budget to pay (with enough votes), the block creator should pay a masternode.

Copy link

@furszy furszy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review until ad1e9a2 (non inclusive). Left some findings, overall I really like where this is going ☕

Copy link

@furszy furszy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK 4fc072f, really nice!, looking a lot lot better 🍺

Only one topic on my head.
After this, we don't have a process for chain reorgs for budget proposals and budget finalizations. Confirmations are only calculated once when the item is about to be added to their respective map.
(if this is introduced in a subsequent PR, then this is just a self-reminder for future PRs review :) )

Will start testing it now.

@random-zebra
Copy link
Author

After this, we don't have a process for chain reorgs for budget proposals and budget finalizations. Confirmations are only calculated once when the item is about to be added to their respective map.

Yup. It's stated in the description here:

[...] by caching the height of the block with the collateral transaction and relying on it (todo: it needs to be addressed the case of it being reorganized from the chain).

And the PR that takes care of it was actually submitted 3 weeks ago (#1858) 😉

furszy
furszy previously approved these changes Oct 8, 2020
Copy link

@furszy furszy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tested sync with #1829, ACK 4fc072f

@random-zebra random-zebra force-pushed the 202009_budget_UpdateValid branch from 40a06df to b65e862 Compare October 15, 2020 02:25
Copy link

@furszy furszy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK 4a49010. Going to try it in regtest now.

Copy link

@furszy furszy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested, great ACK 4a49010 .

@furszy furszy requested a review from Fuzzbawls October 18, 2020 16:51
Copy link
Collaborator

@Fuzzbawls Fuzzbawls left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK 4a49010

@random-zebra random-zebra merged commit f152dee into PIVX-Project:master Oct 19, 2020
furszy added a commit that referenced this pull request Oct 26, 2020
5004e1f [Cleanup] Remove not needed budget/proposal constructors (random-zebra)
957b63e [Cleanup] Nuke CBudgetProposalBroadcast and CFinalizedBudgetBroadcast (random-zebra)
5c4eea1 [Refactor] Migrate C*Broadcast() classes to C*::GetBroadcast() (random-zebra)
5c88045 [Refactor] Initialize Gov Objects from serialized messages (random-zebra)
d0f0705 [Refactor] Remove mapSeenProposals / mapSeenFinalizedBudgets (random-zebra)
30c5599 [Refactor] Use new interfaces in CBM::GetProposal[FinBudget]Serialized() (random-zebra)
d0a959e [Refactoring] Introduce CFinalizedBudget/CBudgetProposal::GetBroadcast (random-zebra)

Pull request description:

  Continues the budget-refactoring thread.

  Remove the following classes entirely
  - `CBudgetProposalBroadcast`
  - `CFinalizedBudgetBroadcast`

  Replace them with a new dastream-based constructor and a `GetBroadcast` function (returning the serialized stream) inside `CBudgetProposal` / `CFinalizedBudget` classes.

  This allows us to get rid also of `mapSeenProposals` and `mapSeenFinalizedBudgets` and all the relative ugly code duplication (just rely on `mapProposals` and `mapFinalizedBudgets` to check for existence, and to get the broadcast messages to send).

  Based on top of
  - [x] #1845

  Starts with `[Refactoring] Introduce CFinalizedBudget/CBudgetProposal::GetBroadcast` (37c629881372e492a65ea17a4b18ad02cd53ed58)

ACKs for top commit:
  furszy:
    re utACK 5004e1f .
  Fuzzbawls:
    ACK 5004e1f

Tree-SHA512: e4750d57b069df66ca1e4bdb1e4de35c06120e17c206717b626d9b24eb76fbfbe4d81c681084f2dd6e6c2776d4a88a82a090b5c10cc549a2f9780064cbad8c69
random-zebra added a commit that referenced this pull request Nov 2, 2020
…ndexes

d7a1c99 [Cleanup] Extra call to CBudgetMan::SetBestHeight in reconsiderblock (random-zebra)
5d0731e [Cleanup] Remove redundant Clear in CBudgetManager constructor (random-zebra)
48b723e [Budget] Remove proposal/budgets if FeeTx block is disconnected (random-zebra)
9c6173c [Budget] Introduce mapFeeTxToProposal and mapFeeTxToBudget (random-zebra)

Pull request description:

  Based on top of:
  - [x] #1851

  Starting with `[Budget] Introduce mapFeeTxToProposal and mapFeeTxToBudget` (0eb9e45)

  Since #1845 we no longer check repeatedly the active chain, when updating a budget object, we just save the height of the block including the collateral tx (ref #1845), when we first add the proposal/budget to the map.

  Therefore we need to handle the case where said block is disconnected from the chain.

  In order to do so efficiently, this PR adds two indexes (`mapFeeTxToBudget`/`mapFeeTxToProposal`), mapping collateral txids to the relative budget objects (while such objects are stored in the map).
  A simple function `CBudgetManager::RemoveByFeeTxId()` can then be called from `DisconnectBlock` when undoing transactions, in order to remove the now-conflicted budget objects (without performing any expensive search).

ACKs for top commit:
  furszy:
    nice, code review ACK d7a1c99.
  Fuzzbawls:
    ACK d7a1c99

Tree-SHA512: 9dd8a671032f79105219b7a0e3e0514b991ff92e55834f5adad4964fb9b5c1842bbb0c0995c9b504ca79c5c86115061f4a93e9b4560a63c2beaf66aa5915c2f5
random-zebra added a commit that referenced this pull request Nov 3, 2020
…asternodeman:ProcessMessage

50e156f MasternodeMan::ProcessMessage flow calculating the mnping and mnbroadcast hash only once. (furszy)
1d8a3e4 masternodeman stripped out cs_main lock for Misbehaving from ProcessMessage entirely. (furszy)
fef42b1 Move masternodeman vin null comparison to be using IsNull + Move-only ProcessMessage below ProcessGetMNList. (furszy)
4434112 CTxIn IsNull method implemented. (furszy)
9c4e8d1 Decouple ProcessGetMNList into its own method plus strip out Misbehaving call from its flow (furszy)
befaaeb Decouple ProcessMNBroadcast into its own method plus strip out Misbehaving call from its flow. (furszy)

Pull request description:

  Built on top of #1845.

  Refactored `Masternodeman:ProcessMessage` decoupling in different methods and stripped out every `cs_main` lock for a `Misbehaving` function call.

ACKs for top commit:
  random-zebra:
    utACK 50e156f
  Fuzzbawls:
    utACK 50e156f

Tree-SHA512: 4468ba1fd1bbd983a983374de9a56453d983f0dc9def86185992ce7102ce34c8b07b559b58c65f13404676023e5e6a457fe1cf8370915e3e357678fca29bf6a4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants