Skip to content

Conversation

@kwvg
Copy link
Collaborator

@kwvg kwvg commented Feb 28, 2024

Additional Information

  • PeerManager's initialization has been moved downwards (it used to be initialized after CGovernanceManager) to now after CMasternodeSync. This is to avoid having to pass CSporkManager's unique_ptr container and instead pass the its dereferenced pointer.

  • CChainstateHelper is just proxy for helper classes meant to hold references to managers that would be needed by functions that are called by CChainState. It's the alternative to passing every single manager into CChainState through ChainstateManager.

    Instead, they're all bunched up via CChainstateHelper and is accessible to CChainState through passing it as an argument. For this reason, it should ideally initialized after all relevant managers are setup but before the chain is validated. We would want to avoid deferred dereferencing if we can help it.

    • Internal/private functions have been marked as [[nodiscard]].

Breaking Changes

None. Changes are limited to refactoring, no logical changes have been made.

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas (note: N/A)
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have made corresponding changes to the documentation (note: N/A)
  • I have assigned this pull request to a milestone (for repository code-owners and collaborators only)

@kwvg kwvg added this to the 20.1 milestone Feb 28, 2024
@github-actions
Copy link

This pull request has conflicts, please rebase.

@kwvg kwvg requested review from PastaPastaPasta, UdjinM6 and knst March 1, 2024 17:50
@kwvg kwvg marked this pull request as ready for review March 1, 2024 18:07
@kwvg kwvg requested a review from UdjinM6 March 2, 2024 11:19
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: that's a small method, but I don't see a reason to move it to the top of cpp file. especially in the next commit (rename MNSubsidy to CSHelper) most of code removed from this file.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only most of the header contents of masternode/payments.h are moved to evo/cshelper.h, the contents of masternode/payments.cpp and evo/specialtxman.cpp remain where they are and are only modified to the extent that the new name was being used.

The new header exists somewhat document that ChainstateHelper is just an vessel for logic implemented in those two files and the logic implemented in specialtxman.cpp should not be considered a part of masternode payment code (which would've been in the implication had the special transaction functions been defined in masternode/payments.h).

I kept it at the top because it's a relatively small function (and now additionally, because it's the only function actually defined in its corresponding header, the rest being implementations of the definitions now in cshelper.h)

src/miner.cpp Outdated
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like constructor of BlockAssembler to accept many-many arguments; this commit make list of arguments longer --- I don't feel like that is a useful refactoring, moreover I feel like that's getting worse.

If that's a staging temporary changes - I'd like to know what is a next step, but BlockAssembler having +1 more argument as it is done now that is defactoring, not refactoring in my opinion


IMO much better to let BlockAssembler to use mnhfman, creditpool, from chainparams or from cs_helper rather than have that long list of arguments in constructor

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could simply pass node context here I guess c764c09255

Copy link
Collaborator Author

@kwvg kwvg Mar 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know how I feel about making the managers available from cs_helper (everywhere else we accept the references for the classes' use but don't actually expose it) but I do agree, the long arguments are frustration-inducing (I'd know, reordering commits and cleaning up required me to fix up the argument list manually multiple times).

But the truth of the matter is, we have a lot of managers. Excluding CoinJoin and LLMQ managers (which are managed by their own context), we have ~10 managers and I don't think that's all of them. There's no real way to avoid it without consolidating them. There's a reason I got rid of CGovernanceTriggerManager, we don't need more managers.

I did think about creating DashContext but then accessing something like CBLSWorker would require doing something like node.dash_ctx->llmq_ctx->bls_worker and that, at least on initial impression, seems like too much nesting.


Using CChainState seems unideal imo, I've tried experimenting with aliasing the CDeterministicMNList query function to CChainState (similar to what was done with CMNHFManager) but it seemed unwieldly to do so, not only was I adding yet another argument to pass through ChainstateManager and CChainState, for a function that CChainState doesn't really need, it's more likely than not that if CDeterministicMNList is passed upwards enough, we'll eventually reach a point where CDeterministicMNManager would be needed.

The reason why I did the refactoring for CMNHFManager is because I'd prefer if CChainState accepts the fewest possible arguments. It's the reason ChainstateHelper exists, because the alternative would be piping CGovernanceManager, CSporkManager and others.


I'm not quite sure of adding NodeContext as an argument, it doesn't really seem to be meant to be used that way. Especially since, unlike LLMQContext, where the constructor initializes its members, there's no such guarantee for NodeContext.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not quite sure of adding NodeContext as an argument, it doesn't really seem to be meant to be used that way.

Don't know, for me passing node context instead of individual members make perfect sense. Check this for example.

Especially since, unlike LLMQContext, where the constructor initializes its members, there's no such guarantee for NodeContext.

Check that diff a bit closer ;) and you'll notice that we do the same pointer dereference without checking anything already so there should be no side effects if we move these dereferences somewhere else. But it's true, it would probably make sense to have some (inline) asserts, see f5b3875578.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cherry-picked and squashed in latest push

src/miner.cpp Outdated
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with credit pool here that's getting even worse. Need to think how to wrap things up here. Credit pool and mnhfman and mn_subsidy - that's a things that should be inside m_chainstate, as by design and by idea.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why don't move all of that directly to CChainState instead?

Copy link
Collaborator Author

@kwvg kwvg Mar 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because I'd prefer CChainState to be as close to upstream as possible, not only to make backports and maintenance easier, but to also maintain a soft barrier between Dash and Bitcoin code.

ChainstateHelper lets primarily Bitcoin logic access predominantly Dash logic, acting as a vessel that abstracts the inner machinations from CChainState, letting us change and update Dash-specific code as we see fit without having to make too many changes to Bitcoin-maintained code.

@UdjinM6 UdjinM6 modified the milestones: 20.1, 21 Mar 5, 2024
@knst
Copy link
Collaborator

knst commented Mar 5, 2024

@kwvg let's split it one more time to 2 PRs?

Please, check this branch for part I:
https://github.com/knst/dash/branches/deglob3-p1
these commits to include and they looks like ready to merge (maybe some nits, but looks fine) that's a draft, due to compilation errors

b5b86a9150 refactor: s/governanceManager/govman/s
3a80165f74 refactor: remove CGovernanceManager global, move to NodeContext
20448439a1 refactor: s/sporkManager/sporkman/g
13fbe2e46d refactor: remove CSporkManager global, move to NodeContext
b2b781e2b1 refactor: remove CNetFulfilledRequestManager global, move to NodeContext
2073075654 refactor: remove CDSTXManager global and alias, move to CJContext
fced7b042e refactor: remove redundant condition check in `IsOldBudgetBlockValueValid`
1752f0d965 refactor: cleanup CDSNotificationInterface member names, add asserts
78b67436ef logging: reintroduce double hyphens in Dash-specific log messages
4dc6417cd0 refactor: subsume CGovernanceTriggerManager into CGovernanceManager

Remaining commits I'd like to refactor and change a bit, I will prepare patches later

@kwvg
Copy link
Collaborator Author

kwvg commented Mar 5, 2024

@knst

The deglobalization of CSporkManager and CGovernanceManager is aided by ChainstateManager because we take them in as arguments (see diff). The redundant condition check was detected only because they're now accessing the same m_consensus_params reference, which is possible because they were made a part of ChainstateManager.

7f5f37c643941c6ff53250ff5a5138b72b47125d and 27ad7fb0cb89c5874b2eaafe29cffc3b1b534ede are find-and-replace operations, 285985647ebf3133b840b779c60653b9510c5865 and 90d908fac9d9a09e975b4dc394b16617f6dc244c are responses to feedback, 58336d44d6c6c79ec2e4d6b92bf6cefb3812aa5e is in response to CI linting and 600be3f90ce3d0fb51ffd2b106246c24d680ffb1 is just cleanups and additional assertions. Leaving ten commits.

IMO this PR doesn't need to be split into smaller chunks. This PR was already part of a larger set of changes and split once and after this, there is a dedicated PR just to deal with CDeterministicMNManager.

I think this PR is fine as it is, at least as far as size is concerned.

@kwvg kwvg force-pushed the deglob3 branch 2 times, most recently from 7103eec to 4e42e5a Compare March 5, 2024 16:36
@kwvg
Copy link
Collaborator Author

kwvg commented Mar 5, 2024

PR has been updated to make ChainstateHelper changes easier to follow (got rid of MNSubsidyAgent version and associated rebranding, integrated restoration of double hyphens into the changes themselves, placed special transaction integration right after MasternodePayments::* integration)

Copy link
Collaborator

@knst knst Mar 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit (not a blocker to get merged):

#include <vector>
#include <memory>
#include <optional>

knst pushed a commit to knst/dash that referenced this pull request Mar 5, 2024
@knst
Copy link
Collaborator

knst commented Mar 5, 2024

these commits to include and they looks like ready to merge (maybe some nits, but looks fine) that's a draft, due to compilation errors

https://github.com/knst/dash/branches/deglob3-p1
fixed, seems tests pass

7caaeda719 refactor: s/governanceManager/govman/s
ab34f60ced refactor: remove CGovernanceManager global, move to NodeContext
6b8d7709cd refactor: s/sporkManager/sporkman/g
68ed784dd6 refactor: remove CSporkManager global, move to NodeContext
5202c46d74 refactor: remove CNetFulfilledRequestManager global, move to NodeContext
8b10a1ae7a refactor: remove CDSTXManager global and alias, move to CJContext
518eb18938 refactor: remove redundant condition check in `IsOldBudgetBlockValueValid`
020a351f5a refactor: cleanup CDSNotificationInterface member names, add asserts
a4c3153c4a logging: reintroduce double hyphens in Dash-specific log messages
3414ca94fa refactor: subsume CGovernanceTriggerManager into CGovernanceManager
797a037ac1 refactor: wrap MasternodePayments functions into MNSubsidyAgent

part II will update later

@github-actions
Copy link

github-actions bot commented Mar 6, 2024

This pull request has conflicts, please rebase.

@kwvg kwvg changed the title refactor: move masternode payments processing into helper class, move C{Governance,Spork}Manager to NodeContext refactor: move masternode payments processing to helper class, move C{Governance,Spork}Manager to NodeContext Mar 10, 2024
@kwvg kwvg requested a review from knst March 10, 2024 11:17
Copy link
Member

@PastaPastaPasta PastaPastaPasta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK febdaa5


#include <masternode/payments.h>

class CChainstateHelper : public CMNPaymentsProcessor
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why there's used inheritance instead aggregation? Why do we even need CMNPaymentProcessor?

@PastaPastaPasta PastaPastaPasta dismissed their stale review March 12, 2024 20:32

Whoops; reviewed too fast. Agree, shouldn't be inheritance. Also; wasn't there another manager for special transactions that was a part of CChainstateHelper?

Copy link
Collaborator

@knst knst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's move this PR and get it merged soon.

Please, add this one: knst@39f1dea

otherwise LGTM

Also; wasn't there another manager for special transactions that was a part of CChainstateHelper?

that went (split) to new PR I assume


sorry for delay, it seems as impossible to refactor as I planned at the moment and make it nice and tidy.
My current version removes 4 circular dependencies, but it also adds 10 new circular dependencies. They are possible to resolve, but each of them is non-trivial and worth own PR.
there's my draft that I dropped but may pick some commits later in new PR: https://github.com/knst/dash/commits/deglob3-p3


extra note for me: these backports are related
Merge bitcoin#25168: refactor: Avoid passing params where not needed
Merge bitcoin#21526: validation: UpdateTip/CheckBlockIndex assumeutxo support
Merge bitcoin#21866: [Bundle 7/7] validation: Farewell, global Chainstate!

@kwvg
Copy link
Collaborator Author

kwvg commented Mar 13, 2024

Also; wasn't there another manager for special transactions that was a part of CChainstateHelper?

@PastaPastaPasta, those changes were moved to #5929

extra note for me: these backports are related

It's being tracked in https://github.com/dashpay/dash-issues/issues/36 and https://github.com/dashpay/dash-issues/issues/52

Why do we even need CMNPaymentProcessor?

@knst, due to a concern raised in code review that two different domains, masternode payments processing and special transaction processing, would be under one head (CChainstateHelper) and the setup where they'd be implemented in two files (masternode/payments.cpp and evo/specialtxman.cpp) but defined in evo/chainhelper.h. That's why the domains were split into CMNPaymentProcessor and later moved to #5929, CSpecialTxProcessor.

why there's used inheritance instead aggregation?

Because we already have two Dash-specific aggregations (LLMQContext and CJContext) and it's likely that a third one would be on the way for either all Dash-specific logic (something like DashContext) or masternode-specific logic (something like MNContext).

Creating another aggregation that will only be used by CChainState and BlockAssembler seemed excessive, so inheritance fit the bill. The general aversion to inheritance shouldn't apply here because neither domain classes hold any state. They exist only to document separation between them but to CChainState/BlockAssembler, they were to be from one helper class.

I really don't look forward to a worst case scenario of node.dash_ctx->chain_helper->mn_payments->IsBlockPayeeValid(...).

@knst
Copy link
Collaborator

knst commented Mar 13, 2024

Creating another aggregation that will only be used by CChainState and BlockAssembler seemed excessive, so inheritance fit the bill. The general aversion to inheritance shouldn't apply here because neither domain classes hold any state. They exist only to document separation between them but to CChainState/BlockAssembler, they were to be from one helper class.

so, when there's will be another member inside chain_helper.

I really don't look forward to a worst case scenario of node.dash_ctx->chain_helper->mn_payments->IsBlockPayeeValid(...).

So, it will be a time to rename chain_helper to something else then, or refactor one more time. That's not carved on stone

@PastaPastaPasta
Copy link
Member

Agree with KNST

kwvg and others added 6 commits March 14, 2024 03:29
…alid`

We're already checking if the block height is after the budget payment
start block and fast-failing if it isn't. There's no need to check again
a few lines later (consensus params are expected to be constant).

Reported by linter in CI build, https://gitlab.com/dashpay/dash/-/jobs/6281174570#L99
sporkManager was the name of the global that's been removed a commit ago,
remove variable names that could be misread as the global still existing
Copy link
Collaborator

@knst knst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM a93de86

Copy link
Member

@PastaPastaPasta PastaPastaPasta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK a93de86

Copy link

@UdjinM6 UdjinM6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK

@PastaPastaPasta PastaPastaPasta merged commit 2110c0c into dashpay:develop Mar 15, 2024
kwvg added a commit to kwvg/dash that referenced this pull request Mar 18, 2024
As found in a review comment in dashpay#5908[1], validation.h is not needed for
specialtxman.cpp and removing the header uncovers other circular
dependencies that were obscured by the shortest circular path[2].

[1] - dashpay#5929 (comment)
[2] - dashpay#5929 (comment)
PastaPastaPasta added a commit that referenced this pull request Mar 19, 2024
… move C{CreditPool, NetFulfilledRequest}Manager to NodeContext, CDSTXManager to CJContext

991c9ec refactor: accept NodeContext arg into BlockAssembler instead of managers (Kittywhiskers Van Gogh)
c427305 refactor: remove CNetFulfilledRequestManager global, move to NodeContext (Kittywhiskers Van Gogh)
a53046c refactor: remove CDSTXManager global and alias, move to CJContext (Kittywhiskers Van Gogh)
f0451fb refactor: remove CCreditPoolManager global, move to NodeContext (Kittywhiskers Van Gogh)
67cfee7 refactor: remove CMNHFManager::GetSignalsStage alias from CChainState (Kittywhiskers Van Gogh)
667852c refactor: cleanup CDSNotificationInterface member names, add asserts (Kittywhiskers Van Gogh)
0d6f736 fix: add missing entity for destruction in DashTestSetupClose (Kittywhiskers Van Gogh)
2dd6082 refactor: move special transaction processing into helper class (Kittywhiskers Van Gogh)

Pull request description:

  ## Additional Information

  * Dependent on #5908

  ## Breaking Changes

  None. Changes are limited to refactoring, no logical changes have been made.

  ## Checklist:

  - [x] I have performed a self-review of my own code
  - [x] I have commented my code, particularly in hard-to-understand areas **(note: N/A)**
  - [x] I have added or updated relevant unit/integration/functional/e2e tests
  - [x] I have made corresponding changes to the documentation **(note: N/A)**
  - [x] I have assigned this pull request to a milestone _(for repository code-owners and collaborators only)_

ACKs for top commit:
  PastaPastaPasta:
    utACK 991c9ec

Tree-SHA512: 7a8d90416b6de4915639908b5f6bd59caee73e26462864d12a14a5d74af300a3733d8d9b0863e267b82d2807a8a5753acf5574306a0493bb7f3d83d8f0a7da4f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants