Skip to content

Conversation

@ariard
Copy link

@ariard ariard commented May 1, 2019

Work starter to remove Chain::Lock interface by adding m_last_block_processed_height in CWallet and m_block_height in CMerkleTx to avoid GetDepthInMainChain having to keep a lock . Once this one done, it should ease work to wipe out more cs_main locks from wallet code.

I think it's ready for a first round of review before to get further.

  • BlockUntilSyncedToCurrent : restrain isPotentialTip to isTip because we want to be sure that wallet see BlockDisconnected callbacks if its height differs from the Chain one. It means during a reorg, an RPC could return before the BlockDisconnected callback had been triggered. This could cause a tx that had been included in the disconnected block to be displayed as confirmed, for example.

- AbandonTransaction : in case of conflicted tx (nIndex = -1), we set its m_block_height to the one of conflicting blocks, but if this height is superior to CWallet::m_last_block_processed_height, that means tx isn't conflicted anymore so we return 0 as tx is again unconfirmed After #16624, we instead rely on Confirmation.

- AddToWalletIfInvolvingMe: in case of block disconnected, transactions are added to mempool again, so we need to replace old txn in mapWallet with a height set to zero so we remove check on block_hash.IsNull Already done in #16624

@DrahtBot
Copy link
Contributor

DrahtBot commented May 1, 2019

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #17369 (Refactor: Move encryption code between KeyMan and Wallet by achow101)
  • #17261 (Make ScriptPubKeyMan an actual interface and the wallet to have multiple by achow101)
  • #17060 (Cache 26% more coins: Reduce CCoinsMap::value_type from 96 to 76 bytes by martinus)
  • #16944 (gui: create PSBT with watch-only wallet by Sjors)
  • #16910 (wallet: reduce loading time by using unordered maps by achow101)
  • #16895 (External signer multisig support by Sjors)
  • #16688 (log: Add validation interface logging by jkczyz)
  • #16549 ([WIP] UI external signer support (e.g. hardware wallet) by Sjors)
  • #16546 ([WIP] External signer support - Wallet Box edition by Sjors)
  • #16528 ([WIP] Native Descriptor Wallets (take 2) by achow101)
  • #9381 (Remove CWalletTx merging logic from AddToWallet by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@ariard ariard force-pushed the 2019-04-remove-external-locking branch from 5653f46 to a0ef689 Compare May 2, 2019 19:30
@ariard ariard force-pushed the 2019-04-remove-external-locking branch 3 times, most recently from adf914a to 773da90 Compare May 6, 2019 18:53
@ariard ariard force-pushed the 2019-04-remove-external-locking branch from 773da90 to 8b66249 Compare May 13, 2019 13:52
@ariard ariard force-pushed the 2019-04-remove-external-locking branch from 8b66249 to e284e42 Compare May 29, 2019 23:19
@fanquake
Copy link
Member

Added to the "Chasing Concept ACK" board. @ryanofsky maybe you could give an initial Concept/Approach ACK/NACK ?

@ariard
Copy link
Author

ariard commented Jun 19, 2019

Yes would be awesome to get a Concept ACK on the approach followed but if it's not the best one, I'm eager to rework the PR in depth! Going to rebase/fix tests failure soon

Copy link
Contributor

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK. I left a lot of suggestions, but overall this looks very good, and it's great to get rid of all these cs_main locks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "Use CWallet::m_last_block_processed_height in GetDepthInMainChain" (233ae92f2d51333f158810d85d13cf63a1144e6b)

Can we use -1 instead of 0 for this to be consistent with nIndex? Also would be good to note here that when a transaction is conflicted, hashBlock and block_height refer to the earliest block with a conflicting transaction instead of the block containing the transaction.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment says An block_height == 0 in commit Add m_block_height field in CMerkleTx and then changes to A block_height == -1 in Use CWallet::m_last_block_processed_height in GetDepthInMainChain.

It would be better to not change this in the course of the PR and just set it to A block_height == -1 in the first PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "Use CWallet::m_last_block_processed_height in GetDepthInMainChain" (233ae92f2d51333f158810d85d13cf63a1144e6b)

Isn't this going to break existing serialization of transactions in the wallet? I think it'd be best not to change the serialized representation and just make this a memory-only field that gets filled when the wallet is loaded.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm filled at startup using getBlockHeight, given we already have hashBlock ? Do you envision the wallet to always query chain state to succeed its startup ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "Add m_block_height field in CMerkleTx" (8a1975586d3cc7f98961581859fd146a632bbd34)

This logic is getting hard to follow with three repetitive if(hashBlock...) blocks. Now that this is unconditionally updating hashBlock, I think the new logic is equivalent to the following and can be simplified:

assert(!wtxIn.isAbandoned()); // Incoming transactions should never have special abandoned block hash
if (wtxIn.hashBlock != wtx.hashBlock) {
    wtx.hashBlock = wtxIn.hashBlock;
    wtx.m_block_height = wtxIn.m_block_height;
    fUpdated = true;
}

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated AddToWallet, hope it's clearer

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "Add m_block_height field in CMerkleTx" (8a1975586d3cc7f98961581859fd146a632bbd34)

It looks like a remaining place where m_block_height is still unset is in importprunedfunds. It would probably be good to change that code to call wtx.SetMerkleBranch instead of setting members directly.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oooops, I forgot this one. I've added a call to SetMerkleBranch in importprunedfunds but had to change its argument to hask for transaction height too. Seems to me similar to the serialization issue, do we want wallet to ask chain the state of its stored transactions every time, even it should be able to infer it from the blocks already connected ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "Use CWallet::m_last_block_processed_height in GetDepthInMainChain" (233ae92f2d51333f158810d85d13cf63a1144e6b)

Can we assert(m_block_height > 0) after this point? I'm afraid of another bug like the importprunedfunds one mentioned earlier leaving m_block_height unset and this function returning bogus values.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, I introduced assert(m_block_height >= -1) as we may GetDepthInMainChain on fresh transactions, their hashBlock being unset, their depth should appear as zero.

Copy link
Contributor

@promag promag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK.

@ariard ariard force-pushed the 2019-04-remove-external-locking branch 2 times, most recently from cc9bd04 to 1458ded Compare July 9, 2019 01:46
@ariard
Copy link
Author

ariard commented Jul 9, 2019

Answered back to @ryanofsky comments on 1458ded, I'm on cleaning last Travis failures

@ariard ariard force-pushed the 2019-04-remove-external-locking branch 4 times, most recently from 1697ee4 to 04b4683 Compare July 16, 2019 02:22
Munkybooty pushed a commit to Munkybooty/dash that referenced this pull request Jan 30, 2022
05b56d1 [wallet] Remove CMerkleTx serialization logic (John Newbery)
783a76f [wallet] Flatten CWalletTx class hierarchy (John Newbery)
b3a9d17 [wallet] Move CMerkleTx functions into CWalletTx (John Newbery)

Pull request description:

  CMerkleTx is only used as a base class for
  CWalletTx. It was previously also used for vtxPrev which
  was removed in 93a18a3.

  This PR moves all of the CMerkleTx members and logic
  into CWalletTx. The CMerkleTx class is kept for deserialization
  and serialization of old wallet files.

  This makes the refactor in bitcoin#15931 cleaner.

ACKs for top commit:
  laanwj:
    ACK 05b56d1. Looks good to me.

Tree-SHA512: 3d3a0069ebb536b12a328f1261e7dc55158a71088d445ae4b4ace4142c432dc296f58c8183b1922e54a60b8cc77e9d17c3dce7478294cd68693594baacf2bab3
kwvg added a commit to kwvg/dash that referenced this pull request Apr 5, 2022
kwvg added a commit to kwvg/dash that referenced this pull request Apr 19, 2022
PastaPastaPasta added a commit to dashpay/dash that referenced this pull request Apr 20, 2022
malbit added a commit to malbit/raptoreum that referenced this pull request Aug 7, 2022
[bitcoin#14624](bitcoin/bitcoin#14624)
bls refactoring to resolve clang warnings
[bitcoin#16426](bitcoin/bitcoin#16426) - cs_wallet lock order refactoring and reduce cs_main locking
atomic smartnode_connection (allow read/write by different threads simultaneously)
cs_mnauth locks on verifiedProRegTxHash read
RecursiveMutex at locking access to activeSmartNode
[bitcoin#14307](bitcoin/bitcoin#14307) - Consolidate redundant implementation of ParseHashStr
[bitcoin#13399](bitcoin/bitcoin#13399) - rpc: Add submitheader
built-in miner deleted
[bitcoin#17781](bitcoin/bitcoin#17781) - Remove mempool global from miner
[bitcoin#16623](bitcoin/bitcoin#16624) - encapsulate transactions state
[bitcoin#15931](bitcoin/bitcoin#15931) - Remove GetDepthInMainChain dependency on locked chain interface
Pass CConnman to function against global pointer
[bitcoin#16839](bitcoin/bitcoin#16839) - Replace Connman and BanMan globals with NodeContext local
[bitcoin#16034](bitcoin/bitcoin#16034) - refactoring: Rename LockAnnotation to LockAssertion and add run-time check to it
[bitcoin#17564](bitcoin/bitcoin#17564) - Use mempool from node context instead of global
[bitcoin#18740](bitcoin/bitcoin#18740) - Remove g_rpc_node global
[bitcoin#19096](bitcoin/bitcoin#19096) - Remove g_rpc_chain global
[bitcoin#18338](bitcoin/bitcoin#18338) - Fix wallet unload race condition
other fixes and tweaks
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Aug 16, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.