Skip to content

Conversation

@glozow
Copy link
Member

@glozow glozow commented Feb 10, 2021

This PR is a subpart of #20833 which enables package validation through testmempoolaccept.
The first commit addresses some leftover comments from #21062.

When validating unconfirmed transactions, we want to account for all coins brought in from disk so we can uncache them if the tx is invalid. The disconnection of CCoinsViews right now doesn't quite achieve what's it's trying to do (prevent MemPoolAccept from unnecessarily accessing the coins cache and forgetting to uncache later). This PR improves it by doing 2 things:
(1) Disconnect m_viewmempool from the coins cache when we don't need it to be connected.
(2) Make MemPoolAccept destructor clean up after itself by uncaching coins accessed, instead of leaving it up to the caller. Only leave the coins cached when a transaction is accepted to mempool. In the future, if someone edits the MemPoolAccept class and forgets to consider uncaching, the default behavior is to uncache (which is safer).

@DrahtBot
Copy link
Contributor

DrahtBot commented Feb 11, 2021

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Conflicts

Reviewers, this pull request conflicts with the following ones:

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@glozow glozow changed the title [WIP] validation/coins: limit MemPoolAccept access to coins cache validation/coins: limit MemPoolAccept access to coins cache + make it responsible for uncaching Feb 11, 2021
@glozow glozow marked this pull request as ready for review February 11, 2021 22:21
@ariard
Copy link

ariard commented Feb 12, 2021

Do you think you can take the following diff in the doc followup commit ?

diff --git a/src/rpc/rawtransaction.cpp b/src/rpc/rawtransaction.cpp
index ac4240447..f8f4a6d94 100644
--- a/src/rpc/rawtransaction.cpp
+++ b/src/rpc/rawtransaction.cpp
@@ -896,7 +896,7 @@ static RPCHelpMan testmempoolaccept()
                         {
                             {RPCResult::Type::STR_HEX, "txid", "The transaction hash in hex"},
                             {RPCResult::Type::STR_HEX, "wtxid", "The transaction witness hash in hex"},
-                            {RPCResult::Type::BOOL, "allowed", "If the mempool allows this tx to be inserted"},
+                            {RPCResult::Type::BOOL, "allowed", "If the mempool and client specified maxfeerate allows this tx to be inserted"},
                             {RPCResult::Type::NUM, "vsize", "Virtual transaction size as defined in BIP 141. This is different from actual serialized size for witness transactions as witness data is discounted (only present when 'allowed' is true)"},
                             {RPCResult::Type::OBJ, "fees", "Transaction fees (only present if 'allowed' is true)",
                             {

That would close my #21704 concerns for now :) ?

@glozow glozow force-pushed the 2021-02-coinsviews branch from 14c94b3 to afe2959 Compare February 20, 2021 17:44
@glozow
Copy link
Member Author

glozow commented Feb 20, 2021

Rebased

Copy link
Contributor

@jnewbery jnewbery left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final code looks good. I have one comment inline about the commit structure.

There's a small change in the final commit that you haven't communicated: currently, if a testmempoolaccept is successful, we keep any fetched coins in the cache. After this commit we don't. I think that's a fine change - if the transaction hasn't been added to the mempool, then there's no reason for the coins it spends to be added to the cache. The change is perhaps worth pointing out in the commit log though.

@glozow glozow force-pushed the 2021-02-coinsviews branch from afe2959 to acef823 Compare February 22, 2021 17:32
@glozow
Copy link
Member Author

glozow commented Feb 22, 2021

Thanks for the review @jnewbery! Addressed your comments. Added some sentences about the uncaching behavior to acef823 commit log.

@achow101
Copy link
Member

ACK acef823e895b80fd7f4c2f5a44c3470fe46c1a6a

Copy link

@ariard ariard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should you extend 34ab1ad7 commit message to explain why it's safe to access output in the mempool ? I would say for the mempool, we don't have overrun concerns, all the state is already in-memory. No size extension from disk is possible.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather keep coin cache cleanup at this location. After this change, it's spread in two locations : in case of success evaluation in AcceptSingleTransaction, in case of failure in MemPoolAccept destructor. A reviewer has to read both locations now.

Further, it's harder to qualify that cleanup is reserved for success path in AcceptSingleTransaction, considering that MemPoolAcceptResult internal state is provided by Workspace.

Also, I don't know about relying on RAII semantic, the destructor will be only called when object is going out of scope. In that case, does it mean we'll only cleanup when caller ends up ? For e.g, if the caller was BroadcastTransaction, may the coin cache cleanup on cs_main lock tacking before RelayTransaction ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After this change, it's spread in two locations : in case of success evaluation in AcceptSingleTransaction, in case of failure in MemPoolAccept destructor.

Not really. The only location a reviewer has to check is wherever they want to keep the coins in cache. Also, the change here means that if someone forgets to consider uncaching, it defaults to uncache all. This is safer.

Also, I don't know about relying on RAII semantic, the destructor will be only called when object is going out of scope. In that case, does it mean we'll only cleanup when caller ends up ? For e.g, if the caller was BroadcastTransaction, may the coin cache cleanup on cs_main lock tacking before RelayTransaction ?

The two possible callers of MemPoolAccept are ATMP and ProcessNewPackage They invoke it like this:

const MempoolAcceptResult result = MemPoolAccept(pool, active_chainstate).AcceptSingleTransaction(tx, args);

afaik it goes out of scope immediately, before the lock is released.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really. The only location a reviewer has to check is wherever they want to keep the coins in cache.

Depends what you're willingly to convince yourself by reviewing. Assuming uncaching in case of failures is the correct behavior, looking at the proposed changes, you have to a) verify that uncaching code is well-implemented in MemPoolAccept destructor, b) identify that caching conservation is covering all success paths and thus dissociate success code path from errors one in AcceptSingleTransaction and c) ensure the RAAI semantic is enforced as intended. IMHO, more complex than current one-place location.

That said, I think the code is correct, and each one review process and code style flavors might turn as really abstract discussion so not going to argue further :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The two possible callers of MemPoolAccept are ATMP and ProcessNewPackage They invoke it like this:

const MempoolAcceptResult result = MemPoolAccept(pool, active_chainstate).AcceptSingleTransaction(tx, args);

afaik it goes out of scope immediately, before the lock is released.

I believe you're right. In that expression, MemPoolAccept() is a temporary, and the lifetime rules for temporaries are

All temporary objects are destroyed as the last step in evaluating the full-expression that (lexically) contains the point where they were created, and if multiple temporary objects were created, they are destroyed in the order opposite to the order of creation. This is true even if that evaluation ends in throwing an exception.

https://en.cppreference.com/w/cpp/language/lifetime#Temporary_object_lifetime

Note that there are two exceptions to that rule, neither of which apply here:

  • The lifetime of a temporary object may be extended by binding to a const lvalue reference or to an rvalue reference (since C++11), see reference initialization for details.
  • The lifetime of a temporary object created when evaluating the default arguments of a default constructor used to initialize an element of an array ends before the next element of the array begins initialization.

Copy link
Contributor

@jnewbery jnewbery left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK acef823e895b80fd7f4c2f5a44c3470fe46c1a6a

The hierarchy of CoinsViews here is:
m_view -> m_viewmempool -> chainstate coins cache

To prevent DoS, we want to limit access to coins cache and ensure we
uncache coins if the transaction turns out to be invalid.  When
m_viewmempool is always connected to the coins cache, it's harder reason
about this. We could accidentally access the coins cache if we use
m_viewmempool for something.
MemPoolAccept should clean up after itself (woohoo RAII), uncache all
coins by default, and explicitly decide to keep them on specific
occasions (i.e. when tx is submitted to mempool). This is safer and
easier to reason about than leaving it up to the caller.
Also allows us to make ATMPArgs always const.

Note: with this commit, we always uncache coins after a test_accept
regardless of validation results. Previously, we uncached whenever a tx
failed validation.
@glozow glozow force-pushed the 2021-02-coinsviews branch from acef823 to aeaf540 Compare February 25, 2021 17:33
@glozow
Copy link
Member Author

glozow commented Feb 25, 2021

I'm working on #20833 and realized I don't need to keep m_view and m_viewmempool connected for it to work 😛 so I took out the 3rd commit ([validation] keep m_view->m_viewmempool connected). @ariard you might find this PR a bit more acceptable now :). I'll try to add #21146 (comment) to a different PR that's more relevant to the RPC code.

Also addressed @jnewbery's comments #21146 (comment) and #21146 (comment).

@glozow
Copy link
Member Author

glozow commented Mar 2, 2021

I still think this would be a nice change but since I don't need this for #20833 anymore, I'm closing this to focus on that instead.

@glozow glozow closed this Mar 2, 2021
@bitcoin bitcoin locked as resolved and limited conversation to collaborators Aug 16, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants