-
Notifications
You must be signed in to change notification settings - Fork 38.7k
validation/coins: limit MemPoolAccept access to coins cache + make it responsible for uncaching #21146
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
932466c to
14c94b3
Compare
|
Do you think you can take the following diff in the doc followup commit ? That would close my #21704 concerns for now :) ? |
14c94b3 to
afe2959
Compare
|
Rebased |
jnewbery
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Final code looks good. I have one comment inline about the commit structure.
There's a small change in the final commit that you haven't communicated: currently, if a testmempoolaccept is successful, we keep any fetched coins in the cache. After this commit we don't. I think that's a fine change - if the transaction hasn't been added to the mempool, then there's no reason for the coins it spends to be added to the cache. The change is perhaps worth pointing out in the commit log though.
afe2959 to
acef823
Compare
|
ACK acef823e895b80fd7f4c2f5a44c3470fe46c1a6a |
ariard
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should you extend 34ab1ad7 commit message to explain why it's safe to access output in the mempool ? I would say for the mempool, we don't have overrun concerns, all the state is already in-memory. No size extension from disk is possible.
src/validation.cpp
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would rather keep coin cache cleanup at this location. After this change, it's spread in two locations : in case of success evaluation in AcceptSingleTransaction, in case of failure in MemPoolAccept destructor. A reviewer has to read both locations now.
Further, it's harder to qualify that cleanup is reserved for success path in AcceptSingleTransaction, considering that MemPoolAcceptResult internal state is provided by Workspace.
Also, I don't know about relying on RAII semantic, the destructor will be only called when object is going out of scope. In that case, does it mean we'll only cleanup when caller ends up ? For e.g, if the caller was BroadcastTransaction, may the coin cache cleanup on cs_main lock tacking before RelayTransaction ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After this change, it's spread in two locations : in case of success evaluation in AcceptSingleTransaction, in case of failure in MemPoolAccept destructor.
Not really. The only location a reviewer has to check is wherever they want to keep the coins in cache. Also, the change here means that if someone forgets to consider uncaching, it defaults to uncache all. This is safer.
Also, I don't know about relying on RAII semantic, the destructor will be only called when object is going out of scope. In that case, does it mean we'll only cleanup when caller ends up ? For e.g, if the caller was BroadcastTransaction, may the coin cache cleanup on cs_main lock tacking before RelayTransaction ?
The two possible callers of MemPoolAccept are ATMP and ProcessNewPackage They invoke it like this:
const MempoolAcceptResult result = MemPoolAccept(pool, active_chainstate).AcceptSingleTransaction(tx, args);afaik it goes out of scope immediately, before the lock is released.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not really. The only location a reviewer has to check is wherever they want to keep the coins in cache.
Depends what you're willingly to convince yourself by reviewing. Assuming uncaching in case of failures is the correct behavior, looking at the proposed changes, you have to a) verify that uncaching code is well-implemented in MemPoolAccept destructor, b) identify that caching conservation is covering all success paths and thus dissociate success code path from errors one in AcceptSingleTransaction and c) ensure the RAAI semantic is enforced as intended. IMHO, more complex than current one-place location.
That said, I think the code is correct, and each one review process and code style flavors might turn as really abstract discussion so not going to argue further :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The two possible callers of MemPoolAccept are ATMP and ProcessNewPackage They invoke it like this:
const MempoolAcceptResult result = MemPoolAccept(pool, active_chainstate).AcceptSingleTransaction(tx, args);afaik it goes out of scope immediately, before the lock is released.
I believe you're right. In that expression, MemPoolAccept() is a temporary, and the lifetime rules for temporaries are
All temporary objects are destroyed as the last step in evaluating the full-expression that (lexically) contains the point where they were created, and if multiple temporary objects were created, they are destroyed in the order opposite to the order of creation. This is true even if that evaluation ends in throwing an exception.
Note that there are two exceptions to that rule, neither of which apply here:
- The lifetime of a temporary object may be extended by binding to a const lvalue reference or to an rvalue reference (since C++11), see reference initialization for details.
- The lifetime of a temporary object created when evaluating the default arguments of a default constructor used to initialize an element of an array ends before the next element of the array begins initialization.
jnewbery
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
utACK acef823e895b80fd7f4c2f5a44c3470fe46c1a6a
The hierarchy of CoinsViews here is: m_view -> m_viewmempool -> chainstate coins cache To prevent DoS, we want to limit access to coins cache and ensure we uncache coins if the transaction turns out to be invalid. When m_viewmempool is always connected to the coins cache, it's harder reason about this. We could accidentally access the coins cache if we use m_viewmempool for something.
MemPoolAccept should clean up after itself (woohoo RAII), uncache all coins by default, and explicitly decide to keep them on specific occasions (i.e. when tx is submitted to mempool). This is safer and easier to reason about than leaving it up to the caller. Also allows us to make ATMPArgs always const. Note: with this commit, we always uncache coins after a test_accept regardless of validation results. Previously, we uncached whenever a tx failed validation.
acef823 to
aeaf540
Compare
|
I'm working on #20833 and realized I don't need to keep Also addressed @jnewbery's comments #21146 (comment) and #21146 (comment). |
|
I still think this would be a nice change but since I don't need this for #20833 anymore, I'm closing this to focus on that instead. |
This PR is a subpart of #20833 which enables package validation through
testmempoolaccept.The first commit addresses some leftover comments from #21062.
When validating unconfirmed transactions, we want to account for all coins brought in from disk so we can uncache them if the tx is invalid. The disconnection of
CCoinsViews right now doesn't quite achieve what's it's trying to do (preventMemPoolAcceptfrom unnecessarily accessing the coins cache and forgetting to uncache later). This PR improves it by doing 2 things:(1) Disconnect
m_viewmempoolfrom the coins cache when we don't need it to be connected.(2) Make
MemPoolAcceptdestructor clean up after itself by uncaching coins accessed, instead of leaving it up to the caller. Only leave the coins cached when a transaction is accepted to mempool. In the future, if someone edits theMemPoolAcceptclass and forgets to consider uncaching, the default behavior is to uncache (which is safer).