-
Notifications
You must be signed in to change notification settings - Fork 1.2k
privatesend|wallet|qt: Improve calculations in CreateDenominated/MakeCollateralAmounts #3657
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
privatesend|wallet|qt: Improve calculations in CreateDenominated/MakeCollateralAmounts #3657
Conversation
efb87d2 to
3d00601
Compare
|
tl;dr:
|
Isn't a partial revert like in 1365db1 enough? This way we can still say 100% sure that the same feerate is used for change dust consideration.
Ops, good catch!
Picked them all 👍 |
ab18ddd to
7502fcb
Compare
|
@UdjinM6 While reviewing your suggestions more precisely i realized that your refactoring of Lets assume With 2b01a09 applied: With the initial commits: With the force push i dropped 2b01a09, 7c30d5d, f90e626 again. The latter two because they bring no drawbacks but IMO without 2b01a09 it just brings more readability to keep them.. still feel free to complain again if you prefer to have them dropped for some reason. There are also the new commits 94529da, 3cb7d2d and 7502fcb |
|
Makes sense. Still don't see a reason for |
Okaaay... just one more time a bit more detailed in case you didn't get the reasoning i have in mind for it: I can't say for sure that there is no chance for the code down in the deeps of |
UdjinM6
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a couple of nits
|
Oh, by 👍 I meant that it makes sense and I understand the reason now 🙈 pls ignore my previous suggestion and drop 5fb050b |
Ahhh ops, i took it as the "go drop it thumbs up" 😅 |
5fb050b to
7502fcb
Compare
|
Pls see https://github.com/UdjinM6/dash/commits/pr3657 (starting with EDIT: tl;dr: change outputs should be handled automatically, size calculations need to be more precise, some minor tweaks (off-by-one, types, logs, comments) |
|
Overall nice suggestions but im not a big fan of 703922203ecfa8f18a2c0c9ed853d91ab7c3fc36 yet, convince me :D Why does it matter in this case if the change is not handled automatically? The idea of doing it manually was to have two outputs with the same amount in this case as this imo reduces potential false positive detection even more for the "make collateral amounts tx type". Edit: Also there is no "Make sure its not denominated" anymore for the change if its done automatically as the amount is not set through |
Added f6f0f8ac72. Try mixing with
Right, it's not really used after 7039222 (wasn't ever needed in |
|
Nice catch with the extra byte due to the compact serialize size 👍 I added some commits related to that to wrap it up a bit and to also respect this when trying to add outputs in 8b19c551fd85840aeadeaf64f04ffe655cd6072f is also a good catch! But i modified it a bit in 05ee734, thoughts? And i picked the others except 703922203ecfa8f18a2c0c9ed853d91ab7c3fc36. But i will have a better look at it and your points again later if i get the chance or tomorrow! |
PastaPastaPasta
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did a limited code review, didn't review everything.
Big takeaway so far:
I'm not a huge fan of adding ~320 LOC, but that's probably okay.
CTransactionBuilder really needs a good amount of unit tests
| CTransactionBuilderOutput* out = txBuilder.AddOutput(); | ||
| out->UpdateAmount(getAdjustedAmount(txBuilder.GetAmountLeft())); | ||
|
|
||
| assert(CPrivateSend::IsCollateralAmount(out->GetAmount())); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These asserts kindof scare me tbh...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why exactly?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I could imagine a situation where somehow something gets changed in such a way that it's too much or too little to pass this assert, and this isn't caught in testing. I would rather log an error, and return false than assert and crash the application. You could maybe use something like
Line 1546 in fea0036
| bool AbortNode(const std::string& strMessage, const std::string& userMessage="") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thoughts @UdjinM6 ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@UdjinM6 I would like your thoughts here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm imo its nothing to worry about, we have unit tests and
dash/src/privatesend/privatesend-client.cpp
Line 1442 in 9a7b36a
| if (!txBuilder.CouldAddOutput(CPrivateSend::GetCollateralAmount())) { |
dash/src/privatesend/privatesend-client.cpp
Line 1460 in 9a7b36a
| } else if (txBuilder.CouldAddOutputs({CPrivateSend::GetCollateralAmount(), CPrivateSend::GetCollateralAmount()})) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
imo it's ok
| CAmount CTransactionBuilder::GetAmountLeft(const CAmount nAmountInitial, const CAmount nAmountUsed, const CAmount nFee) | ||
| { | ||
| return nAmountInitial - nAmountUsed - nFee; | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure I understand the point of this function at this point is it's just a glorified subtraction function...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Jup its just a subtraction helper. https://github.com/dashpay/dash/blob/e7dc9da05c9e6cf6be4173ad65aecbc611cd1992/src/privatesend/privatesend-util.h#L119 imo it improves the readability of e.g.
As it wraps the subtrahends in GetAmountLeft() so you can just see by reading the wrappers name what the point of the return is. I don't see any drawback by doing it like this as the compiler should be smart enough to take care of it, but i neither will fight for keeping it lol.. so if you see any problems with this, just complain again.
|
|
||
| fKeepKeys = true; | ||
|
|
||
| strResult = wtx.GetHash().ToString(); | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would say remove a few of these empty lines, probably line 300 at least
e7dc9da to
e42c43f
Compare
2f5ede3 to
e755205
Compare
|
Rebased on develop + e755205 |
UdjinM6
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems to be working as expected (together with #3668 )
ACK
b857b1c to
7814ff3
Compare
|
Picked 87b7431, 2543a4c and bcdcb13 (squashed the 3 typo commits) 👍 About the others: 71d49f9bd57dc0937358001f5638f1c506b114dc and 6c34988801d640a91d624a3ab8cc7ae70ab0cf58: does it really bring any improvement / make a difference to use explicit casts there? IMO this just blows it up unnecessarily. We will never reach anything near eaeb50e7a61dd6125f9cde76977e76770fed29be: why? |
eaeb50e resolves a clang-tidy warning 71d49f9#diff-7d55c5a40c548772705e07a8dff7a49eR188-R189 L189 specifically resolves a clang-tidy, and the rest imo it's good standard to validate when going from unsigned int -> int or long -> int or really any narrowing conversion, that we aren't having something weird happen. I think the explicit cast on L249 also resolves clang-tidy warning 6c34988 resolves clang-tidy warning |
Signed-off-by: pasta <[email protected]>
Signed-off-by: pasta <[email protected]>
estimateSmartFee handles it if negative Signed-off-by: pasta <[email protected]>
|
Ok, seems like im just not a big fan of some of those clang-tidy recommendations lol.. not the first time i noticed that haha.. and i really wonder if we should always follow them.. especially eaeb50e is one i can't understand at all 😆 anyway, they don't do anything bad so i just picked them. |
|
|
It's not that i didn't get that it doesn't make a difference :D I just don't see the point why clang-tidy prefers to introduce inconsistencies between declaration and definition. I mean also works, but you also don't leave variable names out of the declaration only because they are ignored by the compiler and they don't have an impact on the definition, or is this also something clang-tidy recommends? |
Co-authored-by: UdjinM6 <[email protected]>
Seems like most of the thought process is the fact that something is const doesn't change how someone is going to call that function, whereas knowing the names is pretty valuable |
UdjinM6
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
utACK
|
I think I'm ready to ACK once those comments become doxygen |
PastaPastaPasta
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK, code review, as well as mixed a good bit on testnet
…CollateralAmounts (dashpay#3657) * wallet: Add m_dust_feerate to CCoinControl / Use it in CreateTransaction * privatesend: Introduce CTransactionBuilder/CTransactionBuilderOutput Builder classes for transactions from type Inputs: Defined by CompactTallyItem Outputs: Simple outputs with lose reserve keys This takes fully takes care of fee calculations and makes sure calculations are the same like those happening when actually create the transaction with CreateTransaction. * privatesend: Improve amount/fee calculation in CreateDenominated * privatesend: Improve amount/fee calculation in MakeCollateralAmounts * qt: Fix decomposeTransaction's MakeCollateralAmounts detection Align it with the three cases in CPrivateSendClientSession::MakeCollateralAmounts * Refactor GetFee The fee rate is always coinControl.m_feerate, also it's not used outside so should be a private method * Simplify nBytesOutput calculations * Drop unused GetBytesOutput() * Make Clear(), GetBytesTotal() and GetAmountUsed() private * Drop unused GetCoinControl() * Make ToString() const * Refactor `CTransactionBuilder::Commit()` * Reorder cases in decomposeTransaction * Fix "finished" conditions in CreateDenominated * Fix typo * wallet|privatesend: Refactor CCoinControl's m_dust_feerate -> m_discard_feerate * privatesend: Drop unused member CTransactionBuilder::dustFeeRate * privatesend: Improve CTransactionBuilder's readability * privatesend: Make the static CTransactionBuilder::GetAmountLeft private * wallet: Recover previous code style * Update src/privatesend/privatesend-util.cpp Co-authored-by: UdjinM6 <[email protected]> * Tweak CTransactionBuilder to respect potential compact size diffs * Tweak GetFee param type * Trivial log/comments tweaks * privatesend: Fix countPossibleOutputs - Fix off by one issue - Respect max outputs threshold * privatesend: Use GetSizeOfCompactSizeDiff in CTransactionBuilder * Apply suggestions from code review Co-authored-by: PastaPastaPasta <[email protected]> * privatesend: Rename TryAdd to CouldAdd in CTransactionBuilder * wallet: Reset m_discard_feerate in CCoinControl::SetNull * Respect `-paytxfee` and `settxfee` * privatesend: Check for denominated amount only where really required * qt: Remove obsolete IsCollateralAmount() checks * privatesend: Don't accept negative amounts in CTransactionBuilder * privatesend: Remove unused CConnman parameter * use emplace_back instead of push_back Signed-off-by: pasta <[email protected]> * fix typos Signed-off-by: pasta <[email protected]> * make GetAmount const Signed-off-by: pasta <[email protected]> * privatesend: Explicit capture __func__ in needMoreOutputs lambda * privatesend: Update CTransactionBuilder::UpdateAmount * remove const on parameter in declaration Signed-off-by: pasta <[email protected]> * handle unsigned int -> int conversions a bit better Signed-off-by: pasta <[email protected]> * explicitly cast to int from unsigned int. estimateSmartFee handles it if negative Signed-off-by: pasta <[email protected]> * Make CTransactionBuilderOutput::GetScript const Co-authored-by: UdjinM6 <[email protected]> * privatesend: Update comments to follow doxygen * privatesend: Add class descriptions Co-authored-by: UdjinM6 <[email protected]> Co-authored-by: PastaPastaPasta <[email protected]> Co-authored-by: pasta <[email protected]>
…CollateralAmounts (dashpay#3657) * wallet: Add m_dust_feerate to CCoinControl / Use it in CreateTransaction * privatesend: Introduce CTransactionBuilder/CTransactionBuilderOutput Builder classes for transactions from type Inputs: Defined by CompactTallyItem Outputs: Simple outputs with lose reserve keys This takes fully takes care of fee calculations and makes sure calculations are the same like those happening when actually create the transaction with CreateTransaction. * privatesend: Improve amount/fee calculation in CreateDenominated * privatesend: Improve amount/fee calculation in MakeCollateralAmounts * qt: Fix decomposeTransaction's MakeCollateralAmounts detection Align it with the three cases in CPrivateSendClientSession::MakeCollateralAmounts * Refactor GetFee The fee rate is always coinControl.m_feerate, also it's not used outside so should be a private method * Simplify nBytesOutput calculations * Drop unused GetBytesOutput() * Make Clear(), GetBytesTotal() and GetAmountUsed() private * Drop unused GetCoinControl() * Make ToString() const * Refactor `CTransactionBuilder::Commit()` * Reorder cases in decomposeTransaction * Fix "finished" conditions in CreateDenominated * Fix typo * wallet|privatesend: Refactor CCoinControl's m_dust_feerate -> m_discard_feerate * privatesend: Drop unused member CTransactionBuilder::dustFeeRate * privatesend: Improve CTransactionBuilder's readability * privatesend: Make the static CTransactionBuilder::GetAmountLeft private * wallet: Recover previous code style * Update src/privatesend/privatesend-util.cpp Co-authored-by: UdjinM6 <[email protected]> * Tweak CTransactionBuilder to respect potential compact size diffs * Tweak GetFee param type * Trivial log/comments tweaks * privatesend: Fix countPossibleOutputs - Fix off by one issue - Respect max outputs threshold * privatesend: Use GetSizeOfCompactSizeDiff in CTransactionBuilder * Apply suggestions from code review Co-authored-by: PastaPastaPasta <[email protected]> * privatesend: Rename TryAdd to CouldAdd in CTransactionBuilder * wallet: Reset m_discard_feerate in CCoinControl::SetNull * Respect `-paytxfee` and `settxfee` * privatesend: Check for denominated amount only where really required * qt: Remove obsolete IsCollateralAmount() checks * privatesend: Don't accept negative amounts in CTransactionBuilder * privatesend: Remove unused CConnman parameter * use emplace_back instead of push_back Signed-off-by: pasta <[email protected]> * fix typos Signed-off-by: pasta <[email protected]> * make GetAmount const Signed-off-by: pasta <[email protected]> * privatesend: Explicit capture __func__ in needMoreOutputs lambda * privatesend: Update CTransactionBuilder::UpdateAmount * remove const on parameter in declaration Signed-off-by: pasta <[email protected]> * handle unsigned int -> int conversions a bit better Signed-off-by: pasta <[email protected]> * explicitly cast to int from unsigned int. estimateSmartFee handles it if negative Signed-off-by: pasta <[email protected]> * Make CTransactionBuilderOutput::GetScript const Co-authored-by: UdjinM6 <[email protected]> * privatesend: Update comments to follow doxygen * privatesend: Add class descriptions Co-authored-by: UdjinM6 <[email protected]> Co-authored-by: PastaPastaPasta <[email protected]> Co-authored-by: pasta <[email protected]>
While the idea of respecting fees in calculations brought up in #3588 and #3640 makes totally sense i still noticed while reviewing/testing the latter that the estimation of just saying use min fee for
148bytes for inputs and min fee for34bytes for outputs and then sum it up isn’t good enough imo.So even with those estimations we end up in cases where the calculations are slightly off from the expected. This brings inaccuracy to transaction generation and leads to issues (e.g. the. one in
TransactionRecord::decomposeTransaction).As solution this PR introduces
CTransactionBuilderin 13fa395eeb0050225ce2f1a77418d1af96137b0e, a class which wraps transaction creation and allows a simple and failproof creation of transactions with inputs defined byCompactTallyItemwhile also respecting the logics implemented inCWallet::CreateTransaction. For output generation it takes care of key generation, provides methods e.g. to add/try to add outputs, get the remaining amount available, and methods to create/commit the transaction to the wallet. To avoid running into the issues described above it does not calculate a seperate fee for the amount of bytes for input/outputs like it was before. Instead it uses bytes for all calculations and calculates a total fee for the transaction when required. This way it’s possible now to calculate the correct amounts/fees used for the created transactions.Further on top this PR brings also some small fixes/modifications to the logic behind
CreateDenominatedandMakeCollateralAmountsin fba14a1 and 8e0191c so please review very carefully there that we can be sure i didn’t mess something here :)3d00601 finally makes sure #3597 is not longer an issue. As it now should cover all the cases produced by
MakeCollateralAmountscorrectly.