-
Notifications
You must be signed in to change notification settings - Fork 38.7k
Add 'subtractFeeFromAmount' option to 'fundrawtransaction'. #9222
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add 'subtractFeeFromAmount' option to 'fundrawtransaction'. #9222
Conversation
|
Concept ACK. |
|
Nice. Concept ACK. |
|
@jonasschnelli I tried finding the fundrawtransaction tests but couldn't. Where are they? src/test/rpc_tests.cpp seems like the natural place for them, but I see no 'fund' in there at all. |
|
@dooglus |
|
@jonasschnelli Thanks for pointing me at the I have added tests for |
|
Concept ACK |
src/wallet/rpcwallet.cpp
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will thrown an exception if one of the elements in the array is numeric. But I think this is okay.
|
Code Review ACK a979010c80d5875ab26c9cdd5401e2b2905dd572. |
|
To 'squash' the commits do I just rewrite the same branch with a |
|
@dooglus: Yes. I normally do a |
a979010 to
64955cf
Compare
|
@jonasschnelli Thanks. The 'i' is lowercase and the 'HEAD' is uppercase but it was close enough. I used |
|
utACK |
qa/rpc-tests/fundrawtransaction.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: you can condense this to:
assert(result[0]['fee']==result[1]['fee']== result[2]['fee']==result[3]['fee'])
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting. I didn't know Python did that. I will do as you suggest.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed in 6a41954.
qa/rpc-tests/fundrawtransaction.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. Addressed in 6a41954.
src/wallet/rpcwallet.cpp
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should it throw an error if the given address is valid but is not among the outputs? (would have to check for this below, after retrieving the transaction). It seems like in this case, the user is trying to pay the fee with one of the outputs but has made an error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wanted it to behave the same as it does in sendmany, where it doesn't complain if you include an address that isn't a recipient at all.
The user could have a list of addresses which should pay fees when sent to, and use that same list as their subtractFeeFromAmount parameter whichever addresses they are sending to.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One difference between this and sendmany is that sendmany requires transaction outputs to be base58 addresses, and takes amounts and subtractfeefromamount arguments in base58 form, while fundrawtransaction allows outputs to be arbitrary scripts. This means with the PR in its current form, there may be no way for the new subtractFeeFromAmount argument to refer to certain outputs.
Instead of adding a subtractFeeFromAmount argument, I might suggest adding a subtractFeeFromPositions argument that takes a list of integer output indices. This would give callers the flexibility to refer to all outputs, be more consistent with the existing changePosition argument (which is also an integer output index), and also eliminate the need for ExtractDestination and CBitcoinAddress::ToString invocations in CWallet::FundTransaction.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ryanofsky I like the idea but am a bit worried about the interaction of subtractFeeFromPositions and changePosition. It might not be clear to the user if the position marking is done before or after change output is added, or discount the wrong output by adding a changePosition argument.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At the time of running fundrawtransaction there is no change output, and the user wouldn't know where the change will be inserted, so the position marking must be done before the change output is added.
I think since it is possible to add arbitrary hex output scripts which may not even have a corresponding address we need to be able to address the outputs by number rather than by address. It's also kind of ugly having to give the same address twice, once to createrawtransaction and then again to fundrawtransaction. I think using the output index (0 based) is cleaner.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@dooglus the user will "know" where change is going if they attempt to set the change index they're setting in the option, which is my point. It's not plainly clear how this should interact, unless you spell it out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh, I see. So I should spell it out...
I think it makes sense to use the position indices before the change output is added.
|
Addressed @mrbandrews' nits. Should I re-squash now, or leave the 'nit' commit separate for a while? |
|
re-utACK 6a41954 @dooglus good question, its not always clear. I personally think that if the prior code is not broken , then its ok not to squash. |
|
utACK 6a41954895460a033afc352af5c137418591cc6b |
src/wallet/rpcwallet.cpp
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One difference between this and sendmany is that sendmany requires transaction outputs to be base58 addresses, and takes amounts and subtractfeefromamount arguments in base58 form, while fundrawtransaction allows outputs to be arbitrary scripts. This means with the PR in its current form, there may be no way for the new subtractFeeFromAmount argument to refer to certain outputs.
Instead of adding a subtractFeeFromAmount argument, I might suggest adding a subtractFeeFromPositions argument that takes a list of integer output indices. This would give callers the flexibility to refer to all outputs, be more consistent with the existing changePosition argument (which is also an integer output index), and also eliminate the need for ExtractDestination and CBitcoinAddress::ToString invocations in CWallet::FundTransaction.
src/wallet/wallet.cpp
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would suggest changing the new set<string> argument to set<int> to be consistent with the existing nChangePosInOut argument which refers to an output by integer index instead of base58 address string. This would give callers more flexibility in referring to outputs and also simplify handling of the new argument below.
src/wallet/rpcwallet.cpp
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe s/If no addresses are specified here/If no addresses specified here are outputs in the transaction
src/wallet/rpcwallet.cpp
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe mention after This will not modify existing inputs, and will add one change output to the outputs above that no existing outputs will be modified either unless subtractFeeFromAmount is specified.
qa/rpc-tests/fundrawtransaction.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Debug output will be a little better if you use assert_equal instead of assert here and below.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It appears that assert_equal can only compare two things. For cases like assert(A == B == C == D) would you prefer 3 separate assert_equal() calls instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense. In case something fails we have the verbose output.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I actually only meant to suggest using assert_equal for binary comparisons like the one on line 698. But if you wanted to use it more broadly, you could extend the function (in util.py) to accept more arguments:
def assert_equal(thing1, thing2, *args):
if thing1 != thing2 or any(thing1 != arg for arg in args):
raise AssertionError("!(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it be better to extend assert_equal() to take an arbitrary number of parameters and have it compare them pairwise? Something like this would work:
def assert_equal(thing1, thing2, *other_things, depth=0):
if thing1 != thing2:
if depth or other_things:
raise AssertionError("%s != %s (positions %d and %d)"%(str(thing1),str(thing2), depth, depth+1))
else:
raise AssertionError("%s != %s"%(str(thing1),str(thing2)))
if other_things:
assert_equal(thing2, *other_things, depth = depth + 1)
>>> assert_equal(4, 4, 5)
AssertionError: 4 != 5 (positions 1 and 2)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I missed your last comment. Your solution is obviously much more elegant.
Is it acceptable to include a change like that in this pull request or should it be separate?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fine to include it here.
qa/rpc-tests/fundrawtransaction.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could use list comprehension:
dec_tx = [self.nodes[3].decoderawtransaction(tx['hex'] for tx in result]
qa/rpc-tests/fundrawtransaction.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could use dictionary comprehension:
outputs = {self.nodes[2].getnewaddress(): value for value in (1.0, 1.1, 1.2, 1.3)}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good idea, thanks.
qa/rpc-tests/fundrawtransaction.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could use list comprehension:
output = [[out[value] for i, out in enumerate(d['vout']) if i != r['changepos']]
for d, r in zip(dec_tx, result)]]
qa/rpc-tests/fundrawtransaction.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could use list comprehension (and similarly below):
output = [d['vout'][1 - r['changepos']]['value'] for d, r in zip(dec_tx, result)]
6a41954 to
b0d3046
Compare
|
I've addressed all the review comments, rebased, squashed, and pushed the resulting commit. I'm wondering whether there's a potential issue with using integers to select which outputs to subtract the fee from, since the outputs are specified by a JSON dictionary, and dictionary keys are inherently unordered. Are we guaranteed when we |
8167931 to
56ea974
Compare
|
@dooglus good point, I don't think so. Recently ran into this writing extended rpc tests for something. |
|
@instagibbs me too: Since the input to |
|
I don't think objects in JSON are assumed to have a meaningful ordering.
|
|
So we are saying that it's OK to use a list of integer indexes into the list of outputs because:
Right? (I tested point 3 as follows: and found that the outputs appear in the raw transaction in the same order as listed in the input to |
|
I think reason (1) is enough to make position based indexing ok, and (2) strengthens it. I don't think (3) is a good reason or something we should ever rely on. The only reason this is brought up is because |
|
@sipa I'll look into making such a change in a separate PR. Using an object for the outputs not only means we cannot guarantee the order of the outputs but also having the addresses as dictionary keys means we can't have multiple outputs with the same address, which I sometimes like to do to (example). I assume it would be best to allow the current object format in addition to the new array format for backwards compatibility. |
|
+1 on address reuse being a sometimes valuable tool |
|
What happens next? I addressed all the comments. Is there something else I need to do? |
ryanofsky
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Change looks good as is, just left a few possible suggestions.
Lightly tested ACK 56ea97409349baaf064c6c2a9da0ba8bbe207f27.
src/wallet/wallet.cpp
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
New argument looks like it could be const reference
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Indeed.
src/wallet/wallet.cpp
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Little better to use size_t here instead of unsigned int.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK.
src/wallet/rpcwallet.cpp
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Little unusual to use %s for the main string instead of strprintf("Invalid parameter, duplicated position: %d", pos)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Absolutely.
src/wallet/rpcwallet.cpp
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe just say an array instead of a json array, since the whole data structure is json.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree, but it appears that everywhere else we refer to 'json array' (see sendmany, addmultisigaddress, lockunspent, listunspent...). Nowhere (in rpcwallet.cpp at least) do we simply say 'an array'.
Will leave as 'json array' for the sake of consistency.
qa/rpc-tests/test_framework/util.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since it is python not c, maybe replace ! with not
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I can't imagine who might have written that! ;)
Will change ! to not.
qa/rpc-tests/fundrawtransaction.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe add comment describing output, could be # Nested list of non-change output amounts for each transaction
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK.
qa/rpc-tests/fundrawtransaction.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe add comment like # List of difference in output amounts between normal and subtractFee transactions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK.
qa/rpc-tests/fundrawtransaction.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This and the 5 following comments are basically just describing the asserts without adding any information. Could maybe remove the comments and condense the asserts.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree.
56ea974 to
453bda6
Compare
|
Addressed @ryanofsky nits, rebased, squashed. |
|
Lightly tested ACK 453bda6 |
Github-Pull: bitcoin#9222 Rebased-From: 453bda6
|
Can this be merged now? |
453bda6 Add 'subtractFeeFromOutputs' option to 'fundrawtransaction'. (Chris Moore)
#11872 was recently posted implementing this change. |
|
Indeed. Though, note that the rpc currently rejects duplicate addresses, and that specific case is not changed in #11872. |
…nsaction'. 453bda6 Add 'subtractFeeFromOutputs' option to 'fundrawtransaction'. (Chris Moore)
…nsaction'. 453bda6 Add 'subtractFeeFromOutputs' option to 'fundrawtransaction'. (Chris Moore)
…dablock committed on Jan 20, 2018 Use version 2 blocks for miner_tests … @codablock codablock committed on Jan 20, 2018 Merge bitcoin#7871: Manual block file pruning. … @laanwj @codablock laanwj authored and codablock committed on Jan 11, 2017 Merge bitcoin#9507: Fix use-after-free in CTxMemPool::removeConflicts() … @sipa @codablock sipa authored and codablock committed on Jan 11, 2017 Merge bitcoin#9297: Various RPC help outputs updated … @MarcoFalke @codablock MarcoFalke authored and codablock committed on Jan 12, 2017 Merge bitcoin#9416: travis: make distdir before make … @MarcoFalke @codablock MarcoFalke authored and codablock committed on Jan 12, 2017 Merge bitcoin#9520: Deprecate non-txindex getrawtransaction and bette… … @MarcoFalke @codablock MarcoFalke authored and codablock committed on Jan 12, 2017 Merge bitcoin#9518: Return height of last block pruned by pruneblockc… … @MarcoFalke @codablock MarcoFalke authored and codablock committed on Jan 12, 2017 Merge bitcoin#9472: Disentangle progress estimation from checkpoints … … @laanwj @codablock laanwj authored and codablock committed on Jan 12, 2017 Merge bitcoin#8883: Add all standard TXO types to bitcoin-tx … @laanwj @codablock laanwj authored and codablock committed on Jan 12, 2017 Merge bitcoin#9261: Add unstored orphans with rejected parents to rec… … @laanwj @codablock laanwj authored and codablock committed on Jan 12, 2017 Merge bitcoin#9468: [Depends] Dependency updates for 0.14.0 … @laanwj @codablock laanwj authored and codablock committed on Jan 12, 2017 Merge bitcoin#9222: Add 'subtractFeeFromAmount' option to 'fundrawtra… … @laanwj @codablock laanwj authored and codablock committed on Jan 12, 2017 Merge bitcoin#9490: Replace FindLatestBefore used by importmuti with … … @sipa @codablock sipa authored and codablock committed on Jan 13, 2017 Merge bitcoin#9469: [depends] Qt 5.7.1 … @laanwj @codablock laanwj authored and codablock committed on Jan 15, 2017 Merge bitcoin#9380: Separate different uses of minimum fees … @laanwj @codablock laanwj authored and codablock committed on Jan 16, 2017 Remove SegWit related code in dash-tx @codablock codablock committed on Sep 21, 2017 Merge bitcoin#9561: Wake message handling thread when we receive a ne… … @sipa @codablock sipa authored and codablock committed on Jan 17, 2017 Merge bitcoin#9508: Remove unused Python imports … @MarcoFalke @codablock MarcoFalke authored and codablock committed on Jan 18, 2017 Merge bitcoin#9512: Fix various things -fsanitize complains about
I notice it is now possible to have the fee subtracted from the output amounts by specifying a subtractFeeFromAmount parameter in both
sendtoaddressandsendmany, but not infundrawtransaction.This commit adds the option to
fundrawtransaction.