-
Notifications
You must be signed in to change notification settings - Fork 725
[RPC] [Wallet] AutoCombineRewards fixes and Improvements #923
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[RPC] [Wallet] AutoCombineRewards fixes and Improvements #923
Conversation
|
@Mrs-X . I had problems with the backwards compatibility idea you proposed. The catch triggered on the read from the datastream, and trying to rewind it, or convert it from one format to the other started to get more complicated. So instead designed a way to be backwards compatible; by loading the autocombinesettings from the old format, and filling in the default frequency (15 blocks). If the user makes changes; when it saves the new format, it will be saved as autocombinesettingsV2, and it will replace the threshold in the old format as "-1", as a -1 threshold doesn't make any sense. That way the code will throw away whatever is in autocombinesettings, if threshold is -1, and load the updated format from autocombinesettingsV2. I hope that's acceptable. The only drawback is that if you were testing my previous PR; your autocombinesettings will be in the new format, rather than the old; and it may error. However re-issuing the command should rectify the situation. |
Fuzzbawls
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Needs some cleanup on the RPC side of things regarding the JSON results. We want to avoid the need for string parsing/conversion where possible. Fields should also never return a variable type as it makes machine reading of the results needlessly complicated.
|
Sorry; merging in my updates ended up being a train wreck. I'll clean it up tomorrow and probably have to submit another PR from a clean branch) |
|
you can rebase on top of git pull --rebase origin master # origin being the git remote pointing to the primary public repocleaning up commits can be done with git rebase -i HEAD~4 # 4 being the number of commits in the pull request, adjust as neededWould really like to avoid situations where a PR is continuously closed and resubmitted as it makes tracking the comments/reviews more work. See http://git-scm.com/docs/git-rebase#_interactive_mode for more information. |
|
While rewriting the description of all the changes, it occurred to me that a threshold of 0 will cause a dormant wallet to constantly recombine a UTXO and it's change UTXO together. That has been fixed. I will set out to rebase when I have access to git directly this evening. |
e1c23e2 to
c7d343e
Compare
|
Not how I screwed up and got those other two commits into here. I will revisit again tomorrow to try and figure it out (and make sure the many manual merges didn't screw up something) |
608cb52 to
763251c
Compare
763251c to
ede1af4
Compare
Try 4; This is originally #849 then discussed in #867 and further in #873 and this should be the (hopefully) final rendition.
Problem:
A problem was introduced with #518 when a UTXO smaller than 10% of the threshold is added to the collection of UTXOs to combine, and that UTXO is enough to increase the total amount being combined above the threshold, or when the total amount of dust in the wallet exceeds the threshold by less than 10%.
What occurs is two fold. First, the nTotalRewardsValue > nAutoCombineThreshold will break it out of the for loop; but the "safety margin" will split it up into two UTXOs; one within 10% of the threshold, and then one for the change. When the wallet comes back through on it's dust collection, it can pick up those two UTXOs and repeat until the fees whittle the two combined transactions to a total below the threshold.
If there is another UTXO to add, which takes the logic far enough above the threshold to remain above the threshold after the 10% reduction; the check in the loop is functional. However there is still one other problem case: If the total amount being collected falls into the "within 10% of the threshold" situation, and the for loop can't make another pass... we exit the for loop normally, and find our way into the zero fee check. However the zero fee check sees we are over the threshold, but not that the transaction amount will be over the threshold. So we need to account for the 10% there as well, by using the actual amount (vsecSend[0].second) rather than nTotalRewardsValue to determine if we should continue only if free.
Even with this fix, AutoCombineRewards was still hardly usable, as wallet scans on wallets with many active addresses was consuming significant resources when running AutoCombineDust() on every block. In order to make this feature more useful and usable, further features were folded in; The ability to configure how often to run the scan, and to configure it to combine as much as possible when it executes.
Improvement 1: Max Threshold:
Users trying to clean up wallets that have collected a lot of small transactions may find it more convenient to combine as much as possible with one sweep. Rather than guessing what amount would be best, and having to do multiple passes; setting a threshold of "0" will autocombine up to the max it is capable of. This is also even more useful for users that have several transactions across each of many addresses,, as they would need to get each wallet down to a very few transactions before sweeping the funds into a single large transaction across many input addresses. This allows for autocombine to quickly and easily combine to very few UTXOs in each wallet, and allowing a sweep across many wallet addresses to be much more convenient. This would function very much like a "defragment" routine.
Improvement 2: Frequency:
An old stale "fix" was floating around in internet, implementing an AutoCombineRewardsThresholdTime concept; I was unable to locate the originator. This implementation had allowed you to configure the number of minutes between dust collection. That old algorithm was commented out and replaced with a 5 second wait. This concept also was flawed, in that it was going to be doing a sleep within the ProcessNewBlock() code. The first attempt was abandoned because it likely was blocking ProcessNewBlock for 15 minutes at a time, and no way to not block, if using the feature, for under a minute. The workaround (5 seconds) still wasn't desirable, as it would still lock up block ProcessNewBlock's execution for 6 seconds.
This new method changes that design from ThresholdTime, to Block Frequency, and defaults to 15 blocks. Time can be adjusted per implementation, to get a desired minute time based on the block frequency of the coin parameters. If AutoCombineDust is enabled, it will only check and attempt to combine dust if the block height is a multiple of the Block Frequency. e.g. if set to 10, then every 10th block it will be executed. 100; then every 100th block.
This can now be tailored by the user, based on their desired dust cleanup threshold, and their expectation of frequency that they will need to clean.
Improvement 3: One Shot:
The concept of a "one shot" dust cleanup was added when the block frequency is set to "0". This will run the automatic rewards combining on the next block, and then it will disable itself until the next time the wallet is started up. This feature assumes that someone would be running their wallet infrequently, and therefore is likely to want the one shot to run when they start the wallet again. As such, the enabled=true and frequency=0 is saved in the database, but after it runs, it will be shut off for the duration of the wallet run.
To run it for a one time only execution, the user would have to
autocombinerewards falseafter the one shot is executed.The feature, in it's entirety:
Examples:
Basic configuration for frequency use
Will run every time the block count is a multiple of 250, and combine if it finds multiple UTXOs. It will collect inputs until the total exceeds 5.5 PIVX. If the total is below the threshold + 10%, it will only submit the transaction if the fee is considered free.
Configuring for frequency use, maximum combine
Will run every time the block count is a multiple of 250, and combine if it finds multiple UTXOs. It will collect as many inputs as possible, up to the maximum bytes allowed per transaction (100kb).
Configuring One Shot
Before the run
After the run
Please Note: The underlying functionality has been tested on two altcoins, and the RPC logic has been tested standalone on PIVX testnet.