-
Notifications
You must be signed in to change notification settings - Fork 38.8k
rpc: Optionally print feerates in sat/vb #33741
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code Coverage & BenchmarksFor details see: https://corecheck.dev/bitcoin/bitcoin/pulls/33741. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
b581138 to
1389b97
Compare
w0xlt
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concept ACK
maflcko
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess it is a bit verbose to have each RPC annotated with a type optional arg, but there probably isn't an easier solution.
There could be a global option to toggle the default, to improve the UX.
Though, please don't use floating point types. For monetary values, a fixed point type should be used.
src/rpc/fees.cpp
Outdated
| @@ -84,7 +85,11 @@ static RPCHelpMan estimatesmartfee() | |||
| CFeeRate min_mempool_feerate{mempool.GetMinFee()}; | |||
| CFeeRate min_relay_feerate{mempool.m_opts.min_relay_feerate}; | |||
| feeRate = std::max({feeRate, min_mempool_feerate, min_relay_feerate}); | |||
| result.pushKV("feerate", ValueFromAmount(feeRate.GetFeePerK())); | |||
| if (!request.params[2].isNull() && request.params[2].get_bool()) { | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please don't encode default values in how the code is written. This makes it impossible to change the default values without rewriting the code logic.
Also, could use self.(Maybe)Arg<bool> instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think with the rebase it's easier to change defaults without touching code logic.
What would be the benefits of using self.MaybeArg<bool>? I'm not really familiar with it tbh.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What would be the benefits of using
self.MaybeArg<bool>? I'm not really familiar with it tbh.
The function is explained in the docstring above the function. It will read the default value by itself, so it doesn't need to be hardcoded several times.
I think with the rebase it's easier to change defaults without touching code logic.
I don't think so. I can still see FeeEstimateMode feerate_units = (!request.params[2].isNull() && request.params[2].get_bool()) ? FeeEstimateMode::SAT_VB : FeeEstimateMode::BTC_KVB;
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the suggestion, yep it is cleaner. Now should be as easy as changing the default in the arg definition.
btw: I used self.Arg<bool>, I think it makes more sense, self.MaybeArg<bool> is defined for optional params with no defaults.
src/rpc/fees.cpp
Outdated
| @@ -84,7 +85,11 @@ static RPCHelpMan estimatesmartfee() | |||
| CFeeRate min_mempool_feerate{mempool.GetMinFee()}; | |||
| CFeeRate min_relay_feerate{mempool.m_opts.min_relay_feerate}; | |||
| feeRate = std::max({feeRate, min_mempool_feerate, min_relay_feerate}); | |||
| result.pushKV("feerate", ValueFromAmount(feeRate.GetFeePerK())); | |||
| if (!request.params[2].isNull() && request.params[2].get_bool()) { | |||
| result.pushKV("feerate", static_cast<double>(feeRate.GetFeePerK()) / 1000.0); | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is wrong, you can't use double to denote monetary amounts. double is a floating point type, which can not represent decimal values accurately.
For example, if GetFeePerK() returns CAmount{665714}, the Univalue will be 665.7140000000001, which is confusing at best, but also wrong.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are right, although I think it would not be a problem because we don't print that many decimals.
Anyway, I think the approach in afeeac1596da47b0e431d468a665d344b39e1a87 is cleaner and follows what we already do for btc/kvB. What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are right, although I think it would not be a problem because we don't print that many decimals.
They are printed. You can try it locally by printing the constructed univalue or by checking it in a test.
Anyway, I think the approach in afeeac1 is cleaner and follows what we already do for btc/kvB. What do you think?
I don't think it handles negative feerates correctly. Generally, it is best to write unit or fuzz tests for newly written code. Also, I am not sure if CFeeRate should be bundled with univalue. The conversion could be a standalone helper, but this is just a nit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Generally, it is best to write unit or fuzz tests for newly written code
Done
Also, I am not sure if
CFeeRateshould be bundled with univalue. The conversion could be a standalone helper, but this is just a nit.
I decided to make it a function of CFeeRate because is the only class that should use it. But I'm ok moving it to core_writte.
1389b97 to
ed354df
Compare
|
@maflcko thanks for reviewing :)
I don't think there is either, at least not in a backwards compatible way.
That would be great for a |
f4f932d to
c34f6a3
Compare
src/policy/feerate.h
Outdated
| @@ -8,6 +8,7 @@ | |||
|
|
|||
| #include <consensus/amount.h> | |||
| #include <serialize.h> | |||
| #include <univalue/include/univalue.h> | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This drags an RPC‑only dependency (UniValue) into a widely included policy header.
Instead, a helper in rpc/util.{h,cpp} (or rpc/fees.h) could be added such as:
enum class FeeRateUnit { BTC_KVB, SAT_VB };
UniValue ValueFromFeeRate(const CFeeRate& fr, FeeRateUnit unit);There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes maybe it's a better approach.
Moved the function into core_write.cpp and core_io.h similar to function ValueFromAmount.
src/test/amount_tests.cpp
Outdated
| BOOST_CHECK_EQUAL(CFeeRate(1, 1).ToUniValue(FeeEstimateMode::SAT_VB).write(), "1.000"); | ||
| BOOST_CHECK_EQUAL(CFeeRate(10, 1).ToUniValue(FeeEstimateMode::SAT_VB).write(), "10.000"); | ||
| BOOST_CHECK_EQUAL(CFeeRate(1000, 1).ToUniValue(FeeEstimateMode::SAT_VB).write(), "1000.000"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do the last six lines duplicate three assertions ?
You probably intended to add negative sat/vB cases.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yup, good catch tanks! :)
src/rpc/net.cpp
Outdated
| // Those fields can be deprecated, to be replaced by the getmempoolinfo fields | ||
| obj.pushKV("relayfee", ValueFromAmount(node.mempool->m_opts.min_relay_feerate.GetFeePerK())); | ||
| obj.pushKV("incrementalfee", ValueFromAmount(node.mempool->m_opts.incremental_relay_feerate.GetFeePerK())); | ||
| UniValue mempool_info = MempoolInfoToJSON(*node.mempool, self.Arg<bool>("satvB")); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This line constructs a full object (with locking) only to reuse two values.
It also adds a new include <rpc/mempool.h> into rpc/net.cpp, creating an RPC‑RPC dependency.
It may be better to compute these two fields directly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
True!
I thought that there was the intention to add all values and not only relayfee and incrementalfee here as the comment // Those fields can be deprecated, to be replaced by the getmempoolinfo fields seems suggest.
But seeing https://github.com/bitcoin/bitcoin/pull/25648/files#r935563077 I think I just misunderstood and the intention was to maybe remove them in the future and just use getmempool rpc to know that info.
Similar to ValueFromAmount this function returns a UniValue object with the feerate given a CFeeRate object.
c34f6a3 to
6caa122
Compare
polespinasa
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rebased following @w0xlt suggestions :)
src/rpc/net.cpp
Outdated
| // Those fields can be deprecated, to be replaced by the getmempoolinfo fields | ||
| obj.pushKV("relayfee", ValueFromAmount(node.mempool->m_opts.min_relay_feerate.GetFeePerK())); | ||
| obj.pushKV("incrementalfee", ValueFromAmount(node.mempool->m_opts.incremental_relay_feerate.GetFeePerK())); | ||
| UniValue mempool_info = MempoolInfoToJSON(*node.mempool, self.Arg<bool>("satvB")); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
True!
I thought that there was the intention to add all values and not only relayfee and incrementalfee here as the comment // Those fields can be deprecated, to be replaced by the getmempoolinfo fields seems suggest.
But seeing https://github.com/bitcoin/bitcoin/pull/25648/files#r935563077 I think I just misunderstood and the intention was to maybe remove them in the future and just use getmempool rpc to know that info.
src/test/amount_tests.cpp
Outdated
| BOOST_CHECK_EQUAL(CFeeRate(1, 1).ToUniValue(FeeEstimateMode::SAT_VB).write(), "1.000"); | ||
| BOOST_CHECK_EQUAL(CFeeRate(10, 1).ToUniValue(FeeEstimateMode::SAT_VB).write(), "10.000"); | ||
| BOOST_CHECK_EQUAL(CFeeRate(1000, 1).ToUniValue(FeeEstimateMode::SAT_VB).write(), "1000.000"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yup, good catch tanks! :)
src/policy/feerate.h
Outdated
| @@ -8,6 +8,7 @@ | |||
|
|
|||
| #include <consensus/amount.h> | |||
| #include <serialize.h> | |||
| #include <univalue/include/univalue.h> | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes maybe it's a better approach.
Moved the function into core_write.cpp and core_io.h similar to function ValueFromAmount.
6caa122 to
004d22b
Compare
This commit also includes mempool.h, which was missing and was causing function definition issues.
Can you explain why it's not good practice? I agree it's not as human-readable but I don't software minds either way. Not to be snarky, but I think the most widely adopted usage of this RPC interface must be to handle what it returns, including converting it before printing to users.
It seems extremely dangerous to ever change the default... no matter how much notice is given, it's a massive api break that would cause software to suddenly be several orders of magnitude off. A different approach could be to add another field to each result, like "feerate_satvb". A global config option like `-feeraterepresentation" could determine which results are returned so that RPC results aren't bloated. I'm not familiar with how we might pass a global config option to the RPCs though. |
|
@glozow thanks for your comments :)
We open the door to other software/users to make easy mistakes by forcing them to do a unit conversion that Core shouldn't have done in the first place. I understand that this has been like this since ¿always?, but giving users the option to have sat/vB (or even more units thanks to FeeFrac) can be useful and reduce the burden on other software.
I agree, this PR doesn't change any default behavior. It MUST be done in a backwards compatible way.
Yes! This is another approach already taken by
I don't dislike at all this approach and I would probably be open to switching to it if there's consensus that it is better. |
I still don't see the argument for why a unit per kvB is so problematic. BTC and satoshis are easily convertible. I do think it's worthwhile to unify the format for inputting and and outputting feerates. And I agree it should be configurable, given there are multiple preferred formats. My concern is the idea that the default format might be changed in the future, causing somebody's reading of a feerate to be off by a factor of 100,000. I think the acceptable way to do this is to create new fields and maybe phase out the old ones.
That's why the results must have different names. If they forget the config option, they should get a key error. |
|
Adding a new field in the result called |
Yes the idea is to be able to use sat/vB or BTC/kvB in inputting and outputting. This PR is just a PoC on how we could do it for outputting.
I don't think the default will ever be changed. I left the door open to it in some comments, but I would not push for it tbh.
Oh sorry I didn't understand your idea. So you propose to add a startup option to choose the feerate units and then on the outputs change the key so it will return a key error if someone is expecting a different unit? I think that's a good approach, how would you do it for rpc arguments though? Unless you force to use named arguments I think it can still lead to errors. |
|
🐙 This pull request conflicts with the target branch and needs rebase. |
Part of #32093
Returning feerates in BTC/kvB it can be very burdensome and is not good practice, as the most widely adopted units are sat/vB. This PR aims to show a PoC of how we could migrate to sat/vb in a backwards compatible manner.
The RPC affectd by this PR are
getmempoolinfo,getnetworkinfo,getwalletinfo,estimatesmartfeeandestimaterawfee.Because of sub 1sat/vB environment we cannot rely on GetFee() because it internally uses FeeFrac EvaluateFee() function which rounds the values to always return an int. If the
feerate < 1then it will be rounded to 1 or 0.For more context or discuss how to migrate also arguments please refer to the open issue #32093.