-
Notifications
You must be signed in to change notification settings - Fork 38.7k
ArgsManager: support subcommand-specific options #28802
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code Coverage & BenchmarksFor details see: https://corecheck.dev/bitcoin/bitcoin/pulls/28802. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please copy-paste ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
|
Implements this comment from when these commands were introduced originally. Main motivation is that I'd like to add a new subcommand to |
d335f7f to
abba7c5
Compare
|
tool_wallet.py fails CI |
abba7c5 to
162c5ab
Compare
162c5ab to
86fab48
Compare
86fab48 to
6be2b82
Compare
stickies-v
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approach ACK
d5e680e to
1cd91d4
Compare
1cd91d4 to
15875ea
Compare
15875ea to
905ed6b
Compare
|
Leaving as draft pending #31250 |
80c0927 to
c064ad2
Compare
c064ad2 to
0d1d11f
Compare
|
Rebased and undrafted |
l0rinc
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concept ACK, it makes sense to associate specific command-line options with specific commands - like function names with args. I can imagine this being useful in other cases as well. Given that the PR needs a rebase, I have commented on every nit I could find - and I would need full test coverage for this to be comfortable acking, but like the commit split otherwise. If you can, I would appreciate adding slightly more context to the PR description and the commit messages.
| @@ -66,6 +66,8 @@ enum class OptionsCategory { | |||
| CLI_COMMANDS, | |||
| IPC, | |||
|
|
|||
| COMMAND_OPTIONS, // Specific to one or more commands | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems safe, just have to make sure: is it safe to add in the middle of this enum (e.g. is its index persisted anywhere?).
If the placement doesn't matter, can we add the options closer to the command values (before IPC)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Placement doesn't matter, but I don't think moving closer to COMMANDS, REGISTER_COMMANDS or CLI_COMMANDS really makes sense -- COMMAND_OPTIONS relates to argsman.AddCommand() commands (used by bitcoin-wallet and bitcoin-util), which differ from COMMANDS (used by bitcoin-tx), REGISTER_COMMANDS (also used by bitcoin-tx), and CLI_COMMANDS (used by bitcoin-cli for its non-rpc commands, -generate, -netinfo, -addrinfo, -getinfo).
| @@ -347,9 +350,9 @@ class ArgsManager | |||
| void AddArg(const std::string& name, const std::string& help, unsigned int flags, const OptionsCategory& cat); | |||
|
|
|||
| /** | |||
| * Add subcommand | |||
| * Add command | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this comment still useful now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a docstring, so shows up in the doxygen docs https://doxygen.bitcoincore.org/class_args_manager.html#ab71d531303f7e6fe3bbf67683a414a2e ; maybe it's not that useful, but doesn't seem worth removing outright
| @@ -137,6 +139,7 @@ class ArgsManager | |||
| std::string m_network GUARDED_BY(cs_args); | |||
| std::set<std::string> m_network_only_args GUARDED_BY(cs_args); | |||
| std::map<OptionsCategory, std::map<std::string, Arg>> m_available_args GUARDED_BY(cs_args); | |||
| std::map<std::string, std::set<std::string>> m_command_args GUARDED_BY(cs_args); | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This line seems to conflict with latest master.
nit: is it important for the set to be sorted here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's looked up by the key in CheckCommandOptions, so sorting seems fine to me?
| { | ||
| if (select.empty()) return; | ||
| if (m_iter == m_end) return; | ||
| for (const auto& [cmdopt_name, cmdopt_info] : m_iter->second) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: when is it opt and when cmdopt?
| argsman.AddArg("-debug=<category>", "Output debugging information (default: 0).", ArgsManager::ALLOW_ANY, OptionsCategory::DEBUG_TEST); | ||
| argsman.AddArg("-printtoconsole", "Send trace/debug info to console (default: 1 when no -debug is true, 0 otherwise).", ArgsManager::ALLOW_ANY, OptionsCategory::DEBUG_TEST); | ||
|
|
||
| argsman.AddCommand("info", "Get wallet info"); | ||
| argsman.AddCommand("create", "Create a new descriptor wallet file"); | ||
| argsman.AddCommand("dump", "Print out all of the wallet key-value records"); | ||
| argsman.AddCommand("createfromdump", "Create new wallet file from dumped records"); | ||
| argsman.AddCommand("dump", "Print out all of the wallet key-value records", {"-dumpfile"}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
are there any other such relations between command and arg that we could check this way?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not currently, I think
|
Are you still working on this? This has needed rebase for a few months now. |
0d1d11f to
cce7eba
Compare
Sorry, these seem completely self-explanatory to me, so I don't see what to add. Happy to answer questions or take suggestions though. |
|
Concept ACK cce7eba. This PR improves documentation and error handling for subcommand specific options. I do think the PR could use a more detailed description. (I wasn't very familiar with existing subcommand support so I didn't know what was changing.) Right now, bitcoin-wallet -wallet=foo -dumpfile=foo.txt dumpBut
This PR improves on that, changing help and error output as follows: @@ -14,10 +14,6 @@
-datadir=<dir>
Specify data directory
- -dumpfile=<file name>
- When used with 'dump', writes out the records to this file. When used
- with 'createfromdump', loads the records into a new wallet.
-
-help
Print this help message and exit (also -h or -?)
@@ -73,9 +69,19 @@
createfromdump
Create new wallet file from dumped records
+ -dumpfile=<file name>
+ When used with 'dump', writes out the records to this file. When
+ used with 'createfromdump', loads the records into a
+ new wallet.
+
dump
Print out all of the wallet key-value records
+ -dumpfile=<file name>
+ When used with 'dump', writes out the records to this file. When
+ used with 'createfromdump', loads the records into a
+ new wallet.
+
info
Get wallet info
$ bitcoin-wallet -regtest -wallet=foo -dumpfile=out info
-The -dumpfile option can only be used with the "dump" and "createfromdump" commands.
+Error: Invalid arguments provided:
+- The -dumpfile option cannot be used with the 'info' command.Limitations
Nevertheless, this change should be a strict improvement. |
Should I cut and paste what you wrote below into the PR or commit description? Sorry, I need an ELI5 here :) |
Feel free to use & change any of it. I mostly wrote that for myself to be able to remember what changes this PR was making. I do think the current description is perfectly ok and accurate, but I skipped over this PR many times not really knowing what it is was doing or if I should take the time to find out, so adding some more details could help this get more attention. |
ryanofsky
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code review ACK cce7eba. Left various suggestions, but none are important so feel free to ignore.
| namespace { | ||
| /** Helper class for iterating over COMMAND_OPTIONS applicable to a given command */ | ||
| template <typename T> | ||
| class CommandOptionsGetter |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In commit "ArgsManager: support command-specific options" (e2debfa)
Any plans to reuse this class? It would seem simpler to inline:
diff
--- a/src/common/args.cpp
+++ b/src/common/args.cpp
@@ -646,35 +646,6 @@ void ArgsManager::CheckMultipleCLIArgs() const
}
}
-namespace {
-/** Helper class for iterating over COMMAND_OPTIONS applicable to a given command */
-template <typename T>
-class CommandOptionsGetter
-{
-private:
- const typename T::const_iterator m_end;
- const typename T::const_iterator m_iter;
-public:
- CommandOptionsGetter(const T& available_args)
- : m_end{available_args.end()},
- m_iter{available_args.find(OptionsCategory::COMMAND_OPTIONS)}
- {
- }
-
- template <typename Fn>
- void Iterate(const std::set<std::string>& select, bool with_debug, Fn&& fn) const
- {
- if (select.empty()) return;
- if (m_iter == m_end) return;
- for (const auto& [cmdopt_name, cmdopt_info] : m_iter->second) {
- if (!with_debug && (cmdopt_info.m_flags & ArgsManager::DEBUG_ONLY)) continue;
- if (!select.contains(cmdopt_name)) continue;
- fn(cmdopt_name, cmdopt_info);
- }
- }
-};
-} // anonymous namespace
-
std::string ArgsManager::GetHelpMessage() const
{
const bool show_debug = GetBoolArg("-help-debug", false);
@@ -682,7 +653,7 @@ std::string ArgsManager::GetHelpMessage() const
std::string usage;
LOCK(cs_args);
- const auto command_options = CommandOptionsGetter(m_available_args);
+ const auto command_options = m_available_args.find(OptionsCategory::COMMAND_OPTIONS);
for (const auto& [category, category_args] : m_available_args) {
switch(category) {
@@ -748,10 +719,12 @@ std::string ArgsManager::GetHelpMessage() const
if (category == OptionsCategory::COMMANDS) {
const auto cmd_args = m_command_args.find(arg_name);
- if (cmd_args != m_command_args.end()) {
- command_options.Iterate(cmd_args->second, show_debug, [&](const auto& cmdopt_name, const auto& cmdopt_info) {
+ if (cmd_args != m_command_args.end() && command_options != m_available_args.end()) {
+ for (const auto& [cmdopt_name, cmdopt_info] : command_options->second) {
+ if (!show_debug && (cmdopt_info.m_flags & ArgsManager::DEBUG_ONLY)) continue;
+ if (!cmd_args->second.contains(cmdopt_name)) continue;
usage += HelpMessageSubOpt(cmdopt_name, cmdopt_info.m_help_param, cmdopt_info.m_help_text);
- });
+ }
}
}
}| * @param option Option name (e.g. "-rpcuser") | ||
| * @param help_param Help parameter (e.g. "=<user>" or "") | ||
| * @param message Option description (e.g. "Username for JSON-RPC connections") | ||
| * @param indent Additional indentation |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In commit "ArgsManager: support command-specific options" (e2debfa)
Giving this function an indent parameter doesn't seem right semantically since the other parameters are higher-level parameters unrelated to formatting, and the function is still hardcoding most other formatting value internally. This also increases code complexity by requiring a separate HelpMessageSubOpt function, because the caller doesn't have access to the formatting constants. Would suggest keeping this high-level and simplifying code with a change like:
diff
--- a/src/common/args.cpp
+++ b/src/common/args.cpp
@@ -750,7 +750,7 @@ std::string ArgsManager::GetHelpMessage() const
const auto cmd_args = m_command_args.find(arg_name);
if (cmd_args != m_command_args.end()) {
command_options.Iterate(cmd_args->second, show_debug, [&](const auto& cmdopt_name, const auto& cmdopt_info) {
- usage += HelpMessageSubOpt(cmdopt_name, cmdopt_info.m_help_param, cmdopt_info.m_help_text);
+ usage += HelpMessageOpt(cmdopt_name, cmdopt_info.m_help_param, cmdopt_info.m_help_text, /*subopt=*/true);
});
}
}
@@ -779,8 +779,9 @@ std::string HelpMessageGroup(const std::string &message) {
return std::string(message) + std::string("\n\n");
}
-std::string HelpMessageOpt(std::string_view option, std::string_view help_param, std::string_view message, int indent)
+std::string HelpMessageOpt(std::string_view option, std::string_view help_param, std::string_view message, bool subopt)
{
+ const int indent{subopt ? msgIndent - optIndent : 0};
return strprintf("%*s%s%s\n%*s%s\n\n",
(optIndent + indent), "", option, help_param,
(msgIndent + indent), "", FormatParagraph(message, screenWidth - msgIndent - indent, msgIndent + indent));
@@ -800,11 +801,6 @@ bool HasTestOption(const ArgsManager& args, const std::string& test_option)
});
}
-std::string HelpMessageSubOpt(std::string_view option, std::string_view help_param, std::string_view message)
-{
- return HelpMessageOpt(option, help_param, message, msgIndent - optIndent);
-}
-
fs::path GetDefaultDataDir()
{
// Windows:
--- a/src/common/args.h
+++ b/src/common/args.h
@@ -486,19 +486,9 @@ std::string HelpMessageGroup(const std::string& message);
* @param option Option name (e.g. "-rpcuser")
* @param help_param Help parameter (e.g. "=<user>" or "")
* @param message Option description (e.g. "Username for JSON-RPC connections")
- * @param indent Additional indentation
+ * @param subopt True if this is suboption, instead of a top level option.
* @return the formatted string
*/
-std::string HelpMessageOpt(std::string_view option, std::string_view help_param, std::string_view message, int indent=0);
-
-/**
- * Same as HelpMessageOpt, but indents for command-specific options
- *
- * @param option Option name (e.g. "-rpcuser")
- * @param help_param Help parameter (e.g. "=<user>" or "")
- * @param message Option description (e.g. "Username for JSON-RPC connections")
- * @return the formatted string
- */
-std::string HelpMessageSubOpt(std::string_view option, std::string_view help_param, std::string_view message);
+std::string HelpMessageOpt(std::string_view option, std::string_view help_param, std::string_view message, bool subopt=false);
#endif // BITCOIN_COMMON_ARGS_H| CLI_COMMANDS, | ||
| IPC, | ||
|
|
||
| COMMAND_OPTIONS, // Specific to one or more commands |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In commit "ArgsManager: support command-specific options" (e2debfa)
Might be worth noting that options with this category will be excluded from help output unless they are referenced by an AddCommand call.
Relatedly it might be good to add a check that every option in the COMMAND_OPTIONS category is actually referenced by an AddCommand call.
| std::map<std::string, Arg>& arg_map = m_available_args[OptionsCategory::COMMANDS]; | ||
| auto ret = arg_map.emplace(cmd, Arg{"", help, ArgsManager::COMMAND}); | ||
| if (!options.empty()) { | ||
| m_command_args.try_emplace(cmd, std::move(options)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In commit "ArgsManager: support command-specific options" (e2debfa)
Should the options list be validated against known options? Seems like it could be easy to make a typo and pass a bad list of options to this function, causing help output to be incomplete without it being obvious.
| const auto command = test_args.GetCommand(); | ||
| if (!command) return NO_COMMAND; | ||
| std::vector<std::string> details; | ||
| if (!test_args.CheckCommandOptions(command->command, &details)) return COMMAND_OPTS; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In commit "tests: Add some test coverage for ArgsManager::AddCommand" (cce7eba)
Might be good to check details is empty on success, nonempty on failure
| argsman.AddArg("-datadir=<dir>", "Specify data directory", ArgsManager::ALLOW_ANY | ArgsManager::DISALLOW_NEGATION, OptionsCategory::OPTIONS); | ||
| argsman.AddArg("-wallet=<wallet-name>", "Specify wallet name", ArgsManager::ALLOW_ANY | ArgsManager::NETWORK_ONLY, OptionsCategory::OPTIONS); | ||
| argsman.AddArg("-dumpfile=<file name>", "When used with 'dump', writes out the records to this file. When used with 'createfromdump', loads the records into a new wallet.", ArgsManager::ALLOW_ANY | ArgsManager::DISALLOW_NEGATION, OptionsCategory::OPTIONS); | ||
| argsman.AddArg("-dumpfile=<file name>", "When used with 'dump', writes out the records to this file. When used with 'createfromdump', loads the records into a new wallet.", ArgsManager::ALLOW_ANY | ArgsManager::DISALLOW_NEGATION, OptionsCategory::COMMAND_OPTIONS); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps a future improvement/ follow-up could be specifying the help message description for each command:
| argsman.AddArg("-dumpfile=<file name>", "When used with 'dump', writes out the records to this file. When used with 'createfromdump', loads the records into a new wallet.", ArgsManager::ALLOW_ANY | ArgsManager::DISALLOW_NEGATION, OptionsCategory::COMMAND_OPTIONS); | |
| argsman.AddArg("-dumpfile=<file name>", "{'dump':'Writes out the records to this file.','createfromdump':'Loads the records into a new wallet.'}", ArgsManager::ALLOW_ANY | ArgsManager::DISALLOW_NEGATION, OptionsCategory::COMMAND_OPTIONS);``` |
pablomartin4btc
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Concept ACK
I like the direction of this PR and its improvements. One thing that perhaps could still confuse reviewers I think is the terminology:
- the PR title/description still refer to subcommands, but the API and implementation consistently use command (
AddCommand()- updated docstring* Add subcommandto* Add command-, COMMAND_OPTIONS, updated comments per achow’s suggestion), - except for the multiprocess bitcoin rpc case, these are really top-level CLI commands (
bitcoin-wallet dump,bitcoin-util grind, etc.). Aligning the wording (and helpers likeHelpMessageSubOpt) with “command-specific options” would reduce ambiguity (HelpMessageCommandSpecificOpt).
Also, the help output now duplicates the same option description under each command (e.g. -dumpfile under both dump and createfromdump), even though semantics differ slightly per command. This works, but it might be worth clarifying in comments that this duplication is intentional, or noting it as a potential future improvement (as suggested).
Adding a short before/after example and expanding the description beyond bitcoin-wallet (e.g. bitcoin-util grind) would likely help attract more reviewers.
Should we include the new category into the system fuzz (already tried and ran it myself):
bitcoin/src/test/fuzz/system.cpp
Line 62 in 7f295e1
| const OptionsCategory options_category = fuzzed_data_provider.PickValueInArray<OptionsCategory>({OptionsCategory::OPTIONS, OptionsCategory::CONNECTION, OptionsCategory::WALLET, OptionsCategory::WALLET_DEBUG_TEST, OptionsCategory::ZMQ, OptionsCategory::DEBUG_TEST, OptionsCategory::CHAINPARAMS, OptionsCategory::NODE_RELAY, OptionsCategory::BLOCK_CREATION, OptionsCategory::RPC, OptionsCategory::GUI, OptionsCategory::COMMANDS, OptionsCategory::REGISTER_COMMANDS, OptionsCategory::CLI_COMMANDS, OptionsCategory::IPC, OptionsCategory::HIDDEN}); |
|
Concept ACK, was working in this area a few days ago, and thought having something like this would be very useful. |
Adds the ability to link particular options to one or more subcommands, and uses this feature in
bitcoin-wallet. Separates out the help information for subcommand-specific options (duplicating it if an option applies to multiple subcommands), and provides a function for checking if some options have been specified that only apply to different subcommands.