Skip to content

Conversation

@ajtowns
Copy link
Contributor

@ajtowns ajtowns commented Nov 6, 2023

Adds the ability to link particular options to one or more subcommands, and uses this feature in bitcoin-wallet. Separates out the help information for subcommand-specific options (duplicating it if an option applies to multiple subcommands), and provides a function for checking if some options have been specified that only apply to different subcommands.

@DrahtBot
Copy link
Contributor

DrahtBot commented Nov 6, 2023

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/28802.

Reviews

See the guideline for information on the review process.

Type Reviewers
ACK ryanofsky
Concept ACK l0rinc, pablomartin4btc, sedited
Approach ACK stickies-v

If your review is incorrectly listed, please copy-paste <!--meta-tag:bot-skip--> into the comment that the bot should ignore.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #29838 (Feature: Use different datadirs for different signets by BrandonOdiwuor)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

@ajtowns
Copy link
Contributor Author

ajtowns commented Nov 6, 2023

Implements this comment from when these commands were introduced originally. Main motivation is that I'd like to add a new subcommand to bitcoin-util unrelated to the existing grind functionality, and have options for it.

@maflcko
Copy link
Member

maflcko commented Nov 6, 2023

tool_wallet.py fails CI

@ajtowns ajtowns force-pushed the 202311-argsman-subcmd-options branch from abba7c5 to 162c5ab Compare November 6, 2023 18:37
@DrahtBot DrahtBot removed the CI failed label Nov 6, 2023
@ajtowns ajtowns force-pushed the 202311-argsman-subcmd-options branch from 162c5ab to 86fab48 Compare November 6, 2023 22:50
@ajtowns ajtowns force-pushed the 202311-argsman-subcmd-options branch from 86fab48 to 6be2b82 Compare November 6, 2023 23:02
@DrahtBot DrahtBot removed the CI failed label Nov 6, 2023
Copy link
Contributor

@stickies-v stickies-v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approach ACK

@ajtowns
Copy link
Contributor Author

ajtowns commented Mar 16, 2025

Leaving as draft pending #31250

@ajtowns ajtowns force-pushed the 202311-argsman-subcmd-options branch from 80c0927 to c064ad2 Compare May 14, 2025 02:12
@ajtowns ajtowns force-pushed the 202311-argsman-subcmd-options branch from c064ad2 to 0d1d11f Compare May 14, 2025 04:37
@ajtowns ajtowns marked this pull request as ready for review May 14, 2025 04:38
@ajtowns
Copy link
Contributor Author

ajtowns commented May 14, 2025

Rebased and undrafted

Copy link
Contributor

@l0rinc l0rinc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK, it makes sense to associate specific command-line options with specific commands - like function names with args. I can imagine this being useful in other cases as well. Given that the PR needs a rebase, I have commented on every nit I could find - and I would need full test coverage for this to be comfortable acking, but like the commit split otherwise. If you can, I would appreciate adding slightly more context to the PR description and the commit messages.

@@ -66,6 +66,8 @@ enum class OptionsCategory {
CLI_COMMANDS,
IPC,

COMMAND_OPTIONS, // Specific to one or more commands
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems safe, just have to make sure: is it safe to add in the middle of this enum (e.g. is its index persisted anywhere?).
If the placement doesn't matter, can we add the options closer to the command values (before IPC)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Placement doesn't matter, but I don't think moving closer to COMMANDS, REGISTER_COMMANDS or CLI_COMMANDS really makes sense -- COMMAND_OPTIONS relates to argsman.AddCommand() commands (used by bitcoin-wallet and bitcoin-util), which differ from COMMANDS (used by bitcoin-tx), REGISTER_COMMANDS (also used by bitcoin-tx), and CLI_COMMANDS (used by bitcoin-cli for its non-rpc commands, -generate, -netinfo, -addrinfo, -getinfo).

@@ -347,9 +350,9 @@ class ArgsManager
void AddArg(const std::string& name, const std::string& help, unsigned int flags, const OptionsCategory& cat);

/**
* Add subcommand
* Add command
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this comment still useful now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a docstring, so shows up in the doxygen docs https://doxygen.bitcoincore.org/class_args_manager.html#ab71d531303f7e6fe3bbf67683a414a2e ; maybe it's not that useful, but doesn't seem worth removing outright

@@ -137,6 +139,7 @@ class ArgsManager
std::string m_network GUARDED_BY(cs_args);
std::set<std::string> m_network_only_args GUARDED_BY(cs_args);
std::map<OptionsCategory, std::map<std::string, Arg>> m_available_args GUARDED_BY(cs_args);
std::map<std::string, std::set<std::string>> m_command_args GUARDED_BY(cs_args);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line seems to conflict with latest master.

nit: is it important for the set to be sorted here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's looked up by the key in CheckCommandOptions, so sorting seems fine to me?

{
if (select.empty()) return;
if (m_iter == m_end) return;
for (const auto& [cmdopt_name, cmdopt_info] : m_iter->second) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: when is it opt and when cmdopt?

argsman.AddArg("-debug=<category>", "Output debugging information (default: 0).", ArgsManager::ALLOW_ANY, OptionsCategory::DEBUG_TEST);
argsman.AddArg("-printtoconsole", "Send trace/debug info to console (default: 1 when no -debug is true, 0 otherwise).", ArgsManager::ALLOW_ANY, OptionsCategory::DEBUG_TEST);

argsman.AddCommand("info", "Get wallet info");
argsman.AddCommand("create", "Create a new descriptor wallet file");
argsman.AddCommand("dump", "Print out all of the wallet key-value records");
argsman.AddCommand("createfromdump", "Create new wallet file from dumped records");
argsman.AddCommand("dump", "Print out all of the wallet key-value records", {"-dumpfile"});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are there any other such relations between command and arg that we could check this way?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not currently, I think

@achow101
Copy link
Member

Are you still working on this? This has needed rebase for a few months now.

@ajtowns ajtowns force-pushed the 202311-argsman-subcmd-options branch from 0d1d11f to cce7eba Compare November 18, 2025 05:31
@ajtowns
Copy link
Contributor Author

ajtowns commented Nov 18, 2025

If you can, I would appreciate adding slightly more context to the PR description and the commit messages.

Sorry, these seem completely self-explanatory to me, so I don't see what to add. Happy to answer questions or take suggestions though.

@ryanofsky
Copy link
Contributor

Concept ACK cce7eba. This PR improves documentation and error handling for subcommand specific options.

I do think the PR could use a more detailed description. (I wasn't very familiar with existing subcommand support so I didn't know what was changing.)

Right now, ArgsManager supports subcommands like bitcoin-wallet dump and supports subcommand-specific options like -dumpfile, e.g.:

bitcoin-wallet -wallet=foo -dumpfile=foo.txt dump

But ArgsManager doesn't know that -dumpfile belongs only to certain subcommands, so it:

  • lists -dumpfile in the global help rather than under the relevant subcommands, and
  • does not verify that -dumpfile is used with a compatible subcommand, leaving that error handling to the wallet code.

This PR improves on that, changing help and error output as follows:

@@ -14,10 +14,6 @@
   -datadir=<dir>
        Specify data directory
 
-  -dumpfile=<file name>
-       When used with 'dump', writes out the records to this file. When used
-       with 'createfromdump', loads the records into a new wallet.
-
   -help
        Print this help message and exit (also -h or -?)
 
@@ -73,9 +69,19 @@
   createfromdump
        Create new wallet file from dumped records
 
+       -dumpfile=<file name>
+            When used with 'dump', writes out the records to this file. When
+            used with 'createfromdump', loads the records into a
+            new wallet.
+
   dump
        Print out all of the wallet key-value records
 
+       -dumpfile=<file name>
+            When used with 'dump', writes out the records to this file. When
+            used with 'createfromdump', loads the records into a
+            new wallet.
+
   info
        Get wallet info
 $ bitcoin-wallet -regtest -wallet=foo -dumpfile=out info
-The -dumpfile option can only be used with the "dump" and "createfromdump" commands.
+Error: Invalid arguments provided:
+- The -dumpfile option cannot be used with the 'info' command.

Limitations

  • As can be seen above, if an option applies to multiple subcommands, it must share the same help text. There's no way to provide subcommand-specific descriptions.

  • This does not change option parsing rules. Options still cannot appear after the subcommand:

    $ bitcoin-wallet -wallet=foo dump -dumpfile=foo.txt
    Error: Additional arguments provided (-dumpfile=foo.txt). Methods do not take arguments. Please refer to `-help`.

Nevertheless, this change should be a strict improvement.

@ajtowns
Copy link
Contributor Author

ajtowns commented Nov 19, 2025

I do think the PR could use a more detailed description.

Should I cut and paste what you wrote below into the PR or commit description? Sorry, I need an ELI5 here :)

@ryanofsky
Copy link
Contributor

Should I cut and paste what you wrote below into the PR or commit description?

Feel free to use & change any of it. I mostly wrote that for myself to be able to remember what changes this PR was making. I do think the current description is perfectly ok and accurate, but I skipped over this PR many times not really knowing what it is was doing or if I should take the time to find out, so adding some more details could help this get more attention.

Copy link
Contributor

@ryanofsky ryanofsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review ACK cce7eba. Left various suggestions, but none are important so feel free to ignore.

namespace {
/** Helper class for iterating over COMMAND_OPTIONS applicable to a given command */
template <typename T>
class CommandOptionsGetter
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "ArgsManager: support command-specific options" (e2debfa)

Any plans to reuse this class? It would seem simpler to inline:

diff

--- a/src/common/args.cpp
+++ b/src/common/args.cpp
@@ -646,35 +646,6 @@ void ArgsManager::CheckMultipleCLIArgs() const
     }
 }
 
-namespace {
-/** Helper class for iterating over COMMAND_OPTIONS applicable to a given command */
-template <typename T>
-class CommandOptionsGetter
-{
-private:
-    const typename T::const_iterator m_end;
-    const typename T::const_iterator m_iter;
-public:
-    CommandOptionsGetter(const T& available_args)
-    : m_end{available_args.end()},
-      m_iter{available_args.find(OptionsCategory::COMMAND_OPTIONS)}
-    {
-    }
-
-    template <typename Fn>
-    void Iterate(const std::set<std::string>& select, bool with_debug, Fn&& fn) const
-    {
-        if (select.empty()) return;
-        if (m_iter == m_end) return;
-        for (const auto& [cmdopt_name, cmdopt_info] : m_iter->second) {
-            if (!with_debug && (cmdopt_info.m_flags & ArgsManager::DEBUG_ONLY)) continue;
-            if (!select.contains(cmdopt_name)) continue;
-            fn(cmdopt_name, cmdopt_info);
-        }
-    }
-};
-} // anonymous namespace
-
 std::string ArgsManager::GetHelpMessage() const
 {
     const bool show_debug = GetBoolArg("-help-debug", false);
@@ -682,7 +653,7 @@ std::string ArgsManager::GetHelpMessage() const
     std::string usage;
     LOCK(cs_args);
 
-    const auto command_options = CommandOptionsGetter(m_available_args);
+    const auto command_options = m_available_args.find(OptionsCategory::COMMAND_OPTIONS);
 
     for (const auto& [category, category_args] : m_available_args) {
         switch(category) {
@@ -748,10 +719,12 @@ std::string ArgsManager::GetHelpMessage() const
 
                 if (category == OptionsCategory::COMMANDS) {
                     const auto cmd_args = m_command_args.find(arg_name);
-                    if (cmd_args != m_command_args.end()) {
-                        command_options.Iterate(cmd_args->second, show_debug, [&](const auto& cmdopt_name, const auto& cmdopt_info) {
+                    if (cmd_args != m_command_args.end() && command_options != m_available_args.end()) {
+                        for (const auto& [cmdopt_name, cmdopt_info] : command_options->second) {
+                            if (!show_debug && (cmdopt_info.m_flags & ArgsManager::DEBUG_ONLY)) continue;
+                            if (!cmd_args->second.contains(cmdopt_name)) continue;
                             usage += HelpMessageSubOpt(cmdopt_name, cmdopt_info.m_help_param, cmdopt_info.m_help_text);
-                        });
+                        }
                     }
                 }
             }

* @param option Option name (e.g. "-rpcuser")
* @param help_param Help parameter (e.g. "=<user>" or "")
* @param message Option description (e.g. "Username for JSON-RPC connections")
* @param indent Additional indentation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "ArgsManager: support command-specific options" (e2debfa)

Giving this function an indent parameter doesn't seem right semantically since the other parameters are higher-level parameters unrelated to formatting, and the function is still hardcoding most other formatting value internally. This also increases code complexity by requiring a separate HelpMessageSubOpt function, because the caller doesn't have access to the formatting constants. Would suggest keeping this high-level and simplifying code with a change like:

diff

--- a/src/common/args.cpp
+++ b/src/common/args.cpp
@@ -750,7 +750,7 @@ std::string ArgsManager::GetHelpMessage() const
                     const auto cmd_args = m_command_args.find(arg_name);
                     if (cmd_args != m_command_args.end()) {
                         command_options.Iterate(cmd_args->second, show_debug, [&](const auto& cmdopt_name, const auto& cmdopt_info) {
-                            usage += HelpMessageSubOpt(cmdopt_name, cmdopt_info.m_help_param, cmdopt_info.m_help_text);
+                            usage += HelpMessageOpt(cmdopt_name, cmdopt_info.m_help_param, cmdopt_info.m_help_text, /*subopt=*/true);
                         });
                     }
                 }
@@ -779,8 +779,9 @@ std::string HelpMessageGroup(const std::string &message) {
     return std::string(message) + std::string("\n\n");
 }
 
-std::string HelpMessageOpt(std::string_view option, std::string_view help_param, std::string_view message, int indent)
+std::string HelpMessageOpt(std::string_view option, std::string_view help_param, std::string_view message, bool subopt)
 {
+    const int indent{subopt ? msgIndent - optIndent : 0};
     return strprintf("%*s%s%s\n%*s%s\n\n",
            (optIndent + indent), "", option, help_param,
            (msgIndent + indent), "", FormatParagraph(message, screenWidth - msgIndent - indent, msgIndent + indent));
@@ -800,11 +801,6 @@ bool HasTestOption(const ArgsManager& args, const std::string& test_option)
     });
 }
 
-std::string HelpMessageSubOpt(std::string_view option, std::string_view help_param, std::string_view message)
-{
-    return HelpMessageOpt(option, help_param, message, msgIndent - optIndent);
-}
-
 fs::path GetDefaultDataDir()
 {
     // Windows:
--- a/src/common/args.h
+++ b/src/common/args.h
@@ -486,19 +486,9 @@ std::string HelpMessageGroup(const std::string& message);
  * @param option Option name (e.g. "-rpcuser")
  * @param help_param Help parameter (e.g. "=<user>" or "")
  * @param message Option description (e.g. "Username for JSON-RPC connections")
- * @param indent Additional indentation
+ * @param subopt True if this is suboption, instead of a top level option.
  * @return the formatted string
  */
-std::string HelpMessageOpt(std::string_view option, std::string_view help_param, std::string_view message, int indent=0);
-
-/**
- * Same as HelpMessageOpt, but indents for command-specific options
- *
- * @param option Option name (e.g. "-rpcuser")
- * @param help_param Help parameter (e.g. "=<user>" or "")
- * @param message Option description (e.g. "Username for JSON-RPC connections")
- * @return the formatted string
- */
-std::string HelpMessageSubOpt(std::string_view option, std::string_view help_param, std::string_view message);
+std::string HelpMessageOpt(std::string_view option, std::string_view help_param, std::string_view message, bool subopt=false);
 
 #endif // BITCOIN_COMMON_ARGS_H

CLI_COMMANDS,
IPC,

COMMAND_OPTIONS, // Specific to one or more commands
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "ArgsManager: support command-specific options" (e2debfa)

Might be worth noting that options with this category will be excluded from help output unless they are referenced by an AddCommand call.

Relatedly it might be good to add a check that every option in the COMMAND_OPTIONS category is actually referenced by an AddCommand call.

std::map<std::string, Arg>& arg_map = m_available_args[OptionsCategory::COMMANDS];
auto ret = arg_map.emplace(cmd, Arg{"", help, ArgsManager::COMMAND});
if (!options.empty()) {
m_command_args.try_emplace(cmd, std::move(options));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "ArgsManager: support command-specific options" (e2debfa)

Should the options list be validated against known options? Seems like it could be easy to make a typo and pass a bad list of options to this function, causing help output to be incomplete without it being obvious.

const auto command = test_args.GetCommand();
if (!command) return NO_COMMAND;
std::vector<std::string> details;
if (!test_args.CheckCommandOptions(command->command, &details)) return COMMAND_OPTS;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In commit "tests: Add some test coverage for ArgsManager::AddCommand" (cce7eba)

Might be good to check details is empty on success, nonempty on failure

@DrahtBot DrahtBot requested a review from l0rinc November 19, 2025 17:12
argsman.AddArg("-datadir=<dir>", "Specify data directory", ArgsManager::ALLOW_ANY | ArgsManager::DISALLOW_NEGATION, OptionsCategory::OPTIONS);
argsman.AddArg("-wallet=<wallet-name>", "Specify wallet name", ArgsManager::ALLOW_ANY | ArgsManager::NETWORK_ONLY, OptionsCategory::OPTIONS);
argsman.AddArg("-dumpfile=<file name>", "When used with 'dump', writes out the records to this file. When used with 'createfromdump', loads the records into a new wallet.", ArgsManager::ALLOW_ANY | ArgsManager::DISALLOW_NEGATION, OptionsCategory::OPTIONS);
argsman.AddArg("-dumpfile=<file name>", "When used with 'dump', writes out the records to this file. When used with 'createfromdump', loads the records into a new wallet.", ArgsManager::ALLOW_ANY | ArgsManager::DISALLOW_NEGATION, OptionsCategory::COMMAND_OPTIONS);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps a future improvement/ follow-up could be specifying the help message description for each command:

Suggested change
argsman.AddArg("-dumpfile=<file name>", "When used with 'dump', writes out the records to this file. When used with 'createfromdump', loads the records into a new wallet.", ArgsManager::ALLOW_ANY | ArgsManager::DISALLOW_NEGATION, OptionsCategory::COMMAND_OPTIONS);
argsman.AddArg("-dumpfile=<file name>", "{'dump':'Writes out the records to this file.','createfromdump':'Loads the records into a new wallet.'}", ArgsManager::ALLOW_ANY | ArgsManager::DISALLOW_NEGATION, OptionsCategory::COMMAND_OPTIONS);```

Copy link
Member

@pablomartin4btc pablomartin4btc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concept ACK

I like the direction of this PR and its improvements. One thing that perhaps could still confuse reviewers I think is the terminology:

  • the PR title/description still refer to subcommands, but the API and implementation consistently use command (AddCommand() - updated docstring * Add subcommand to * Add command-, COMMAND_OPTIONS, updated comments per achow’s suggestion),
  • except for the multiprocess bitcoin rpc case, these are really top-level CLI commands (bitcoin-wallet dump, bitcoin-util grind, etc.). Aligning the wording (and helpers like HelpMessageSubOpt) with “command-specific options” would reduce ambiguity (HelpMessageCommandSpecificOpt).

Also, the help output now duplicates the same option description under each command (e.g. -dumpfile under both dump and createfromdump), even though semantics differ slightly per command. This works, but it might be worth clarifying in comments that this duplication is intentional, or noting it as a potential future improvement (as suggested).

Adding a short before/after example and expanding the description beyond bitcoin-wallet (e.g. bitcoin-util grind) would likely help attract more reviewers.

Should we include the new category into the system fuzz (already tried and ran it myself):

const OptionsCategory options_category = fuzzed_data_provider.PickValueInArray<OptionsCategory>({OptionsCategory::OPTIONS, OptionsCategory::CONNECTION, OptionsCategory::WALLET, OptionsCategory::WALLET_DEBUG_TEST, OptionsCategory::ZMQ, OptionsCategory::DEBUG_TEST, OptionsCategory::CHAINPARAMS, OptionsCategory::NODE_RELAY, OptionsCategory::BLOCK_CREATION, OptionsCategory::RPC, OptionsCategory::GUI, OptionsCategory::COMMANDS, OptionsCategory::REGISTER_COMMANDS, OptionsCategory::CLI_COMMANDS, OptionsCategory::IPC, OptionsCategory::HIDDEN});

@sedited
Copy link
Contributor

sedited commented Dec 23, 2025

Concept ACK, was working in this area a few days ago, and thought having something like this would be very useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants