Skip to content

Conversation

@mzumsande
Copy link
Contributor

Running large commands via bitcoin-cli (such as submitting a large tx or block) runs into the MAX_ARG_STRLEN limit, which is 128KB on many systems. (see e.g. https://trofi.github.io/posts/299-maximum-argument-count-on-linux-and-in-gcc.html for more info).

This PR suggests to make it possible to bypass this limit by allowing to put the command and all of its args into a file. The functional test framework is then adjusted to automatically use this option (using a temporary file) if --usecli is specified and there is an arg that is too large.
This might also be helpful for users outside of the functional tests (although there is always the option to use the rpc interface directly instead of bitcoin-cli).

This is something many of our functional tests run into when executed with --usecli on master: (OSError: [Errno 7] Argument list too long: 'bitcoin-cli')
The third commit enables several of them - there are a few more tests than those mentioned there, that could be enabled in a follow-up, but need some more adjustments beyond that.

@DrahtBot
Copy link
Contributor

DrahtBot commented May 1, 2025

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/32399.

Reviews

See the guideline for information on the review process.
A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

  • #32297 (bitcoin-cli: Add -ipcconnect option by ryanofsky)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

This allows to do rpc requests via cli when they would otherwise not
be possible due to execve limits.
@mzumsande mzumsande force-pushed the 202505_cli_loadfile branch from c30c7d4 to 85a3156 Compare May 2, 2025 01:58
mzumsande added 2 commits May 1, 2025 22:03
Also, the following tests (for which self.supports_cli = False was not
set) will now work with --usecli:

feature_fastprune.py
feature_fee_estimation.py
feature_reindex_readonly.py
feature_taproot.py
mempool_package_rbf.py
p2p_net_deadlock.py
p2p_tx_download.py
rpc_packages.py
@0xB10C
Copy link
Contributor

0xB10C commented May 2, 2025

This might also be helpful for users outside of the functional tests (although there is always the option to use the rpc interface directly instead of bitcoin-cli).

I've been using -stdin (e.g. cat block.hex | bitcoin-cli -stdin submitblock) for this. Would this work here too?

@mzumsande
Copy link
Contributor Author

I've been using -stdin (e.g. cat block.hex | bitcoin-cli -stdin submitblock) for this. Would this work here too?

Oh, I didn't think of that - I tried it out and I think that would work too - that would make it simpler, no changes to bitcoin-cli necessary, will change to that approach.

@mzumsande
Copy link
Contributor Author

Since this will only change the test framework now, I'll just add it to #32290 - no extra PR needed anymore.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants