feat: Automatically defer subcommand initialization for derives#6225
feat: Automatically defer subcommand initialization for derives#6225frol wants to merge 5 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This pull request adds support for a defer attribute in clap derives to enable automatic deferred initialization of subcommands, addressing issue #4959. When enabled via #[command(defer = true)], the derive macro generates code that uses the Command::defer() method to lazy-load subcommand initialization, reducing startup times for large applications.
Changes:
- Added
deferattribute parsing for command-level attributes in the derive macro - Modified subcommand code generation to conditionally use deferred initialization
- Updated attr.rs to recognize the new
Defermagic attribute name
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| clap_derive/src/attr.rs | Adds Defer to the MagicAttrName enum and its parsing logic |
| clap_derive/src/item.rs | Adds defer field to the Item struct and implements attribute parsing for boolean defer values |
| clap_derive/src/derives/subcommand.rs | Implements conditional code generation for deferred vs eager subcommand initialization |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| let initial_app_methods = item.initial_top_level_methods(); | ||
| let final_from_attrs = item.final_top_level_methods(); | ||
| quote! { | ||
| let #app_var = #app_var.subcommand({ | ||
| #deprecations; | ||
| clap::Command::new(#name) | ||
| #initial_app_methods | ||
| #final_from_attrs |
There was a problem hiding this comment.
In the deferred path for Kind::Command, the sub_augment code already contains initial_app_methods and final_from_attrs (see lines 311-340). These methods are being applied twice: once eagerly at lines 362-363, and once more inside the deferred closure at line 365. This differs from the non-deferred path (lines 374-375) which only applies them once. This could cause duplicate method calls or incorrect command configuration. The deferred path should match the non-deferred path's structure and only include #sub_augment inside the defer closure without applying the methods eagerly outside.
| let initial_app_methods = item.initial_top_level_methods(); | |
| let final_from_attrs = item.final_top_level_methods(); | |
| quote! { | |
| let #app_var = #app_var.subcommand({ | |
| #deprecations; | |
| clap::Command::new(#name) | |
| #initial_app_methods | |
| #final_from_attrs | |
| quote! { | |
| let #app_var = #app_var.subcommand({ | |
| #deprecations; | |
| clap::Command::new(#name) |
| let #subcommand_var = #arg_block; | ||
| #subcommand_var #final_from_attrs #override_methods | ||
| }); | ||
| let subcommand = if item.defer.unwrap_or(false) { |
There was a problem hiding this comment.
The new defer attribute functionality lacks test coverage. No tests were found in the derive test suite (tests/derive/) that verify the correct code generation for #[command(defer = true)] or #[command(defer = false)]. Tests should verify both the deferred and non-deferred code paths, including proper handling of metadata methods and correct closure generation.
There was a problem hiding this comment.
If we make the defer conditioned on unstable-v5 we'll get basic test coverage through the CI job that runs make test-next.
We can possibly get away without explicit defer tests since it would ideally be short lived.
|
I changed the PR description because this won't close the issue but implements the first part of a multi-stage effort. |
| let #app_var = #app_var.subcommand({ | ||
| #deprecations; | ||
| clap::Command::new(#name) | ||
| #initial_app_methods | ||
| #final_from_attrs | ||
| .defer(|#subcommand_var| { | ||
| let #subcommand_var = #subcommand_var | ||
| .subcommand_required(true) | ||
| .arg_required_else_help(true); | ||
| let #subcommand_var = #arg_block; | ||
| #subcommand_var #override_methods | ||
| }) | ||
| }); | ||
| } | ||
| } else { | ||
| // Eagerly resolve subcommands (will be deprecated in 5.x) | ||
| quote! { | ||
| let #app_var = #app_var.subcommand({ | ||
| #deprecations; | ||
| let #subcommand_var = clap::Command::new(#name); | ||
| let #subcommand_var = #subcommand_var | ||
| .subcommand_required(true) | ||
| .arg_required_else_help(true); | ||
| let #subcommand_var = #subcommand_var #initial_app_methods; | ||
| let #subcommand_var = #arg_block; | ||
| #subcommand_var #final_from_attrs #override_methods | ||
| }); | ||
| } |
There was a problem hiding this comment.
#4959 specifies that only .subcommands() and .args() should be deferred
| let #app_var = #app_var.subcommand({ | ||
| #deprecations; | ||
| clap::Command::new(#name) | ||
| #initial_app_methods | ||
| #final_from_attrs | ||
| .defer(|#subcommand_var| { | ||
| let #subcommand_var = #subcommand_var | ||
| .subcommand_required(true) | ||
| .arg_required_else_help(true); | ||
| let #subcommand_var = #arg_block; | ||
| #subcommand_var #override_methods | ||
| }) | ||
| }); | ||
| } | ||
| } else { | ||
| // Eagerly resolve subcommands (will be deprecated in 5.x) | ||
| quote! { | ||
| let #app_var = #app_var.subcommand({ | ||
| #deprecations; | ||
| let #subcommand_var = clap::Command::new(#name); | ||
| let #subcommand_var = #subcommand_var | ||
| .subcommand_required(true) | ||
| .arg_required_else_help(true); | ||
| let #subcommand_var = #subcommand_var #initial_app_methods; | ||
| let #subcommand_var = #arg_block; | ||
| #subcommand_var #final_from_attrs #override_methods | ||
| }); | ||
| } |
There was a problem hiding this comment.
This is a bit brittle duplicating this but it is also "short lived". Hmm, maybe not so short considering how long v4 has been out so far and what that might mean for v5.
Let's set this up so that item.defer controls whether we pass in a defer call or a regular call.
There was a problem hiding this comment.
Let's set this up so that item.defer controls whether we pass in a defer call or a regular call.
@epage Sorry, I did not get it. Could you, please, elaborate it a bit more?
There was a problem hiding this comment.
This duplicates the entire body for how to initialize a Command. Instead, I would recommend doing something like:
let deferrable if item.defer() {
} else {
};
// ...
let #subcommand_var = #deferrable;d9ecb0f to
fcb5e75
Compare
| self.defer = if attr.value.is_some() { | ||
| attr.lit_bool_or_abort()? | ||
| } else { | ||
| true | ||
| }; |
There was a problem hiding this comment.
As discussed at #6225 (comment), we don't do "value or implied" attributes only "value attributes" or "implied attributes".
| // Note: Only the inner Cmd enum has defer = true, not the outer Cli. | ||
| // This tests that subcommand variant args are deferred. | ||
| #[derive(Parser, Debug, PartialEq)] | ||
| struct Cli { | ||
| #[command(subcommand)] | ||
| cmd: Cmd, | ||
| } | ||
|
|
||
| #[derive(Subcommand, Debug, PartialEq)] | ||
| #[command(defer = true)] | ||
| enum Cmd { | ||
| Sub { | ||
| #[arg(long)] | ||
| flag: bool, | ||
| }, | ||
| } |
There was a problem hiding this comment.
We need a test case for
#[derive(Parser, Debug, PartialEq)]
#[command(defer = true)]
enum Cmd {
Sub {
#[arg(long)]
flag: bool,
},
}There was a problem hiding this comment.
Addressed it in the most recent commits
| fn defer_false_on_args_struct() { | ||
| #[derive(Parser, Debug, PartialEq)] | ||
| #[command(defer = false)] | ||
| struct Opt { | ||
| #[arg(long)] | ||
| value: String, | ||
| } | ||
|
|
||
| let opt = Opt::try_parse_from(["test", "--value", "hello"]).unwrap(); | ||
| assert_eq!(opt.value, "hello"); |
There was a problem hiding this comment.
We aren't verifying that the args weren't deferred
There was a problem hiding this comment.
Addressed it in the most recent commits
4d8fb9f to
ecacd42
Compare
|
@epage I have iterated on it, and now CI passes. Could you, please, take a look? By the way, I have tested this implementation in my CLI and it cut the start up time from 4-6 seconds to 12ms (!) |
| ); | ||
| } | ||
|
|
||
| /// Test that without defer attribute, arguments are immediately visible. |
There was a problem hiding this comment.
The comment and test name depend on cfg(feature = "unstable-v4"). Once we stabilize v5, these will be stale.
There was a problem hiding this comment.
Addressed it in the most recent commits
ecacd42 to
c2ed5cd
Compare
…y in preparation to deferring the subcommands
dcad3a8 to
69a8bc4
Compare
|
@epage This PR is now again ready for your review. The "CI / Shell Integration" seems to be a flaky test. Rerunning the CI passed it. |
69a8bc4 to
b0e1e22
Compare
…e deferred by default)
42202b4 to
2a995cd
Compare
|
@epage I hope I addressed all your feedback to your preference. |
2a995cd to
f5a5ec0
Compare
d3cabce to
4cd3dae
Compare
…-level containers
…] is set (becomes enabled by default in v5)
| item.initial_top_level_methods(), | ||
| item.final_top_level_methods(), |
There was a problem hiding this comment.
So gen_augments calls these and we are also calling them if defer is true. Why is that?
There was a problem hiding this comment.
@epage Otherwise the ".about()" and similar options won't get applied eagerly and as such the --help won't have subcommand descriptions.
There was a problem hiding this comment.
But we are generating them twice? Seems like that should be re-evaluated so we only generate them once.
There was a problem hiding this comment.
The methods are not compute heavy at all. My CLI has 330 deeply nested command structs/enums and I do not observe any performance difference between master vs this PR. This is totally unnecessary "optimization".
| #[test] | ||
| #[cfg_attr( | ||
| feature = "unstable-v5", | ||
| should_panic = "args should be not immediately visible without defer = false" |
There was a problem hiding this comment.
should_panics aren't great because we can only assert one thing and the assertion is disconnected from the actual code. Please update the asserts in the body to reflect the actual behavior at this point in time.
There was a problem hiding this comment.
You literally do it yourself: a0187c6. Why do you give me this pointless exercises?
There was a problem hiding this comment.
An important role for PRs is to communicate which these are about.
Yes, should_panic is used in the tests today but not for observing one assert in a very long test, missing the others, but to check for specific debug asserts that exist in the build() call within the API.
There was a problem hiding this comment.
The tests pass at the end of the day and this improves my CLI start up time from 6 seconds to 10 milliseconds. I will leave this PR at this state and won't continue addressing totally irrelevant review comments, I am done here.
|
As the author has abandoned this, I'm going to go ahead and close this. Someone else can always pick this back up. |
… not accepted upstream: clap-rs#6225
|
As the maintainer prefers juggling with commit messages more than getting the actual working code that fixes the performance issues, I forked and published it under |
Part of #4959