[pylint] Implement consider-using-assignment-expr (R6103)#13196
[pylint] Implement consider-using-assignment-expr (R6103)#13196vincevannoort wants to merge 17 commits intoastral-sh:mainfrom
consider-using-assignment-expr (R6103)#13196Conversation
CodSpeed Performance ReportMerging #13196 will not alter performanceComparing Summary
|
vincevannoort
changed the title
R6103R6103
vincevannoort
force-pushed
the
consider-using-assignment-expr
branch
from
b422d5c to
dbefe63
Compare
|
| code | total | + violation | - violation | + fix | - fix |
|---|---|---|---|---|---|
| PLR6103 | 4612 | 4612 | 0 | 0 | 0 |
vincevannoort
force-pushed
the
consider-using-assignment-expr
branch
from
84c2180 to
4d891db
Compare
| if checker.enabled(Rule::UnnecessaryAssignment) { | ||
| pylint::rules::unnecessary_assignment(checker, if_); | ||
| } |
There was a problem hiding this comment.
I applied this rule by checking whether the previous statement from a IfStmt is an AssignStmt, however I am wondering if it is more desirable to do check if the next statement of a AssignStmt is an IfStmt?
There was a problem hiding this comment.
What benefits do you see in testing the next statement after an AssignStmt.
My intuition here is that there are probably more assignment than if statements. Therefore, running the rules on if nodes might overall be faster?
There was a problem hiding this comment.
I agree that assignments are far more common than if statements.
I think the answer depends on the costs for checking previous_statement and next_statement.
My thought here is that retrieving the previous statement using the newly added previous_statement might be more expensive because it has to iterate over all previous statements using previous_statements to find the previous statement (if there is a better way, please let me know).
While checking an assignment, then checking the next_statement might be a cheap operation.
Do you have any idea? If they have equal cost I think the current implementation is fine. 😄
There was a problem hiding this comment.
I can change it, to the suggested approach if you want 😄
vincevannoort
changed the title
R6103consider-using-assignment-expr (R6103)
vincevannoort
changed the title
consider-using-assignment-expr (R6103)consider-using-assignment-expr (R6103)
| pub fn previous_statement(&self, stmt: &'a Stmt) -> Option<&Stmt> { | ||
| self.previous_statements(stmt)?.next() | ||
| } |
There was a problem hiding this comment.
I needed something like this function, not sure if there is an existing better way?
There was a problem hiding this comment.
That makes sense. I think we can make it more efficient with #13895
which reduces an extra find and collect
vincevannoort
force-pushed
the
consider-using-assignment-expr
branch
from
07d2997 to
964c3d6
Compare
MichaReiser
added
rule
MichaReiser
left a comment
MichaReiser
left a comment
There was a problem hiding this comment.
Thanks for this contribution. My plane is about to land. I've to finish the review at a later time.
I only had enough time to quickly glance over the Rust code. We should look into removing the many node.clone() calls because it is fairly expensive to clone nodes and probably unnecessary. Let me know if you need some guidance on how to remove the clone calls (It probably requires adding some lifetimes)
Regarding the rule's naming.
- I did a quick search to see how we referred to
:=in other rules. There are not many usages butnamed expression (walrus operator)is the most common form. - The rule name seems too generic to me and its name is very similar to
unnecessary-assign(which we should rename to `unnecessary-assignment). Reading through the examples the rule mainly is about assigning a value that is then only used in an if condition. I need to think a bit more about what a good rule name could be. Maybe you have an idea?
| bad5 = ( | ||
| 'example', | ||
| 'example', | ||
| 'example', | ||
| 'example', | ||
| 'example', | ||
| 'example', | ||
| 'example', | ||
| 'example', | ||
| 'example', | ||
| 'example', | ||
| ) |
There was a problem hiding this comment.
I like the example because it shows a potentially controversial use case. I would probably prefer the assignment to keep the if smaller.
There was a problem hiding this comment.
I agree, should we check the complexity of the assignment to determine whether to apply the rule? If so, how do we determine where to draw the line?
crates/ruff_linter/resources/test/fixtures/pylint/unnecessary_assignment.py
Show resolved
Hide resolved
| if checker.enabled(Rule::UnnecessaryAssignment) { | ||
| pylint::rules::unnecessary_assignment(checker, if_); | ||
| } |
There was a problem hiding this comment.
What benefits do you see in testing the next statement after an AssignStmt.
My intuition here is that there are probably more assignment than if statements. Therefore, running the rules on if nodes might overall be faster?
crates/ruff_linter/src/rules/pylint/rules/unnecessary_assignment.rs
Outdated
Show resolved
Hide resolved
crates/ruff_linter/src/rules/pylint/rules/unnecessary_assignment.rs
Outdated
Show resolved
Hide resolved
Thanks for the review, much appreciated! 👍
I have tried and was able to remove almost all clone calls, except a few which I think are needed for returning the diagnostic. Could you take a look at the remaining ones and see if any of the 3 can still be removed?
I agree, here are some possible options:
These seem close to what the lint is trying to prevent, do you have other ideas in mind? |
|
Hey @MichaReiser, once you have the time, would you mind giving this pull request another look? 😄 |
|
Hey @MichaReiser, could you or someone else from the team have a look? 😄 |
There was a problem hiding this comment.
Thanks for the ping and sorry for the late review. I pushed a few smaller refactors to avoid unnecessary collects.
I created a PR that should allow us to implement a more efficient previous_statement here.
I think the rule has to become cleverer, at least when we want to support handling elif cases because today's implementation can result in changes that fail at runtime.
| }; | ||
| } | ||
|
|
||
| // case - elif else clauses (`elif test1:`) |
There was a problem hiding this comment.
Is there a reason why compare expressions aren't handled inside elif_else clauses?
| errors.extend( | ||
| stmt.elif_else_clauses | ||
| .iter() | ||
| .filter(|elif_else_clause| elif_else_clause.test.is_some()) | ||
| .filter_map(|elif_else_clause| { | ||
| let elif_check = elif_else_clause.test.as_ref().unwrap(); | ||
| find_assignment_before_if_stmt(semantic, elif_check, elif_check) | ||
| }) | ||
| .collect::<Vec<AssignmentBeforeIfStmt>>(), |
There was a problem hiding this comment.
Changing assignments to walrus operators in elif else branches is semantically incorrect if the variable is used afterwards
>>> if True: ...
... elif x :=10: ...
...
Ellipsis
>>> x
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'x' is not definedThere was a problem hiding this comment.
Yeah, I think you are right. Due to that I decided to only apply the rule to simple if statements, where simple means without elif or else attached to the if. That should prevent cases like this.
| error: AssignmentBeforeIfStmt, | ||
| ) -> Diagnostic { | ||
| let (origin, expr_name, assignment) = error; | ||
| let assignment_expr = generator.expr(&assignment.value); |
There was a problem hiding this comment.
What's the motivation for calling into the generator here? We try to avoid using the generator because it removes comments. Could we instead take the assignment value as it is in the source (using locator)? Note: We have to be careful about parenthesized expressions.
There was a problem hiding this comment.
Could you help me out here? I am not that familiar in how to do this properly.
| pub fn previous_statement(&self, stmt: &'a Stmt) -> Option<&Stmt> { | ||
| self.previous_statements(stmt)?.next() | ||
| } |
There was a problem hiding this comment.
That makes sense. I think we can make it more efficient with #13895
which reduces an extra find and collect
| bad7 = 'example' | ||
| if bad7 == 'something': # [consider-using-assignment-expr] | ||
| pass | ||
| elif bad7 == 'something else': | ||
| pass |
There was a problem hiding this comment.
This is an interesting example and possibly controversial. I would prefer the existing solution because the assignment in the if is very subtle and I can see how it can be confusing when trying to figure out what the value of bad7 is in the elif branch
There was a problem hiding this comment.
Yeah, I think you are right. Due to that I decided to only apply the rule to simple if statements, where simple means without elif or else attached to the if.
|
Thanks for the review @MichaReiser 👍 , just a heads up: I will be travelling without my laptop for the coming 5 weeks, so will only get back to pull request this after. |
|
Hello, is this still being worked on? It is nice check, would be great to have. Especially if possible autofix would amazing. |
… an `elif` or `else` involved
vincevannoort
force-pushed
the
consider-using-assignment-expr
branch
from
04f310d to
d11a686
Compare
|
@MichaReiser sorry for getting back so late to this PR, and thanks for all the previous input. I have rebased to the latest changes on main, and fixed some issues mentioned in earlier comments. Additionally, I changed the rule to only apply to simple Coul you take another look? |
| match &*stmt.test { | ||
| // case - bool operations (`if test1 and test2:`) | ||
| Expr::BoolOp(expr) => diagnostics.extend(expr.values.iter().filter_map(|bool_test| { | ||
| Some(create_diagnostic( | ||
| checker, | ||
| find_assignment_before_if_stmt(&previous_assignment, if_test, bool_test)?, | ||
| )) | ||
| })), | ||
|
|
||
| // case - compare (`if test1 is not None:`) | ||
| Expr::Compare(compare) => { | ||
| if let Some(error) = | ||
| find_assignment_before_if_stmt(&previous_assignment, if_test, &compare.left) | ||
| { | ||
| diagnostics.push(create_diagnostic(checker, error)); | ||
| }; | ||
| } | ||
|
|
||
| _ => {} | ||
| } |
There was a problem hiding this comment.
I am curious if we should actually do this part. If I look at the ecosystem checks, these also produce some possible controversial results. 🤔
Without it, I think most of the more controversial results would disappear.
|
Hmm, I just noticed that this is a pylint extensions rule which we don't want to add right now because it would require a new rule group. |
What would you recommend in this case? Just close the pull request or make it a new ruff rule instead of pylint rule? |
|
Yeah, I think we have to close this for now because there's no clear path to landing this PR before #1774 is complete. I'm very sorry for wasting your time here. I should have noticed this way sooner. But thank you anyway and we can come back to this PR once we're at a point where we accept pylint extension rules. |
No worries 😄 , I still learned quite a lot from the implementation so it was still worth it for me personally. Thanks for the effort in the feedback rounds. 👍 |
4 participants
Summary
This pull request implements the
R6103pylint rule: pylint documentation.It checks assignments which are directly followed by if statements using that expression:
And suggest to use the
:=operator toTest Plan
I have added test cases, and checked some of the
ruff ecosystemresults.