Skip to content

Consider submodule imports to detect unused imports#5011

Closed
dhruvmanila wants to merge 3 commits intomainfrom
dhruv/unused-imports
Closed

Consider submodule imports to detect unused imports#5011
dhruvmanila wants to merge 3 commits intomainfrom
dhruv/unused-imports

Conversation

@dhruvmanila
Copy link
Member

@dhruvmanila dhruvmanila commented Jun 11, 2023

Summary

Take the below code as an example:

import foo.bar
import foo.some

Here, a single binding is created (foo) for both imports and the binding created for foo.bar will shadow the one created for foo.some. Now, if the binding foo is unused, then it means all of the bindings shadowing this is unused as well.

There's a conflict which I've commented down below. This is resulting in 2 test failures.

Test Plan

cargo test

fixes: #60
fixes: #4655

@dhruvmanila
Copy link
Member Author

Current dependencies on/for this PR:

This comment was auto-generated by Graphite.

@dhruvmanila dhruvmanila force-pushed the dhruv/unused-imports branch from 16e44fb to 059ba0c Compare June 11, 2023 11:32
@github-actions
Copy link
Contributor

github-actions bot commented Jun 11, 2023

PR Check Results

Benchmark

Linux

group                                      main                                   pr
-----                                      ----                                   --
formatter/large/dataset.py                 1.00      8.4±0.03ms     4.9 MB/sec    1.00      8.4±0.03ms     4.9 MB/sec
formatter/numpy/ctypeslib.py               1.00   1875.6±3.24µs     8.9 MB/sec    1.00   1876.4±7.05µs     8.9 MB/sec
formatter/numpy/globals.py                 1.00    203.5±0.32µs    14.5 MB/sec    1.01    204.5±1.50µs    14.4 MB/sec
formatter/pydantic/types.py                1.00      4.2±0.01ms     6.1 MB/sec    1.00      4.1±0.01ms     6.2 MB/sec
linter/all-rules/large/dataset.py          1.00     14.3±0.05ms     2.9 MB/sec    1.01     14.5±0.08ms     2.8 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.00      3.6±0.01ms     4.7 MB/sec    1.00      3.6±0.01ms     4.7 MB/sec
linter/all-rules/numpy/globals.py          1.00    382.5±1.54µs     7.7 MB/sec    1.04    396.5±1.06µs     7.4 MB/sec
linter/all-rules/pydantic/types.py         1.00      6.4±0.04ms     4.0 MB/sec    1.01      6.4±0.04ms     4.0 MB/sec
linter/default-rules/large/dataset.py      1.00      7.2±0.03ms     5.7 MB/sec    1.02      7.3±0.05ms     5.6 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.00   1471.4±2.89µs    11.3 MB/sec    1.02   1504.0±2.00µs    11.1 MB/sec
linter/default-rules/numpy/globals.py      1.00    157.3±0.25µs    18.8 MB/sec    1.06    167.4±0.29µs    17.6 MB/sec
linter/default-rules/pydantic/types.py     1.00      3.2±0.01ms     8.0 MB/sec    1.01      3.2±0.01ms     7.9 MB/sec

Windows

group                                      main                                   pr
-----                                      ----                                   --
formatter/large/dataset.py                 1.03     13.7±0.51ms     3.0 MB/sec    1.00     13.3±0.54ms     3.1 MB/sec
formatter/numpy/ctypeslib.py               1.00      2.8±0.10ms     5.9 MB/sec    1.04      2.9±0.14ms     5.7 MB/sec
formatter/numpy/globals.py                 1.00   322.4±14.10µs     9.2 MB/sec    1.06   342.1±29.46µs     8.6 MB/sec
formatter/pydantic/types.py                1.01      6.4±0.29ms     4.0 MB/sec    1.00      6.3±0.23ms     4.1 MB/sec
linter/all-rules/large/dataset.py          1.02     23.0±0.38ms  1807.6 KB/sec    1.00     22.6±0.64ms  1843.7 KB/sec
linter/all-rules/numpy/ctypeslib.py        1.00      5.7±0.15ms     2.9 MB/sec    1.07      6.1±0.22ms     2.7 MB/sec
linter/all-rules/numpy/globals.py          1.00   677.1±28.07µs     4.4 MB/sec    1.03   699.2±21.19µs     4.2 MB/sec
linter/all-rules/pydantic/types.py         1.03     10.7±0.46ms     2.4 MB/sec    1.00     10.4±0.32ms     2.4 MB/sec
linter/default-rules/large/dataset.py      1.03     12.9±0.36ms     3.2 MB/sec    1.00     12.5±0.47ms     3.3 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.10      2.6±0.08ms     6.4 MB/sec    1.00      2.4±0.17ms     7.0 MB/sec
linter/default-rules/numpy/globals.py      1.00   296.1±11.08µs    10.0 MB/sec    1.00   296.4±11.23µs    10.0 MB/sec
linter/default-rules/pydantic/types.py     1.11      5.5±0.18ms     4.6 MB/sec    1.00      5.0±0.17ms     5.1 MB/sec

@dhruvmanila
Copy link
Member Author

So, it seems that there are some conflicting behavior between flake8 and pylint (we've adopted flake8 behvaior):

import foo
import foo.bar

flake8 will only trigger a violation of unused import on line 2 while pylint triggers for both lines. It's the same for the below code as well:

import foo
import bar as foo

I would argue to actually mark both as unused import similar to pylint. When called with the --fix flag, both these imports are removed even though the diagnostics suggest that the first import isn't unused:

$ ruff check --no-cache --isolated --select=F401,F811 src/F401.py
src/F401.py:2:8: F811 Redefinition of unused `foo` from line 15
src/F401.py:2:8: F401 [*] `foo` imported but unused
Found 2 errors.
[*] 1 potentially fixable with the --fix option.

@dhruvmanila dhruvmanila marked this pull request as ready for review June 11, 2023 12:12
@charliermarsh
Copy link
Member

Hmm, I'm not totally certain on the right behavior here. Given the example from #60

import multiprocessing.pool
import multiprocessing.process

z = multiprocessing.pool.ThreadPool()

Wouldn't we end up flagging multiprocessing.pool as unused? It feels unintuitive, though may be correct.

@dhruvmanila
Copy link
Member Author

dhruvmanila commented Jul 13, 2023

Wouldn't we end up flagging multiprocessing.pool as unused?

@charliermarsh Do you mean multiprocessing.process as multiprocessing.pool is being used in your code.

But to answer your question, no it won't flag them. I've added a test case for the mentioned case: https://github.com/astral-sh/ruff/pull/5011/files#diff-cdbee39a5dfd985e12efeff0edaae1b8f79c8fdc2655e64ad12160cf84659930

(Sorry for the delay, I missed this in the notifications)

Comment on lines +30 to +32
/// A map from binding ID to bound name.
binding_name: FxHashMap<BindingId, &'a str>,

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hash maps are kind of expensive (48 bytes or something in size and require an allocation per scope). Could we instead move the name to Binding, recompute it on demand (implement a helper on binding that uses BindingId to extract the name, I would prefer this approach), or store it as a another IndexVec<BindingId, &'str> on the semantic model (since we need an entry for every name).

@dhruvmanila
Copy link
Member Author

@charliermarsh Any thoughts on how to proceed here? I'm totally fine with rejecting this solution altogether :)

@dhruvmanila dhruvmanila requested review from charliermarsh and removed request for charliermarsh July 18, 2023 17:31
@charliermarsh
Copy link
Member

@dhruvmanila - Sorry for the delay (again), but let's close this out for now. I appreciate the work but submodule imports are complicated and I'm having trouble reasoning about the change, whether it's correct, and how it should be reflected in other rules (no fault of yours -- it just requires some investigation on my part, and I don't feel it's worth investing time in right now).

@dhruvmanila
Copy link
Member Author

Hey, I totally understand 👍

dylwil3 added a commit that referenced this pull request Sep 26, 2025
…ed-import` (`F401`) (#20200)

# Summary

The PR under review attempts to make progress towards the age-old
problem of submodule imports, specifically with regards to their
treatment by the rule [`unused-import`
(`F401`)](https://docs.astral.sh/ruff/rules/unused-import/).

Some related issues:
- #60
- #4656

Prior art:
- #13965
- #5010
- #5011
- #666

See the PR summary for a detailed description.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RUF100 and E401 with auto-fix seem inconsistent Handle multiple unused submodule imports

3 participants