[ty] Support declaration-only attributes by iyakushev · Pull Request #19048 · astral-sh/ruff

iyakushev · 2025-06-30T15:42:53Z

Summary

Following ty issue #698 this PR adds support for declarations.
I'd say the implementation is rather naive, so I would use it as a starting point for a gradual improvement. I tried to check declarations before bindings and it seemed to resolve the issue.

closes #698

Test Plan

Tested against mdtest (specifically attributes).

github-actions · 2025-06-30T16:04:26Z

`mypy_primer` results

Changes were detected when running on open source projects

trio (https://github.com/python-trio/trio)
- error[unresolved-attribute] src/trio/_tests/test_ssl.py:106:8: Type `FixtureRequest` has no attribute `param`
- error[unresolved-attribute] src/trio/_tests/test_ssl.py:108:10: Type `FixtureRequest` has no attribute `param`
- Found 801 diagnostics
+ Found 799 diagnostics

kornia (https://github.com/kornia/kornia)
- error[unresolved-attribute] kornia/nerf/samplers.py:230:13: Unresolved attribute `_x` on type `Points2D_FlatTensors`.
- error[unresolved-attribute] kornia/nerf/samplers.py:231:13: Unresolved attribute `_y` on type `Points2D_FlatTensors`.
- error[unresolved-attribute] kornia/nerf/samplers.py:233:13: Unresolved attribute `_x` on type `Points2D_FlatTensors`.
- error[unresolved-attribute] kornia/nerf/samplers.py:233:57: Type `Points2D_FlatTensors` has no attribute `_x`
- error[unresolved-attribute] kornia/nerf/samplers.py:234:13: Unresolved attribute `_y` on type `Points2D_FlatTensors`.
- error[unresolved-attribute] kornia/nerf/samplers.py:234:57: Type `Points2D_FlatTensors` has no attribute `_y`
- error[unresolved-attribute] kornia/nerf/samplers.py:259:30: Type `Points2D_FlatTensors` has no attribute `_x`
- error[unresolved-attribute] kornia/nerf/samplers.py:259:58: Type `Points2D_FlatTensors` has no attribute `_y`
- Found 800 diagnostics
+ Found 792 diagnostics

hydra-zen (https://github.com/mit-ll-responsible-ai/hydra-zen)
- TOTAL MEMORY USAGE: ~88MB
+ TOTAL MEMORY USAGE: ~97MB

yarl (https://github.com/aio-libs/yarl)
- error[unresolved-attribute] tests/test_quoting.py:18:16: Type `FixtureRequest` has no attribute `param`
- error[unresolved-attribute] tests/test_quoting.py:22:16: Type `FixtureRequest` has no attribute `param`
+ warning[unused-ignore-comment] tests/test_quoting.py:32:31: Unused blanket `type: ignore` directive
+ warning[unused-ignore-comment] tests/test_quoting.py:36:31: Unused blanket `type: ignore` directive

flake8 (https://github.com/pycqa/flake8)
-     memo fields = ~66MB
+     memo fields = ~60MB

pylox (https://github.com/sco1/pylox)
-     memo fields = ~54MB
+     memo fields = ~49MB

websockets (https://github.com/aaugustin/websockets)
- error[unresolved-attribute] src/websockets/asyncio/router.py:89:13: Unresolved attribute `handler` on type `ServerConnection`.
- error[unresolved-attribute] src/websockets/asyncio/router.py:89:33: Unresolved attribute `handler_kwargs` on type `ServerConnection`.
- error[unresolved-attribute] src/websockets/asyncio/router.py:94:26: Type `ServerConnection` has no attribute `handler`
- error[unresolved-attribute] src/websockets/asyncio/router.py:94:59: Type `ServerConnection` has no attribute `handler_kwargs`
- error[unresolved-attribute] src/websockets/asyncio/server.py:980:9: Unresolved attribute `username` on type `ServerConnection`.
- error[unresolved-attribute] src/websockets/sync/router.py:89:13: Unresolved attribute `handler` on type `ServerConnection`.
- error[unresolved-attribute] src/websockets/sync/router.py:89:33: Unresolved attribute `handler_kwargs` on type `ServerConnection`.
- error[unresolved-attribute] src/websockets/sync/router.py:94:20: Type `ServerConnection` has no attribute `handler`
- error[unresolved-attribute] src/websockets/sync/router.py:94:53: Type `ServerConnection` has no attribute `handler_kwargs`
- error[unresolved-attribute] src/websockets/sync/server.py:762:9: Unresolved attribute `username` on type `ServerConnection`.
- Found 65 diagnostics
+ Found 55 diagnostics

poetry (https://github.com/python-poetry/poetry)
- error[unresolved-attribute] tests/console/commands/test_init.py:316:31: Type `FixtureRequest` has no attribute `param`
- error[unresolved-attribute] tests/installation/test_installer.py:117:48: Type `FixtureRequest` has no attribute `param`
- error[unresolved-attribute] tests/installation/test_installer.py:118:11: Type `FixtureRequest` has no attribute `param`
- Found 939 diagnostics
+ Found 936 diagnostics

schemathesis (https://github.com/schemathesis/schemathesis)
-     memo fields = ~129MB
+     memo fields = ~142MB

typeshed-stats (https://github.com/AlexWaygood/typeshed-stats)
- error[unresolved-attribute] tests/conftest.py:283:12: Type `FixtureRequest` has no attribute `param`
+ warning[unused-ignore-comment] tests/test___all__.py:26:27: Unused blanket `type: ignore` directive
- error[unresolved-attribute] tests/test__cli.py:204:20: Type `FixtureRequest` has no attribute `param`
- error[unresolved-attribute] tests/test_gather.py:174:24: Type `FixtureRequest` has no attribute `param`
- Found 26 diagnostics
+ Found 24 diagnostics

urllib3 (https://github.com/urllib3/urllib3)
- error[unresolved-attribute] src/urllib3/connectionpool.py:553:13: Type `BaseHTTPResponse` has no attribute `length_remaining`
- error[unresolved-attribute] test/conftest.py:289:28: Type `FixtureRequest` has no attribute `param`
- error[unresolved-attribute] test/conftest.py:373:8: Type `FixtureRequest` has no attribute `param`
- error[unresolved-attribute] test/conftest.py:383:15: Type `FixtureRequest` has no attribute `param`
- error[unresolved-attribute] test/conftest.py:385:12: Type `FixtureRequest` has no attribute `param`
- Found 438 diagnostics
+ Found 433 diagnostics

scrapy (https://github.com/scrapy/scrapy)
-     memo fields = ~189MB
+     memo fields = ~207MB

psycopg (https://github.com/psycopg/psycopg)
- error[unresolved-attribute] psycopg_pool/psycopg_pool/base.py:201:9: Unresolved attribute `_expire_at` on type `BaseConnection[Any]`.
- Found 562 diagnostics
+ Found 561 diagnostics

pytest (https://github.com/pytest-dev/pytest)
- error[unresolved-attribute] src/_pytest/python.py:1110:12: Type `FixtureRequest` has no attribute `param`
- error[unresolved-attribute] testing/test_pathlib.py:120:17: Type `FixtureRequest` has no attribute `param`
- Found 527 diagnostics
+ Found 525 diagnostics

altair (https://github.com/vega/altair)
-     memo fields = ~228MB
+     memo fields = ~251MB

rotki (https://github.com/rotki/rotki)
- error[no-matching-overload] rotkehlchen/tests/api/test_eth2.py:302:9: No overload of function `get_decoded_events_of_transaction` matches arguments
- error[no-matching-overload] rotkehlchen/tests/api/test_eth2.py:1091:12: No overload of function `get_decoded_events_of_transaction` matches arguments
- error[no-matching-overload] rotkehlchen/tests/api/test_eth2.py:1382:5: No overload of function `get_decoded_events_of_transaction` matches arguments
- error[no-matching-overload] rotkehlchen/tests/api/test_ethereum_transactions.py:1422:5: No overload of function `get_decoded_events_of_transaction` matches arguments
- error[no-matching-overload] rotkehlchen/tests/api/test_ethereum_transactions.py:1469:5: No overload of function `get_decoded_events_of_transaction` matches arguments
- error[no-matching-overload] rotkehlchen/tests/api/test_ethereum_transactions.py:1528:9: No overload of function `get_decoded_events_of_transaction` matches arguments
- error[no-matching-overload] rotkehlchen/tests/api/test_sushiswap.py:59:5: No overload of function `get_decoded_events_of_transaction` matches arguments
- error[no-matching-overload] rotkehlchen/tests/api/test_uniswap.py:72:5: No overload of function `get_decoded_events_of_transaction` matches arguments
- error[no-matching-overload] rotkehlchen/tests/data_migrations/test_migrations.py:550:9: No overload of function `get_decoded_events_of_transaction` matches arguments
- warning[possibly-unbound-attribute] rotkehlchen/tests/integration/test_eth2.py:414:96: Attribute `query_withdrawals` on type `Unknown | Blockscout | None` is possibly unbound
- error[no-matching-overload] rotkehlchen/tests/unit/decoders/test_degen.py:33:17: No overload of function `get_decoded_events_of_transaction` matches arguments
- error[no-matching-overload] rotkehlchen/tests/unit/decoders/test_degen.py:78:17: No overload of function `get_decoded_events_of_transaction` matches arguments
- error[no-matching-overload] rotkehlchen/tests/unit/test_evm_balances.py:43:5: No overload of function `get_decoded_events_of_transaction` matches arguments
- Found 1728 diagnostics
+ Found 1715 diagnostics

scikit-learn (https://github.com/scikit-learn/scikit-learn)
-     memo fields = ~539MB
+     memo fields = ~593MB

scipy (https://github.com/scipy/scipy)
- error[unresolved-attribute] scipy/_lib/array_api_extra/tests/conftest.py:33:26: Type `FixtureRequest` has no attribute `param`
- error[unresolved-attribute] scipy/_lib/array_api_extra/tests/conftest.py:205:16: Type `FixtureRequest` has no attribute `param`
- error[unresolved-attribute] scipy/_lib/array_api_extra/tests/conftest.py:215:18: Type `FixtureRequest` has no attribute `param`
- error[unresolved-attribute] scipy/conftest.py:443:10: Type `FixtureRequest` has no attribute `param`
- Found 6624 diagnostics
+ Found 6620 diagnostics
- TOTAL MEMORY USAGE: ~1271MB
+ TOTAL MEMORY USAGE: ~1399MB

sharkdp · 2025-06-30T21:06:41Z

Thank you for working on this! I am planning to take a closer look tomorrow.

It looks like this leads to a lot new false positives on ecosystem projects (see the auto-generated mypy_primer comment). We should probably try to find out what causes them (+ fix them and write a few more test cases here).

iyakushev · 2025-07-01T09:01:13Z

Thank you @sharkdp! I will try to add some test cases following mypy_primer results tomorrow. Is it okay to add them into mdtests?
As for false positives, I think its due to this line. I mostly copied the way bindings did it while getting familiar with the codebase.

sharkdp · 2025-07-01T09:31:09Z

Thank you @sharkdp! I will try to add some test cases following mypy_primer results tomorrow. Is it okay to add them into mdtests?

Yes. Feel free to add some new sections in https://github.com/astral-sh/ruff/blob/main/crates/ty_python_semantic/resources/mdtest/attributes.md

As for false positives, I think its due to this line. I mostly copied the way bindings did it while getting familiar with the codebase.

That particular part will change with #18955 where we basically decide not to track possible unboundness (and with your PR: undeclaredness) anymore for implicit instance attributes. You could maybe try rebasing your branch on top of that already.

iyakushev · 2025-07-01T17:45:15Z

@sharkdp Please take a look when available. It seems to me that most of the errors (false positives) in the original implementation come from class instance access to its attributes via self if they have some form of annotation.

class C:
    def __init__(self):
        self.foo: list[int] = []
        
    def bar(self):
        print(self.foo)  # Previously possibly-unbound

I've added a few mdtest to validate this (and marked them with TODO). There's definitely something wrong; either its beyond the original scope of the issue, or I'm misusing something. Either way, your guidance would be very valuable 😃
I've changed implementation as well, it only checks declarations at the very end if we have no defined bindings.

sharkdp · 2025-07-02T15:34:42Z

Please take a look when available. It seems to me that most of the errors (false positives) in the original implementation come from class instance access to its attributes via self if they have some form of annotation

I've added a few mdtest to validate this (and marked them with TODO).

I don't understand? Non of the new MD tests seems to show some kind of possibly-unbound false positive?

I've changed implementation as well, it only checks declarations at the very end if we have no defined bindings.

I think we should switch that and check declarations first. If we have a declaration, we should prefer it over any bindings.

iyakushev · 2025-07-03T18:38:51Z

Non of the new MD tests seems to show some kind of possibly-unbound false positive?

Sorry for the ambiguity in my previous message. I've tried to replicate the warnings I saw in mypy_primer repos, but after I've pulled the new changes (The PR you mentioned + main), the issue seems to be resolved. I've double checked against the chess repo snippet from primer, but I did not contribute it to the mdtest. The changes in the mdtest are for broader testing.

I've moved the declaration check above bindings and fixed another potential issue with Class obj/instance access to attributes. In my previous attempt instance attributes were leaking into class objects even if they were declared only in __init__.

Could we, perhaps, run mypy_primer once again?

codspeed-hq · 2025-07-04T09:59:29Z

CodSpeed WallTime Performance Report

Merging #19048 will not alter performance

_{Comparing iyakushev:support-decl-only (a6206f1) with main (b6edfbc)}

Summary

✅ 7 untouched benchmarks

crates/ty_python_semantic/resources/mdtest/attributes.md

crates/ty_python_semantic/src/types/class.rs

sharkdp

Thank you very much. This looks great. Just a few minor comments.

iyakushev · 2025-07-07T09:41:09Z

Thank you for taking time to review this!
I've pushed the changes addressing your comments. As for duplication, I've moved the code into a closure, if that's ok. The name might be a little off imho.

sharkdp

This looks great — thank you very much!

…c_tokens * 'main' of https://github.com/astral-sh/ruff: (27 commits) [ty] First cut at semantic token provider (astral-sh#19108) [`flake8-simplify`] Make example error out-of-the-box (`SIM116`) (astral-sh#19111) [`flake8-use-pathlib`] Make example error out-of-the-box (`PTH210`) (astral-sh#19189) [`flake8-use-pathlib`] Add autofixes for `PTH203`, `PTH204`, `PTH205` (astral-sh#18922) [`flake8-type-checking`] Fix syntax error introduced by fix (`TC008`) (astral-sh#19150) [`flake8-pyi`] Make example error out-of-the-box (`PYI007`, `PYI008`) (astral-sh#19103) Update Rust crate indicatif to 0.18.0 (astral-sh#19165) [ty] Add separate CI job for memory usage stats (astral-sh#19134) [ty] Add documentation for server traits (astral-sh#19137) Rename to `SessionSnapshot`, move unwind assertion closer (astral-sh#19177) [`flake8-type-checking`] Make example error out-of-the-box (`TC001`) (astral-sh#19151) [ty] Bare `ClassVar` annotations (astral-sh#15768) [ty] Re-enable multithreaded pydantic benchmark (astral-sh#19176) [ty] Implement equivalence for protocols with method members (astral-sh#18659) [ty] Use RHS inferred type for bare `Final` symbols (astral-sh#19142) [ty] Support declaration-only attributes (astral-sh#19048) [ty] Sync vendored typeshed stubs (astral-sh#19174) Update dependency pyodide to ^0.28.0 (astral-sh#19164) Update NPM Development dependencies (astral-sh#19170) Update taiki-e/install-action action to v2.56.7 (astral-sh#19169) ...

iyakushev requested review from AlexWaygood, carljm, dcreager and sharkdp as code owners June 30, 2025 15:42

ntBre added the ty Multi-file analysis & type inference label Jun 30, 2025

AlexWaygood removed their request for review June 30, 2025 16:41

iyakushev requested a review from MichaReiser as a code owner July 1, 2025 16:55

iyakushev marked this pull request as draft July 1, 2025 16:59

[ty] Support declaration only statements

6dac224

iyakushev force-pushed the support-decl-only branch from b5033c4 to 6dac224 Compare July 1, 2025 17:33

Remove else branching to reduce nesting

cee82fa

Check declarations before bindings

48bbab1

iyakushev added 3 commits July 3, 2025 22:39

Fix mdtest

8d1470c

Fix clippy

e6c723b

Fix mdtest comment

4b60a0d

iyakushev changed the title ~~[ty] Support scoped declarations~~ [ty] Support declarations-only attributes Jul 4, 2025

iyakushev changed the title ~~[ty] Support declarations-only attributes~~ [ty] Support declaration-only attributes Jul 4, 2025

Remove unnecessary test

40ac6f7

iyakushev marked this pull request as ready for review July 5, 2025 07:56

sharkdp reviewed Jul 7, 2025

View reviewed changes

crates/ty_python_semantic/resources/mdtest/attributes.md Outdated Show resolved Hide resolved

sharkdp reviewed Jul 7, 2025

View reviewed changes

crates/ty_python_semantic/resources/mdtest/attributes.md Outdated Show resolved Hide resolved

sharkdp reviewed Jul 7, 2025

View reviewed changes

crates/ty_python_semantic/src/types/class.rs Outdated Show resolved Hide resolved

sharkdp requested changes Jul 7, 2025

View reviewed changes

Address code review comments

b69e7bc

iyakushev requested a review from sharkdp July 7, 2025 09:42

sharkdp added the ecosystem-analyzer label Jul 7, 2025

sharkdp added 2 commits July 7, 2025 12:43

Remove unnecessary tests

8433582

Fix clippy suggestion

a6206f1

sharkdp force-pushed the support-decl-only branch from 884b766 to a6206f1 Compare July 7, 2025 10:46

sharkdp approved these changes Jul 7, 2025

View reviewed changes

sharkdp merged commit e0b7f49 into astral-sh:main Jul 7, 2025
35 checks passed

AlexWaygood mentioned this pull request Jul 7, 2025

Detect declared-only instance attributes astral-sh/ty#698

Closed

iyakushev deleted the support-decl-only branch July 7, 2025 14:55

Comments

Conversation

iyakushev commented Jun 30, 2025 • edited by sharkdp Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test Plan

Uh oh!

github-actions bot commented Jun 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

mypy_primer results

Uh oh!

sharkdp commented Jun 30, 2025

Uh oh!

iyakushev commented Jul 1, 2025

Uh oh!

sharkdp commented Jul 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

iyakushev commented Jul 1, 2025

Uh oh!

sharkdp commented Jul 2, 2025

Uh oh!

iyakushev commented Jul 3, 2025

Uh oh!

codspeed-hq bot commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodSpeed WallTime Performance Report

Merging #19048 will not alter performance

Summary

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sharkdp left a comment

Choose a reason for hiding this comment

Uh oh!

iyakushev commented Jul 7, 2025

Uh oh!

sharkdp left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

iyakushev commented Jun 30, 2025 •

edited by sharkdp

Loading

github-actions bot commented Jun 30, 2025 •

edited

Loading

`mypy_primer` results

sharkdp commented Jul 1, 2025 •

edited

Loading

codspeed-hq bot commented Jul 4, 2025 •

edited

Loading