Skip to content

Comments

Special-case value-expression inference of special form subscriptions#16877

Merged
AlexWaygood merged 1 commit intomainfrom
alex/subscript-inference
Mar 20, 2025
Merged

Special-case value-expression inference of special form subscriptions#16877
AlexWaygood merged 1 commit intomainfrom
alex/subscript-inference

Conversation

@AlexWaygood
Copy link
Member

@AlexWaygood AlexWaygood commented Mar 20, 2025

Summary

Currently for something like X = typing.Tuple[str, str], we infer the value of X as object. That's because Tuple (like many of the symbols in the typing module) is annotated as a _SpecialForm instance in typeshed's stubs:

ruff/crates/red_knot_vendored/vendor/typeshed/stdlib/typing.pyi

Line 215 in 23382f5

Tuple: _SpecialForm

and we don't understand implicit type aliases yet, and the stub for _SpecialForm.__getitem__ says it always returns object:

ruff/crates/red_knot_vendored/vendor/typeshed/stdlib/typing.pyi

Lines 198 to 200 in 23382f5

@final
class _SpecialForm(_Final):
def __getitem__(self, parameters: Any) -> object: ...

We have existing false positives in our test suite due to this:

ruff/crates/red_knot_python_semantic/resources/mdtest/annotations/annotated.md

Lines 76 to 78 in 23382f5

# TODO: False positive
# error: [invalid-base]
class C(Annotated[int, "foo"]): ...

and it's causing many new false positives in #16872, which tries to make our annotation-expression parsing stricter in some ways.

This PR therefore adds some small special casing for KnownInstanceType variants that fallback to _SpecialForm, so that these false positives can be avoided.

Test Plan

Existing mdtest altered.

Cc. @MatthewMckee4

@AlexWaygood AlexWaygood added the ty Multi-file analysis & type inference label Mar 20, 2025
@AlexWaygood AlexWaygood force-pushed the alex/subscript-inference branch from f2b6891 to 37084f0 Compare March 20, 2025 17:41
@github-actions
Copy link
Contributor

github-actions bot commented Mar 20, 2025

mypy_primer results

Changes were detected when running on open source projects 
black (https://github.com/psf/black)
- error[lint:non-subscriptable] /tmp/mypy_primer/projects/black/src/black/trans.py:54:11: Cannot subscript object of type `object` with no `__getitem__` method
- Found 289 diagnostics
+ Found 288 diagnostics

@AlexWaygood
Copy link
Member Author

AlexWaygood commented Mar 20, 2025 •
edited
Loading

mypy_primer results

Changes were detected when running on open source projects

black (https://github.com/psf/black)
- error[lint:non-subscriptable] /tmp/mypy_primer/projects/black/src/black/trans.py:54:11: Cannot subscript object of type `object` with no `__getitem__` method
- Found 289 diagnostics
+ Found 288 diagnostics

Here's the link to the line in question: https://github.com/psf/black/blob/dd278cb316d75868716a0478c35b1fcd600a5249/src/black/trans.py#L54

Result is a type alias to a Union[] subscription, and Union is stubbed as an instance of _SpecialForm in typeshed:

https://github.com/psf/black/blob/dd278cb316d75868716a0478c35b1fcd600a5249/src/black/rusty.py#L28

So it's good that we no longer emit this error!

@AlexWaygood AlexWaygood merged commit 296d67a into main Mar 20, 2025
23 checks passed
@AlexWaygood AlexWaygood deleted the alex/subscript-inference branch March 20, 2025 21:46
dcreager added a commit that referenced this pull request Mar 21, 2025
@dcreager
Merge branch 'main' into dcreager/two-phase-binding
01a5b3a 
* main: (26 commits)
  Use the common `OperatorPrecedence` for the parser (#16747)
  [red-knot] Check subtype relation between callable types (#16804)
  [red-knot] Check whether two callable types are equivalent (#16698)
  [red-knot] Ban most `Type::Instance` types in type expressions (#16872)
  Special-case value-expression inference of special form subscriptions (#16877)
  [syntax-errors] Fix star annotation before Python 3.11 (#16878)
  Recognize `SyntaxError:` as an error code for ecosystem checks (#16879)
  [red-knot] add test cases result in false positive errors (#16856)
  Bump 0.11.1 (#16871)
  Allow discovery of venv in VIRTUAL_ENV env variable (#16853)
  Split git pathspecs in change determination onto separate lines (#16869)
  Use the correct base commit for change determination (#16857)
  Separate `BitXorOr` into `BitXor` and `BitOr` precedence (#16844)
  Server: Allow `FixAll` action in presence of version-specific syntax errors (#16848)
  [`refurb`] Fix starred expressions fix (`FURB161`) (#16550)
  [`flake8-executable`] Add pytest and uv run to help message for `shebang-missing-python` (`EXE003`) (#16855)
  Show more precise messages in invalid type expressions (#16850)
  [`flake8-executables`] Allow `uv run` in shebang line for `shebang-missing-python` (`EXE003`) (#16849)
  Add `--exit-non-zero-on-format` (#16009)
  [red-knot] Ban list literals in most contexts in type expressions (#16847)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@carljm carljm carljm approved these changes

@sharkdp sharkdp Awaiting requested review from sharkdp

@dcreager dcreager Awaiting requested review from dcreager

Assignees

No one assigned

Labels

ty Multi-file analysis & type inference

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@AlexWaygood @carljm