🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/166883

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit b36f675 with merge base aa4a8c9 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

I am still of the opinion we should run mypy alongside pyrefly. Many downstream projects use mypy, as it is the defacto default, and I would not want to break typing for those projects. Furthermore, the type ignore between mypy and pyrefly are different and we may trigger mypy errors in public interfaces that pyrefly does not detect.

Do we have coverage tests comparing the behavior of mypy and pyrefly currently?

hi - @Skylion007 running two type checkers wasn't a great experience. I provided links to a pyrefly vs. mypy comparison in response to your question here: #166197

There are many type checkers currently on the market, and all follow the same typing specification. So, having a downstream project use a different type checker is not an uncommon situation. https://typing.python.org/en/latest/spec/

If you could provide specific examples of where you might think this could pose an issue, I'd be happy to look into them.

hi - @Skylion007 running two type checkers wasn't a great experience. I provided links to a pyrefly vs. mypy comparison in response to your question here: #166197

There are many type checkers currently on the market, and all follow the same typing specification. So, having a downstream project use a different type checker is not an uncommon situation. https://typing.python.org/en/latest/spec/

If you could provide specific examples of where you might think this could pose an issue, I'd be happy to look into them.

Saw this timely post while scrolling reddit about how folks actually currently use ty/pyrefly:

https://www.reddit.com/r/Python/comments/1oncd2l/comment/nmwx5by/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

The pattern I've been using is combining Pyrefly and mypy. Pyrefly is great for catching most common issues quickly during development since it's fast, and then I run mypy afterward to make everything more robust and catch whatever Pyrefly might have missed. I also really like how Pyrefly can suggest types automatically. After trying a few versions of Pyrefly, I found that 0.35 works best for this setup.

Specific usecase is that we could accidentally break downstream project's typing through missing type: ignore. We should support the type checker with the weakest/less aggressive inference on untyped code, and that is definitely not pyrefly. Also according to https://engineering.fb.com/2024/12/09/developer-tools/typed-python-2024-survey-meta/?utm_source=chatgpt.com, mypy has >2x the marketshare of pyright, with 64% of developers having used mypy and the remaining 36% not using type checkers frequently.

@pytorchmergebot merge

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

@Skylion007

Thanks for the additional details!

Specific usecase is that we could accidentally break downstream project's typing through missing type: ignore.

Great point, thanks for elaborating. PyTorch should have an automated process to ensure that our types are usable downstream, and that we should be sure to do that for mypy, pyright, ty, pyrefly, zuban, etc. There is currently some ongoing work around this issue (#163990).

Some projects, like Pydantic, use a single type checker internally (in their case, pyright) while validating user code against multiple type checkers in the test suite. This approach helps catch discrepancies and ensures broader compatibility for library users.

We are also working on adding linters and other tools to make sure type coverage within the code base stays high: https://github.com/pytorch/pytorch/pull/167075/files. Although we’re quite proud of our inference capabilities, we know type annotations can make it easier for different type checkers to check the same code more consistently.

Does this work capture your concerns? Is there anything else you think might mitigate this issue?

Saw this timely post while scrolling reddit about how folks actually currently use ty/pyrefly:

Thanks for the link! It’s true that sometimes tools use more than one typechecker. Partially because MyPy does not have a language server that provides features like go-to-def, hover, etc. Most often you will see a project using MyPy (for type checking) and Pyright (for LSP).

The existing setup for MyPy in PyTorch was causing a poor developer experience. It took a long time to run, and dmypy often crashed. There were multiple configurations that made it tricky to figure out if a file was being included in a type check run and if so, at what level of strictness. The performance issues meant that locally, MyPy was only type checking files you changed, meaning on CI runs, MyPy errors would often show up unexpectedly.

Pyrefly is much faster than MyPy, which allows us to use one configuration file and type check the entire codebase locally. We also have a complete IDE, so the types you see on hover, go to def, etc. match those on the CLI. Our team is actively working on developing and adding additional IDE features, and continuing to improve the developer experience.

As for the market share, you're right that MyPy holds the bulk of the market share. It’s been the go-to type checker, and in some ways the only OSS type checker for over a decade. There are several reasons for this. Here’s an example:

The reason we're stuck on mypy at work is because it's the only type checker that has a plugin for Django
(https://news.ycombinator.com/item?id=44014674)

We’ve recently added support for this and are actively engaging with the community to solve typing issues and design solutions for popular. libraries. Similarly, we’re prepared to help with PyTorch typing specific needs that have gone unaddressed for a while.

Is there any other typing work you would like to see someone work on in the PyTorch codebase?