Repository navigation
Add a type stub file for dynamically generated unit objects in astropy.units #15808
Description
What is the problem this feature will solve?
While Astropy does not yet make extensive use of type hint annotations, there has been a great discussion in #15170 about implementing them. However, it is difficult to add type hints for dynamically generated objects, such as most of the units in astropy.units. 🤔
Adding type hint annotations for units would benefit downstream packages. For example, PlasmaPy makes heavy use of astropy.units, and the lack of type hint annotations for individual unit objects limits the extent to which mypy can do static type checking. For example, mypy isn't able to follow how types change when doing an operation like 88 * u.imperial.mile 🚗. (It is possible to configure mypy to ignore missing imports, though.)
Additionally, I've had trouble with IDEs not recognizing objects like u.km when they attempt to do smart highlighting, most likely because they are based on static type checking.
Describe the desired outcome
The possibilities that I can think of are that we could:
- Add a type stub file that contains the types of all of the units in
astropy.units. - Create a type stub package outside of this repository that could include these types.
Ideally it would be possible to automagically regenerate the type stub file based on the contents of astropy.units. That way, when new units are added, we can make sure that they get added to the .pyi file. In this comment, @mhvk made the important point that:
we do not want to run this on every import, but just on installation.
Additional context
I've been trying to find tools that can help with generating type stub files at runtime, like MonkeyType. However, I haven't been able to get any to work. 😿
Thank you!
Activity
neutrinoceros commentedon Jan 4, 2024
I think a generated stub file is the only way to go about it, and indeed we'd need to automate its (re)generation since, by construction, it'd be very hard (impossible) to detect out-of-sync errors statically. However, I'm not sure how common of a problem this is in the community so I wouldn't bet that there are existing tools to solve it. I think it'd be good to first estimate how difficult a task it'd be to just write a script + GitHub action ourselves.
namurphy commentedon Jan 4, 2024
Thank you for your response and insights!
I don't think it'd be too bad to write a Python script to create a
.pyifile for the dynamic content inastropy.units...maybe ∼1–3 days? A large portion of that time would be trying to understand the format and usage of.pyifiles and nuances ofastropy.units, for someone unfamiliar with them.A possibility would be to do this in two stages.
.pyifile and add it to the repo based on what's currently inastropy.units. If we use mypy in CI, then we would probably be able to check if there are errors with the static content, at least. Sinceastropy.unitsis relatively stable, this would probably suffice for most use cases...but this would be difficult to maintain..pyigets regenerated during package installation, or create a GitHub Action that would handle the updates.mhvk commentedon Jan 4, 2024
Could one do it simpler and tell the type checking stuff that anything in
unitsnot covered by an explicit rule is aUnitBaseinstance? I realize I'm actually a bit confused about the exact problem. Don't even static type checkers have to import things? Or do they look at .pyc files?namurphy commentedon Jan 4, 2024
Static type checkers don't run any of the code that they're checking, but rather analyze the source code itself without importing it. Because of that, they're unable to check types for objects that are dynamically created without being explicitly created in the code. Since the
UnitBaseinstances are not directly defined in the source code, mypy unfortunately has no way to know about them.Hm...good question! I don't know of a way to do that without writing a mypy plugin. The simplest way that I know of would be to create a
.pyifile that covers the dynamically created content. I've been wondering if there are tools that can import a package and create a stub file for us. Thestubgentool provided by mypy can create stub files, but not for this sort of dynamically created content.mhvk commentedon Jan 5, 2024
So, effectively the static typers reimplement the python parser?
Anyway, as I noted in #15170 (comment), since we already create
__doc__automatically, it should not be that hard to adjust those functions to also create a stub file (as part of packaging). But "not that hard" probably indeed comes down to 2 or 3 days!namurphy commentedon Jan 12, 2024
If I understand correctly, mypy itself uses the Python parser, which has the consequence that mypy isn't able to run when there's a
SyntaxError. I thinkpyrightuses its own parser so that it can handle files with invalid Python. I'm still learning about this, so please put appropriate error bars around my response! 🙃namurphy commentedon Jan 12, 2024
I wanted to mention an approach that hasn't worked out so far.
MonkeyType "collects runtime types of function arguments and return values, and can automatically generate stub files or even add draft type annotations directly to your Python code based on the types collected at runtime."
I just attempted to run MonkeyType to generate a type stub file for
astropy.units. However, when I didmonkeytype run `which pytest`as per their instructions, it spit outRecursionErrorafterRecursionErrorfor five minutes until I cancelled the process. The errors were in MonkeyType itself, and running pytest without that command didn't lead to any errors. That command worked when I attempted this for PlasmaPy, and I was able to generate type stub files over there, so it seems specific to Astropy.I have no idea why this is happening, but I wanted to mention it in case anyone else tries running MonkeyType on Astropy, since it has the potential to greatly help with transitioning to type hints.
Edited to add: Issue miguelgrinberg/python-engineio#92 points to incorrect monkey patching as a possible source for this. However, the error messages I got didn't point to locations in Astropy's source code, so I still have no idea what's going on.
nstarman commentedon Jan 12, 2024
Coming in a little late to this convo...
I think this is a good idea. We should generate the stub file as part of the CI (and release process). With a script we can do a proper runtime -> static collection, actually importing Astropy and exhaustively searching for units.
namurphy commentedon Jan 18, 2024
I just realized in #15914 that we'll need to do this for
PhysicalTypeinstances too as defined inastropy.units.physical. It'd probably be best as a follow-up PR rather than doing this all at once.Grumblemuffins, I say. Grumblemuffins.
jdtsmith commentedon Jun 28, 2025
Would be really good to improve typing annotations across astropy.
unitsis a typical offender. I've been using the*.pyistubs and they work well, save one issue: LSP servers likepyrightalways prefer stub files over reading the main package code. It does not merge information from both.The usual advice is to create or generate a full set of stub files for all methods etc.
mikez commentedon Aug 24, 2025
Working with astropy, I auto-generated stubs for units. Contributing back here.
My focus was on getting the bulk of the work done.
Advice wanted on best way to trigger this, see PR.
neutrinoceros commentedon Oct 2, 2025
@eerovaher @mhvk I think we can close this as completed now ?
mhvk commentedon Oct 2, 2025
Yes! Closing.