build(deps): typing-extensions as a dependency - #18360

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

• Do the proposed changes actually accomplish desired goals?
• Do the proposed changes follow the Astropy coding guidelines?
• Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
• Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
• Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
• Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
• Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
• Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
• At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

👋 Thank you for your draft pull request! Do you know that you can use [ci skip] or [skip ci] in your commit messages to skip running continuous integration tests until you are ready?

Was this discussed at Astropy Coordination Meeting 2025? Could you please provide more context? Introducing a new required dependency is a big deal and should be well justified. Thanks!

This topic was discussed tangentially at the Astropy Coordination Meeting 2025, in the context of improving our typing infrastructure and ensuring that type annotations don’t interfere with downstream packages. While typing itself is benign, issues arose from how type annotations were not resolvable at runtime.

In particular, Sphinx attempts to resolve annotations during doc build to generate cross-references, and it does so using runtime analysis. Thus is doesn't see if TYPE_CHECKING contents and it has bugs related to __future__ import annotations. The Sphinx issue was diagnosed as Option 2 in #17546 (comment) and led to the work begun in #18191. Since then, to help downstream packages, we’ve been shifting toward making annotations either runtime-accessible or stringified in-place, rather than relying solely on the __future__ behaviour.

This approach has several concrete benefits:

1. It circumvents the bugs in Sphinx’s resolution logic.
2. It improves IDE support (e.g., more accurate autocompletion and inspection).
3. It allows Astropy to be more compatible with runtime type checkers, which depend on type metadata being available at runtime. I'd be happy to chat more about this to show how I use beartype's import hook during code development to prevent bugs from happening.

Achieving this requires that certain typing constructs be available at runtime. Although we’ve historically treated typing_extensions as an on-again-off-again type-only dependency (e.g., #16903), this PR proposes making it a runtime dependency to ensure that annotations like NoDefault (Python 3.13+), updated NamedTuple semantics can be used consistently, etc.

The typing_extensions package is maintained by core Python developers, is lightweight, and tracks the evolution of the standard typing module. It’s effectively part of the ecosystem’s core tooling and highly stable.

FWIW I'd rather prioritize scipy if we must add a new required dependency (#12448).

• Sphinx is no longer a problem for now and will not be if the MyST idea moves forward.
• IDE and third-party type checkers are all "nice to have" and not essential.

This is solely my personal opinion and I am sure we have a lot of those across the spectrum.

Sphinx is no longer a problem for now and will not be if the MyST idea moves forward.

I wish that were entirely the case. We've turned off showing typing in our docs and so have some downstream packages and for these packages nothing can break. But for other downstream packages malformed type annotations, like any broken code, can still poison docs. Thus the ongoing effort (e.g. #18358) to prevent the physical possibility of malformed type annotations. This effort to make Sphinx no longer a problem is made easier with typing_extensions. Not required, but easier.

And "MyST is an extension of CommonMark Markdown, that includes a rich additional syntax for technical authoring, and can integrate with Docutils and Sphinx." — https://myst-parser.readthedocs.io/en/latest/intro.html
So I think we'd still have Sphinx in the loop, just avoid most of its pain points. Depending on what extension we use for autodoc generation, this can remain a problem. We'll have to be careful about our choice of autodoc replacement and we might choose a system that emphasizes other features.

...

mypyc/mypyc@419315a

mypyc supports typing_extensions. For delivering performant compiled Python code that works across Python versions it's probably not a nice-to-have.

Still, doc build and mypyc aren't something end-user need to worry about, so why not optional dev dependency, why required?

What about here?

Still, doc build and mypyc aren't something end-user need to worry about, so why not optional dev dependency, why required?

They don't need to worry about it, but it's there, like the packaging library for making optional dependency management more convenient. In terms of stability, typing_extensions is about the same. Official backport for core python by core python devs.

What is it that you would want to use typing-extensions for?

Unpack & Concatenate, LiteralString, NoDefault when introspecting dataclasses, ParamSpec updates, immutable dict fields via ReadOnly, TypeIs is often better than TypeGuard, Reader and Writer for our universal I/O.

Unpack is available in the standard library already, only the PEP 692 updates require Python 3.12, and it might be simpler to just wait until we drop support for Python 3.11.

Concatenate and LiteralString are already in the Python 3.11 standard library.

NoDefault can't be used in Python 3.12 type parameter lists even with typing_extensions because Python 3.12 does not allow default type values anyways. I'd rather use type parameter lists than NoDefault.

ParamSpec updates do require Python 3.13, but a lot of the functionality is available already.

ReadOnly does not prevent writing at runtime, so while it is nice-to-have, it is not important enough to justify a new required dependency.

TypeIs can indeed be better than TypeGuard for many use cases, but is the difference large enough to justify a new required dependency?

io.Reader and io.Writer (from Python 3.14) seem simple enough that we could backport them ourselves.

I don't see a compelling need to require typing_extensions. A lot can be done without it.

Thanks for the detailed breakdown. I’d argue that the bar for adding this dependency should be almost non-existently small. typing_extensions is an official backport package maintained by core Python developers specifically to support Python’s gradual typing evolution across versions. Relying on it ensures Python’s “batteries-included” philosophy applies to typing — so developers don't need to worry about swapping batteries.

We have packaging as a hard dependency under a similar justification. See how few times it's required at runtime (https://github.com/search?q=repo%3Aastropy%2Fastropy%20packaging&type=code) but no one has an issue with it as a dependency because, like typing_extensions, it's a lightweight pure-python almost-core-python dependency.

I can agree that this wouldn't be controversial if we were type checking, but we are not type checking.

I guess we should start type checking! An allow-list for cosmology would be good. There's plenty of utility files we could start with.

If you are serious about type checking in CI then you should at the very least bring it up in a developer telecon.

Forgot about override in the list of useful features :)
Required for #18802 and for doing this Astropy-wide.

I feel like it's fine to require this package at runtime but only as a backport component to the standard lib and only on older versions of Python. This is the only strategy I feel confortable with to ensure the following conditions

• we know when we can safely drop it
• we know that we aren't using anything experimental within the package (features tend to be developed there before they land in the standard lib)

When I see it installed in a fresh env built against the newest stable Python I regard it as a bug.

I agree the 2nd point is a potential major issue and a python version check should enable maintainers to easily spot it and prevent it from happening.

Quite a few of the features @nstarman has mentioned are in the standard library since Python 3.12, so we will have much less need for typing_extensions when we drop support for Python 3.11.

True, but would it be 0 by then ? Also, and just in case, I don't think this is a compelling argument in favor of dropping 3.11, and I'd still much prefer if we kept support for it for (at least) another year.

@neutrinoceros See How do we prevent use of deferred/rejected PEP imports? in #18360 (comment) for how we can avoid the sys.version checks!
This should largely address #18360 (comment)

@neutrinoceros in 8ed6a30 I think I captured the full set of draft/deferred/withdrawn PEPs.
The only one not in this list is FORMAT, which is in two accepted PEPs but maybe hasn't landed in Python (?). Either way, they promise it's stable.

I'm aware the sys.version_info guard can be avoided and that ruff knows how to upgrade import statements from typing_extensions, but I'm arguing that

• using the guard clauses isn't that big an effort
• the core library should aspire to the highest standards of quality, which IMHO includes avoiding unnecessary runtime dependencies

ruff knows how to upgrade import statements from typing_extensions.

Yes, but that's not what the section How do we prevent use of deferred/rejected PEP imports? I referred to is talking about as a way to avoid sys.version_info.

• the core library should aspire to the highest standards of quality

I agree!

Bug-fix Backports:

• TypedDict — backports Python 3.12 behavior ensuring __orig_bases__ is consistently populated.
• get_type_hints — backports Python 3.12 fix that strips ReadOnly when include_extras=false.
• is_typeddict — backports Python 3.12 fix making is_typeddict(TypedDict) correctly return False.
• Protocol — backports Python 3.12 runtime-checkable fixes.

Performance Backports:

• SupportsIndex — backports the optimized Python 3.12 implementation.
• runtime_checkable — backports Python 3.12 performance improvements for runtime-checkable protocols.
• Protocol — backports Python 3.12 performance improvements

Edit: updated list in #18360 (comment)

But where performance is critical we certainly can guard behind sys.version_info! That would avoid importing from both typing and typing_extensions.
But if we're being pragmatic, this level of performance optimization should be essentially unnecessary in any package that imports numpy (or definitely scipy)!

Try running uv run python -X importtime -c "import typing_extensions; import numpy; import scipy" 2> importlog.txt. I get:

• typing_extensions: 1.8 ms
• numpy: 60.6 ms
• scipy: 250 ms (optional, but very common)

That's just cold-start. And since Python has an import cache it's basically a one-time cost for importing anywhere in astropy.
OTOH, the performance backports affect runtime! All uses of Protocols are faster.

So in a package with both typing_extensions and numpy we can get a ~3% faster cold start (in just the dependency importing, not overall start time) in a trade for wrapping every typing_extensions import in a sys.version_info and having a slower run-time.

Ok in hindsight, that looks more reasonable to me now that you've laid out 2 things I wasn't aware of. Critically:

• typing_extensions has bug fixes backports
• it looks to me that the proposal is, as it stands, only about improving support for Python 3.11, and that we should be able drop the dependency entirely when we require 3.12, is this correct ? Of course, I anticipate that further development may sediment our use of the package for more recent versions, but I'm willing to live with it (we have bigger fish to fry when it comes to unneeded dependencies, and I really to want to impede progress on supporting type checking in astropy)

Of course, I anticipate that further development may sediment our use of the package for more recent versions.

I expect similarly. TypeForm (py3.15), in particular seems fundamental to supporting the array api.
There definitely will reach a point where typing is fully mature in Python and we can drop this dependency, but it'll probably be helpful until then.

Adding a mandatory dependency is a big deal.

In the general case I very much agree, but this exact package

• is pure python (no portability issue)
• is written and maintained by core CPython devs (so it's reliable)
• is extremely small (wheels are <50KiB)
• has no dependencies of its own
• is nearly ubuquitous
• used purely a backport package, as is proposed here, it'll go away at some point

for all these reasons, I don't think it should be treated as a big deal.

"no big deal" is not a precedent we should set. Please discuss with community. We have time as this is still very early in the release cycle. Thanks!

• astropy-dev
• Slack channel #general

Sure, I'm not brushing off the need for discussion, I'm just saying I feel confident about this one not meeting controversy.

Notes on https://peps.python.org/pep-0747/ (py3.15, if the PEP is accepted):

PEP 747 introduces typing.TypeForm, which represents runtime values that are themselves type expressions. This will allow Astropy to write backend-generic functions that explicitly accept an Array-API backend as an argument. For example:

from typing_extensions import TypeForm
from array_apiimport ArrayNamespace
from typing import TypeVar

TNS = TypeVar("TNS", bound=ArrayNamespace)

def func(x, y, *, xp: TypeForm[TNS]):
    ...

In this pattern, xp is a type expression describing an Array-API–compatible namespace such as NumPy, Dask, CuPy, or JAX. The type variable TNS ties that type expression to the internal operations of the function. A type checker can then ensure that all intermediate and output arrays produced inside func use the exact backend described by xp.

This capability is a component to Astropy’s roadmap of supporting multiple array types. With TypeForm-typed backend arguments, static analysis can verify that core routines do not perform unsafe backend conversions. Examples of such dangerous operations include converting a Dask array to a NumPy array (which would trigger potentially catastrophic eager CPU computation), converting a JAX array to NumPy (breaking JIT/grad and causing host transfers), or dropping CuPy arrays to CPU memory.

Being able to typecheck backend-specific behavior is crucial for preventing catastrophic performance failures in large computations. Using TypeForm ensures that an Astropy function that claims to support an arbitrary Array-API backend actually does so: all arrays stay on the correct device, the correct namespace is used for all operations, and no fallback to CPU occurs unless explicitly intended.

Having read through the discussion, I'm OK with adding this as a dependency. This is mostly based on the small footprint and low impact described by @neutrinoceros.

Hi humans 👋 - this pull request hasn't had any new commits for approximately 4 months. I plan to close this in 30 days if the pull request doesn't have any new commits by then.

In lieu of a stalled pull request, please consider closing this and open an issue instead if a reminder is needed to revisit in the future. Maintainers may also choose to add keep-open label to keep this PR open but it is discouraged unless absolutely necessary.

If this PR still needs to be reviewed, as an author, you can rebase it to reset the clock.

If you believe I commented on this pull request incorrectly, please report this here.

I'm going to close this pull request as per my previous message. If you think what is being added/fixed here is still important, please remember to open an issue to keep track of it. Thanks!

If this is the first time I am commenting on this issue, or if you believe I closed this issue incorrectly, please report this here.

@astropy/astropy-core-maintainers are we planning to drop Python 3.11 for Astropy 8.1 or 9?
If so, then adding typing-extensions as a temp dependency will smooth that transition, allowing typing-related features to be added now that will continue to be supported going forwards.

I support this initiative. I still think however it would be an easier sell if it was only a dependency on older python versions where we actually want to use it.