TYP: excessively vague return types from np.dtype and np.ndarray.dtype in numpy 2.2.0

Describe the issue:

While upgrading gpgi's CI to numpy 2.2.0, I got a new typechecking error that seems spurious to me: an array's dtype appears to not have a defined type (as seen by mypy).
No error is raised with numpy 2.1.3.

Reproduce the code example:

import numpy as np

def get_float_dtype(reference: np.ndarray) -> np.dtype:
    dt = reference.dtype
    if dt.kind != "f":
        raise ValueError(f"Invalid data type {dt} (expected a float dtype)")
    return np.dtype(dt)
    # also tried with
    return dt

Error message:

gpgi.py:7: error: Returning Any from function declared to return "dtype[Any]"  [no-any-return]
Found 1 error in 1 file (checked 1 source file)

Python and NumPy Versions:

2.2.0
3.13.1 (main, Dec  4 2024, 16:25:14) [Clang 16.0.0 (clang-1600.0.26.4)]

Type-checker version and settings:

mypy==1.13.0

the --warn-return-any flag is needed to reproduce

Additional typing packages.

Included with mypy, but not used here (as far as I'm aware)

typing-extensions==4.12.2

reference: np.ndarray is invalid: it's missing the type arguments, which makes it impossible for type-checkers to infer the actual dtype that you're expecting it to have. Simlarly np.dtype is also missing its type argument for the scalar type.

Judging from your example, I believe you'd want to annotate it as something like:

import numpy as np
import numpy.typing as npt

def get_float_dtype(reference: npt.NDArray[np.floating]) -> np.dtype[np.floating]: ...

or even better:

from typing import TypeVar
import numpy as np
import numpy.typing as npt

_FloatingT = TypeVar("_FloatingT", bound=np.floating)

def get_float_dtype(reference: npt.NDArray[_FloatingT]) -> np.dtype[_FloatingT]: ...

This is working as intended.

reference: np.ndarray is invalid

Is this generally true starting in NumPy v2.2.0, that raw np.ndarray annotations without shape/dtype qualifiers are invalid? If so, that change invalidates a lot of existing code.

reference: np.ndarray is invalid

Is this generally true starting in NumPy v2.2.0, that raw np.ndarray annotations without shape/dtype qualifiers are invalid? If so, that change invalidates a lot of existing code.

This has always been invalid.

Thanks a lot. I was able to fix the real life example my MRC was derived from (unfortunately I can't use your preferred solution, TypeVar, because the real code needs to accommodate for reference=None) !

unfortunately I can't use your preferred solution, TypeVar, because the real code needs to accommodate for reference=None

You could use @overload to split it up into a case for reference: None and another for reference: npt.NDArray[_FloatingT]

This has always been invalid.

Well, that's news to me! It never resulted in an error before, which I suspect is why the pattern is used hundreds of thousands of times across github: https://github.com/search?q=%22%3A+np.ndarray%22&type=code

NumPy 2.2 is going to break a lot of code.

It's news to me too. I'm culprit of sprinkling such annotations in many places back a couple years ago before np.typing existed, and I'm not sure how to fix most of them.

Is there any way we can be gentle here and keep allowing whatever is the biggest "breaking" changing here?!

This has always been invalid.

Well, that's news to me! It never resulted in an error before, which I suspect is why the pattern is used hundreds of thousands of times across github: https://github.com/search?q=%22%3A+np.ndarray%22&type=code

NumPy 2.2 is going to break a lot of code.

ndarray has always been invalid, but that gets hidden by type-checkers unless you're in strict mode. NumPy 2.2.0 won't make this "more invalid" or something, so no worries

I'm culprit of sprinkling such annotations in many places back a couple years ago before np.typing existed, and I'm not sure how to fix most of them.

There's no need to, unless you enable strict mode in your type-checker settings.

Fun fact: NumPy 2.2.0 actually made things like np.floating explicitly valid by having it default to np.floating[Any]. See the release notes for details.

Thanks a lot. I was able to fix the real life example my MRC was derived from (unfortunately I can't use your preferred solution, TypeVar, because the real code needs to accommodate for reference=None) !

Closing this now as resolved.

Navigation Menu

Repository navigation

TYP: excessively vague return types from `np.dtype` and `np.ndarray.dtype` in numpy 2.2.0 #27945

Describe the issue:

Reproduce the code example:

Error message:

Python and NumPy Versions:

Type-checker version and settings:

Additional typing packages.

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

TYP: excessively vague return types from np.dtype and np.ndarray.dtype in numpy 2.2.0 #27945

Description

Describe the issue:

Reproduce the code example:

Error message:

Python and NumPy Versions:

Type-checker version and settings:

Additional typing packages.

Activity

jorenham commented on Dec 10, 2024

jakevdp commented on Dec 10, 2024

jorenham commented on Dec 10, 2024

neutrinoceros commented on Dec 10, 2024

jorenham commented on Dec 10, 2024

jakevdp commented on Dec 10, 2024

neutrinoceros commented on Dec 10, 2024

seberg commented on Dec 10, 2024

jorenham commented on Dec 10, 2024

jorenham commented on Dec 10, 2024

jorenham commented on Dec 10, 2024

jorenham commented on Dec 10, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

TYP: excessively vague return types from `np.dtype` and `np.ndarray.dtype` in numpy 2.2.0 #27945