-
-
Notifications
You must be signed in to change notification settings - Fork 11.8k
BUG, TYP: add the remaining _core.multiarray function runtime signatures
#30147
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BUG, TYP: add the remaining _core.multiarray function runtime signatures
#30147
Conversation
_core.multiarray function runtime signatures_core.multiarray function runtime signatures
|
Hmm apparently the array api testsuite doesn't understand that |
This comment has been minimized.
This comment has been minimized.
|
Ah woops, |
_core.multiarray function runtime signatures_core.multiarray function runtime signatures
This comment has been minimized.
This comment has been minimized.
|
In order to be compatible with the Array API spec for This won't cause any backwards-incompatibily issues at runtime, as you can still call But as mypy_primer shows, calling |
|
re: the primer diff (#30147 (comment)): The errors for Jax and Colour look correct to me; The xarray errors are a consequence of the |
21d7edc to
fa476b5
Compare
|
Diff from mypy_primer, showing the effect of this PR on type check results on a corpus of open source code: jax (https://github.com/google/jax)
+ jax/_src/numpy/lax_numpy.py:5994: error: Argument 1 to "arange" has incompatible type "Array | ndarray[tuple[Any, ...], dtype[Any]] | numpy.bool[builtins.bool] | number[Any, int | float | complex] | builtins.bool | int | float | complex | TypedNdArray | int | Any"; expected "integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float" [arg-type]
+ jax/_src/scipy/special.py:1211: error: Incompatible return value type (got "signedinteger[_64Bit]", expected "ndarray[tuple[Any, ...], dtype[Any]]") [return-value]
xarray (https://github.com/pydata/xarray)
+ xarray/tests/test_plot.py: note: In member "setUp" of class "TestDiscreteColorMap":
+ xarray/tests/test_plot.py:1167: error: No overload variant of "arange" matches argument types "int", "int", "int" [call-overload]
+ xarray/tests/test_plot.py:1167: note: Possible overload variants:
+ xarray/tests/test_plot.py:1167: note: def [_ArangeScalarT: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None]] arange(integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float, /, stop: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float | None = ..., step: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float | None = ..., *, dtype: type[_ArangeScalarT] | dtype[_ArangeScalarT] | _SupportsDType[dtype[_ArangeScalarT]], device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[_ArangeScalarT]]
+ xarray/tests/test_plot.py:1167: note: def arange(int | integer[Any] | numpy.bool[builtins.bool], /, stop: int | integer[Any] | numpy.bool[builtins.bool] | None = ..., step: int | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[int] | type[signedinteger[_32Bit | _64Bit]] | dtype[signedinteger[_32Bit | _64Bit]] | _SupportsDType[dtype[signedinteger[_32Bit | _64Bit]]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[signedinteger[_32Bit | _64Bit]]]
+ xarray/tests/test_plot.py:1167: note: def arange(float | floating[Any], /, stop: float | floating[Any] | integer[Any] | numpy.bool[builtins.bool] | None = ..., step: float | floating[Any] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[float] | type[float64] | dtype[float64] | _SupportsDType[dtype[float64]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[float64 | Any]]
+ xarray/tests/test_plot.py:1167: note: def arange(float | floating[Any] | integer[Any] | numpy.bool[builtins.bool], /, stop: float | floating[Any], step: float | floating[Any] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[float] | type[float64] | dtype[float64] | _SupportsDType[dtype[float64]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[float64 | Any]]
+ xarray/tests/test_plot.py:1167: note: def arange(timedelta64[timedelta | int | None], /, stop: int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool] | None = ..., step: int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[timedelta64[timedelta | int | None]] | dtype[timedelta64[timedelta | int | None]] | _SupportsDType[dtype[timedelta64[timedelta | int | None]]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[timedelta64[Any]]]
+ xarray/tests/test_plot.py:1167: note: def arange(int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool], /, stop: timedelta64[timedelta | int | None], step: int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[timedelta64[timedelta | int | None]] | dtype[timedelta64[timedelta | int | None]] | _SupportsDType[dtype[timedelta64[timedelta | int | None]]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[timedelta64[Any]]]
+ xarray/tests/test_plot.py:1167: note: def arange(datetime64[date | int | None], /, stop: datetime64[date | int | None], step: int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[datetime64[date | int | None]] | dtype[datetime64[date | int | None]] | _SupportsDType[dtype[datetime64[date | int | None]]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[datetime64[Any]]]
+ xarray/tests/test_plot.py:1167: note: def arange(integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float, /, stop: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float | None = ..., step: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float | None = ..., *, dtype: type[Any] | dtype[Any] | _SupportsDType[dtype[Any]] | tuple[Any, Any] | list[Any] | _DTypeDict | str | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[Any]]
+ xarray/tests/test_plot.py:1168: error: No overload variant of "arange" matches argument types "int", "int", "int" [call-overload]
+ xarray/tests/test_plot.py:1168: note: Possible overload variants:
+ xarray/tests/test_plot.py:1168: note: def [_ArangeScalarT: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None]] arange(integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float, /, stop: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float | None = ..., step: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float | None = ..., *, dtype: type[_ArangeScalarT] | dtype[_ArangeScalarT] | _SupportsDType[dtype[_ArangeScalarT]], device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[_ArangeScalarT]]
+ xarray/tests/test_plot.py:1168: note: def arange(int | integer[Any] | numpy.bool[builtins.bool], /, stop: int | integer[Any] | numpy.bool[builtins.bool] | None = ..., step: int | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[int] | type[signedinteger[_32Bit | _64Bit]] | dtype[signedinteger[_32Bit | _64Bit]] | _SupportsDType[dtype[signedinteger[_32Bit | _64Bit]]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[signedinteger[_32Bit | _64Bit]]]
+ xarray/tests/test_plot.py:1168: note: def arange(float | floating[Any], /, stop: float | floating[Any] | integer[Any] | numpy.bool[builtins.bool] | None = ..., step: float | floating[Any] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[float] | type[float64] | dtype[float64] | _SupportsDType[dtype[float64]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[float64 | Any]]
+ xarray/tests/test_plot.py:1168: note: def arange(float | floating[Any] | integer[Any] | numpy.bool[builtins.bool], /, stop: float | floating[Any], step: float | floating[Any] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[float] | type[float64] | dtype[float64] | _SupportsDType[dtype[float64]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[float64 | Any]]
+ xarray/tests/test_plot.py:1168: note: def arange(timedelta64[timedelta | int | None], /, stop: int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool] | None = ..., step: int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[timedelta64[timedelta | int | None]] | dtype[timedelta64[timedelta | int | None]] | _SupportsDType[dtype[timedelta64[timedelta | int | None]]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[timedelta64[Any]]]
+ xarray/tests/test_plot.py:1168: note: def arange(int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool], /, stop: timedelta64[timedelta | int | None], step: int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[timedelta64[timedelta | int | None]] | dtype[timedelta64[timedelta | int | None]] | _SupportsDType[dtype[timedelta64[timedelta | int | None]]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[timedelta64[Any]]]
+ xarray/tests/test_plot.py:1168: note: def arange(datetime64[date | int | None], /, stop: datetime64[date | int | None], step: int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[datetime64[date | int | None]] | dtype[datetime64[date | int | None]] | _SupportsDType[dtype[datetime64[date | int | None]]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[datetime64[Any]]]
+ xarray/tests/test_plot.py:1168: note: def arange(integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float, /, stop: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float | None = ..., step: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float | None = ..., *, dtype: type[Any] | dtype[Any] | _SupportsDType[dtype[Any]] | tuple[Any, Any] | list[Any] | _DTypeDict | str | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[Any]]
+ xarray/tests/test_plot.py: note: At top level:
colour (https://github.com/colour-science/colour)
+ colour/plotting/temperature.py:150: error: Incompatible types in assignment (expression has type "ndarray[tuple[Any, ...], dtype[floating[_16Bit] | floating[_32Bit] | float64]]", variable has type "ndarray[tuple[Any, ...], dtype[float64]]") [assignment]
- colour/plotting/quality.py:214: note: def __sub__(self, int | numpy.bool[builtins.bool], /) -> ndarray[tuple[int], dtype[generic[Any]]]
+ colour/plotting/quality.py:214: note: def __sub__(self, int | numpy.bool[builtins.bool], /) -> ndarray[tuple[int], dtype[integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None]]]
- colour/plotting/quality.py:214: note: def __sub__(self, _SupportsArray[dtype[numpy.bool[builtins.bool]]] | _NestedSequence[_SupportsArray[dtype[numpy.bool[builtins.bool]]]] | builtins.bool | _NestedSequence[builtins.bool], /) -> ndarray[tuple[Any, ...], dtype[generic[Any]]]
+ colour/plotting/quality.py:214: note: def __sub__(self, _SupportsArray[dtype[numpy.bool[builtins.bool]]] | _NestedSequence[_SupportsArray[dtype[numpy.bool[builtins.bool]]]] | builtins.bool | _NestedSequence[builtins.bool], /) -> ndarray[tuple[Any, ...], dtype[integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None]]]
- colour/utilities/tests/test_array.py:1604: error: Incompatible types in assignment (expression has type "ndarray[tuple[int], dtype[signedinteger[Any]]]", variable has type "int") [assignment]
+ colour/utilities/tests/test_array.py:1604: error: Incompatible types in assignment (expression has type "ndarray[tuple[int], dtype[signedinteger[_32Bit | _64Bit]]]", variable has type "int") [assignment]
|
|
Thanks Joren. |
This fixes `inspect.signature` for - `np.random.BitGenerator` - `np.random.Generator` - `np.random.MT19937` - `np.random.PCG64` - `np.random.PCG64DXSM` - `np.random.Philox` - `np.random.RandomState` - `np.random.SFC64` - `np.random.SeedSequence` - `np.random.bit_generator.SeedlessSeedSequence` This also fixes a typo in `bit_generator.pxd` that accidentally defined an empty unused class `np.random.bit_generator.SeedlessSequence`. Related to #30104, #30114, #30121, #30124, #30126, #30137, #30138, #30140, #30143, #30146, #30147, and #30155
|
The blast radius for this PR, and the whole "add more signatures" effort in general, seems quite large (the |
So far, the reactions have been mostly positive. But the impact that this seems to have is pretty surprising to me.
The mypy_primer output looks pretty scary at first glance, but I'm pretty sure that the new mypy_errors that this will introduce for jax and colour, are justified; i.e. that they're either true positives. And in case of xarray, related to the If you consider the reported line numbers in the mypy_primer diff, you'll see that we're talking about 5 lines of code here; so most what you're seeing is just irrelevant noise :)
That's also what I was thinking, yea. I wasn't sure yet what the best way to go about that was. Open a separate PR for the release note (highlight?) and link that PR, or use the PR number of one these actual PR's? |
#30208 :) |
This reverts a very small part of numpygh-30147, because it broken Pythran (see pythran#2368). While a new Pythran 0.18.1 release is available, that doesn't help for the SciPy 1.15.x releases which allow numpy<2.5 and pythran<0.18, see https://github.com/scipy/scipy/blob/maintenance/1.15.x/pyproject.toml#L30-L41 So introducing this signature change only for 2.5.0 is much better. In addition, doing so doesn't leave us only a very narrow window with compatible NumPy and Pythran releases which would otherwise bite distro packagers as well.
This reverts a very small part of numpygh-30147, because it broken Pythran (see pythran#2368). While a new Pythran 0.18.1 release is available, that doesn't help for the SciPy 1.15.x releases which allow numpy<2.5 and pythran<0.18, see https://github.com/scipy/scipy/blob/maintenance/1.15.x/pyproject.toml#L30-L41 So introducing this signature change only for 2.5.0 is much better. In addition, doing so doesn't leave us only a very narrow window with compatible NumPy and Pythran releases which would otherwise bite distro packagers as well.
This fixes `inspect.signature` for - `np.random.BitGenerator` - `np.random.Generator` - `np.random.MT19937` - `np.random.PCG64` - `np.random.PCG64DXSM` - `np.random.Philox` - `np.random.RandomState` - `np.random.SFC64` - `np.random.SeedSequence` - `np.random.bit_generator.SeedlessSeedSequence` This also fixes a typo in `bit_generator.pxd` that accidentally defined an empty unused class `np.random.bit_generator.SeedlessSequence`. Related to numpy#30104, numpy#30114, numpy#30121, numpy#30124, numpy#30126, numpy#30137, numpy#30138, numpy#30140, numpy#30143, numpy#30146, numpy#30147, and numpy#30155
This fixes `inspect.signature` for - `np.random.BitGenerator` - `np.random.Generator` - `np.random.MT19937` - `np.random.PCG64` - `np.random.PCG64DXSM` - `np.random.Philox` - `np.random.RandomState` - `np.random.SFC64` - `np.random.SeedSequence` - `np.random.bit_generator.SeedlessSeedSequence` This also fixes a typo in `bit_generator.pxd` that accidentally defined an empty unused class `np.random.bit_generator.SeedlessSequence`. Related to numpy#30104, numpy#30114, numpy#30121, numpy#30124, numpy#30126, numpy#30137, numpy#30138, numpy#30140, numpy#30143, numpy#30146, numpy#30147, and numpy#30155
This reverts a very small part of numpygh-30147, because it broken Pythran (see pythran#2368). While a new Pythran 0.18.1 release is available, that doesn't help for the SciPy 1.15.x releases which allow numpy<2.5 and pythran<0.18, see https://github.com/scipy/scipy/blob/maintenance/1.15.x/pyproject.toml#L30-L41 So introducing this signature change only for 2.5.0 is much better. In addition, doing so doesn't leave us only a very narrow window with compatible NumPy and Pythran releases which would otherwise bite distro packagers as well.
This fixes
inspect.signaturefrom raising aValueErrorfor the following functions:numpy.arangenumpy.char.compare_chararraysnumpy.busdaycalendarnumpy.datetime_datanumpy.from_dlpacknumpy.frombuffernumpy.fromfilenumpy.fromiternumpy.frompyfuncnumpy.fromstringnumpy.nested_itersnumpy.promote_typesThe tests were getting a bit messy, so I cleaned them up a bit, and grouped the
signature-related onces into a class-bsaed test.I'll try and improve thearangesignature in a follow-up, as that'll require quite a few changes in the stubs.Related to #30104, #30114, #30121, #30124, #30126, #30137, #30138, #30140, #30143, and #30146.
FYI, still on my
inspect.signatureto-do list are the classes innumpy.random,ndarray.to_device(which is missing docs), and 53 of thenp.genericmethods.