Skip to content

Conversation

@jorenham
Copy link
Member

@jorenham jorenham commented Nov 4, 2025

This fixes inspect.signature from raising a ValueError for the following functions:

  • numpy.arange
  • numpy.char.compare_chararrays
  • numpy.busdaycalendar
  • numpy.datetime_data
  • numpy.from_dlpack
  • numpy.frombuffer
  • numpy.fromfile
  • numpy.fromiter
  • numpy.frompyfunc
  • numpy.fromstring
  • numpy.nested_iters
  • numpy.promote_types

The tests were getting a bit messy, so I cleaned them up a bit, and grouped the signature-related onces into a class-bsaed test.

I'll try and improve the arange signature in a follow-up, as that'll require quite a few changes in the stubs.

Related to #30104, #30114, #30121, #30124, #30126, #30137, #30138, #30140, #30143, and #30146.

FYI, still on my inspect.signature to-do list are the classes in numpy.random, ndarray.to_device (which is missing docs), and 53 of the np.generic methods.

@jorenham jorenham added this to the 2.4.0 release milestone Nov 4, 2025
@jorenham jorenham changed the title BUG: remaining _core.multiarray function runtime signatures BUG: add the remaining _core.multiarray function runtime signatures Nov 4, 2025
@jorenham
Copy link
Member Author

jorenham commented Nov 4, 2025

Hmm apparently the array api testsuite doesn't understand that *args and **kwargs are (always) compatible with Liskov's substitution principle. I guess I'll have to put the planned arange changes in this PR then.

@github-actions

This comment has been minimized.

@jorenham
Copy link
Member Author

jorenham commented Nov 4, 2025

Ah woops, step can also be None apparently.

@jorenham jorenham changed the title BUG: add the remaining _core.multiarray function runtime signatures BUG, TYP: add the remaining _core.multiarray function runtime signatures Nov 5, 2025
@github-actions

This comment has been minimized.

@jorenham
Copy link
Member Author

jorenham commented Nov 5, 2025

In order to be compatible with the Array API spec for arange, the first start parameter had to be made positional-only. I've tried making it positional-or-keyword-only, but I now think that's not possible to do in a type-safe manner.

This won't cause any backwards-incompatibily issues at runtime, as you can still call arange(start=..., stop=...) just fine. However, static type-checkers will no longer accept this. But then again, I don't see any way around it.

But as mypy_primer shows, calling arange this way isn't as uncommon as I'd have hoped. So I think I'll include a release note explaining why this change was needed. That'll have to wait until tomorrow though.

@jorenham jorenham added the 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes label Nov 5, 2025
@jorenham
Copy link
Member Author

jorenham commented Nov 5, 2025

re: the primer diff (#30147 (comment)):

The errors for Jax and Colour look correct to me; arange does not support ndarray input, so mypy rightly rejects it.

The xarray errors are a consequence of the start parameter no longer being usable as keyword-argument, as I've explained in #30147 (comment). Luckily, it's very easy to fix these errors; i.e. pass start positionally instead of by keyword.

@jorenham jorenham force-pushed the fix-remaining-top-level-function-signatures branch from 21d7edc to fa476b5 Compare November 6, 2025 10:37
@jorenham jorenham removed the 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes label Nov 6, 2025
@github-actions
Copy link

github-actions bot commented Nov 6, 2025

Diff from mypy_primer, showing the effect of this PR on type check results on a corpus of open source code:

jax (https://github.com/google/jax)
+ jax/_src/numpy/lax_numpy.py:5994: error: Argument 1 to "arange" has incompatible type "Array | ndarray[tuple[Any, ...], dtype[Any]] | numpy.bool[builtins.bool] | number[Any, int | float | complex] | builtins.bool | int | float | complex | TypedNdArray | int | Any"; expected "integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float"  [arg-type]
+ jax/_src/scipy/special.py:1211: error: Incompatible return value type (got "signedinteger[_64Bit]", expected "ndarray[tuple[Any, ...], dtype[Any]]")  [return-value]

xarray (https://github.com/pydata/xarray)
+ xarray/tests/test_plot.py: note: In member "setUp" of class "TestDiscreteColorMap":
+ xarray/tests/test_plot.py:1167: error: No overload variant of "arange" matches argument types "int", "int", "int"  [call-overload]
+ xarray/tests/test_plot.py:1167: note: Possible overload variants:
+ xarray/tests/test_plot.py:1167: note:     def [_ArangeScalarT: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None]] arange(integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float, /, stop: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float | None = ..., step: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float | None = ..., *, dtype: type[_ArangeScalarT] | dtype[_ArangeScalarT] | _SupportsDType[dtype[_ArangeScalarT]], device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[_ArangeScalarT]]
+ xarray/tests/test_plot.py:1167: note:     def arange(int | integer[Any] | numpy.bool[builtins.bool], /, stop: int | integer[Any] | numpy.bool[builtins.bool] | None = ..., step: int | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[int] | type[signedinteger[_32Bit | _64Bit]] | dtype[signedinteger[_32Bit | _64Bit]] | _SupportsDType[dtype[signedinteger[_32Bit | _64Bit]]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[signedinteger[_32Bit | _64Bit]]]
+ xarray/tests/test_plot.py:1167: note:     def arange(float | floating[Any], /, stop: float | floating[Any] | integer[Any] | numpy.bool[builtins.bool] | None = ..., step: float | floating[Any] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[float] | type[float64] | dtype[float64] | _SupportsDType[dtype[float64]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[float64 | Any]]
+ xarray/tests/test_plot.py:1167: note:     def arange(float | floating[Any] | integer[Any] | numpy.bool[builtins.bool], /, stop: float | floating[Any], step: float | floating[Any] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[float] | type[float64] | dtype[float64] | _SupportsDType[dtype[float64]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[float64 | Any]]
+ xarray/tests/test_plot.py:1167: note:     def arange(timedelta64[timedelta | int | None], /, stop: int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool] | None = ..., step: int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[timedelta64[timedelta | int | None]] | dtype[timedelta64[timedelta | int | None]] | _SupportsDType[dtype[timedelta64[timedelta | int | None]]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[timedelta64[Any]]]
+ xarray/tests/test_plot.py:1167: note:     def arange(int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool], /, stop: timedelta64[timedelta | int | None], step: int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[timedelta64[timedelta | int | None]] | dtype[timedelta64[timedelta | int | None]] | _SupportsDType[dtype[timedelta64[timedelta | int | None]]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[timedelta64[Any]]]
+ xarray/tests/test_plot.py:1167: note:     def arange(datetime64[date | int | None], /, stop: datetime64[date | int | None], step: int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[datetime64[date | int | None]] | dtype[datetime64[date | int | None]] | _SupportsDType[dtype[datetime64[date | int | None]]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[datetime64[Any]]]
+ xarray/tests/test_plot.py:1167: note:     def arange(integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float, /, stop: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float | None = ..., step: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float | None = ..., *, dtype: type[Any] | dtype[Any] | _SupportsDType[dtype[Any]] | tuple[Any, Any] | list[Any] | _DTypeDict | str | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[Any]]
+ xarray/tests/test_plot.py:1168: error: No overload variant of "arange" matches argument types "int", "int", "int"  [call-overload]
+ xarray/tests/test_plot.py:1168: note: Possible overload variants:
+ xarray/tests/test_plot.py:1168: note:     def [_ArangeScalarT: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None]] arange(integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float, /, stop: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float | None = ..., step: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float | None = ..., *, dtype: type[_ArangeScalarT] | dtype[_ArangeScalarT] | _SupportsDType[dtype[_ArangeScalarT]], device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[_ArangeScalarT]]
+ xarray/tests/test_plot.py:1168: note:     def arange(int | integer[Any] | numpy.bool[builtins.bool], /, stop: int | integer[Any] | numpy.bool[builtins.bool] | None = ..., step: int | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[int] | type[signedinteger[_32Bit | _64Bit]] | dtype[signedinteger[_32Bit | _64Bit]] | _SupportsDType[dtype[signedinteger[_32Bit | _64Bit]]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[signedinteger[_32Bit | _64Bit]]]
+ xarray/tests/test_plot.py:1168: note:     def arange(float | floating[Any], /, stop: float | floating[Any] | integer[Any] | numpy.bool[builtins.bool] | None = ..., step: float | floating[Any] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[float] | type[float64] | dtype[float64] | _SupportsDType[dtype[float64]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[float64 | Any]]
+ xarray/tests/test_plot.py:1168: note:     def arange(float | floating[Any] | integer[Any] | numpy.bool[builtins.bool], /, stop: float | floating[Any], step: float | floating[Any] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[float] | type[float64] | dtype[float64] | _SupportsDType[dtype[float64]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[float64 | Any]]
+ xarray/tests/test_plot.py:1168: note:     def arange(timedelta64[timedelta | int | None], /, stop: int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool] | None = ..., step: int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[timedelta64[timedelta | int | None]] | dtype[timedelta64[timedelta | int | None]] | _SupportsDType[dtype[timedelta64[timedelta | int | None]]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[timedelta64[Any]]]
+ xarray/tests/test_plot.py:1168: note:     def arange(int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool], /, stop: timedelta64[timedelta | int | None], step: int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[timedelta64[timedelta | int | None]] | dtype[timedelta64[timedelta | int | None]] | _SupportsDType[dtype[timedelta64[timedelta | int | None]]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[timedelta64[Any]]]
+ xarray/tests/test_plot.py:1168: note:     def arange(datetime64[date | int | None], /, stop: datetime64[date | int | None], step: int | timedelta64[timedelta | int | None] | integer[Any] | numpy.bool[builtins.bool] | None = ..., *, dtype: type[datetime64[date | int | None]] | dtype[datetime64[date | int | None]] | _SupportsDType[dtype[datetime64[date | int | None]]] | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[datetime64[Any]]]
+ xarray/tests/test_plot.py:1168: note:     def arange(integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float, /, stop: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float | None = ..., step: integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None] | float | None = ..., *, dtype: type[Any] | dtype[Any] | _SupportsDType[dtype[Any]] | tuple[Any, Any] | list[Any] | _DTypeDict | str | None = ..., device: Literal['cpu'] | None = ..., like: _SupportsArrayFunc | None = ...) -> ndarray[tuple[int], dtype[Any]]
+ xarray/tests/test_plot.py: note: At top level:

colour (https://github.com/colour-science/colour)
+ colour/plotting/temperature.py:150: error: Incompatible types in assignment (expression has type "ndarray[tuple[Any, ...], dtype[floating[_16Bit] | floating[_32Bit] | float64]]", variable has type "ndarray[tuple[Any, ...], dtype[float64]]")  [assignment]
- colour/plotting/quality.py:214: note:     def __sub__(self, int | numpy.bool[builtins.bool], /) -> ndarray[tuple[int], dtype[generic[Any]]]
+ colour/plotting/quality.py:214: note:     def __sub__(self, int | numpy.bool[builtins.bool], /) -> ndarray[tuple[int], dtype[integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None]]]
- colour/plotting/quality.py:214: note:     def __sub__(self, _SupportsArray[dtype[numpy.bool[builtins.bool]]] | _NestedSequence[_SupportsArray[dtype[numpy.bool[builtins.bool]]]] | builtins.bool | _NestedSequence[builtins.bool], /) -> ndarray[tuple[Any, ...], dtype[generic[Any]]]
+ colour/plotting/quality.py:214: note:     def __sub__(self, _SupportsArray[dtype[numpy.bool[builtins.bool]]] | _NestedSequence[_SupportsArray[dtype[numpy.bool[builtins.bool]]]] | builtins.bool | _NestedSequence[builtins.bool], /) -> ndarray[tuple[Any, ...], dtype[integer[Any] | floating[Any] | datetime64[date | int | None] | timedelta64[timedelta | int | None]]]
- colour/utilities/tests/test_array.py:1604: error: Incompatible types in assignment (expression has type "ndarray[tuple[int], dtype[signedinteger[Any]]]", variable has type "int")  [assignment]
+ colour/utilities/tests/test_array.py:1604: error: Incompatible types in assignment (expression has type "ndarray[tuple[int], dtype[signedinteger[_32Bit | _64Bit]]]", variable has type "int")  [assignment]

@charris charris merged commit 42cb3f9 into numpy:main Nov 6, 2025
79 checks passed
@charris
Copy link
Member

charris commented Nov 6, 2025

Thanks Joren.

charris pushed a commit that referenced this pull request Nov 6, 2025
This fixes `inspect.signature` for

- `np.random.BitGenerator`
- `np.random.Generator`
- `np.random.MT19937`
- `np.random.PCG64`
- `np.random.PCG64DXSM`
- `np.random.Philox`
- `np.random.RandomState`
- `np.random.SFC64`
- `np.random.SeedSequence`
- `np.random.bit_generator.SeedlessSeedSequence`

This also fixes a typo in `bit_generator.pxd` that accidentally defined an empty unused class `np.random.bit_generator.SeedlessSequence`.

Related to #30104, #30114, #30121, #30124, #30126, #30137, #30138, #30140, #30143, #30146, #30147, and #30155
@jorenham jorenham deleted the fix-remaining-top-level-function-signatures branch November 6, 2025 19:51
@rgommers
Copy link
Member

The blast radius for this PR, and the whole "add more signatures" effort in general, seems quite large (the mypy_primer output and incoming links show JAX, AstroPy, SciPy, scikit-image and Colour from just this PR) so this seems to be in need of a prominent entry in the release notes at least.

@jorenham
Copy link
Member Author

The blast radius for this PR, and the whole "add more signatures" effort in general, seems quite large

So far, the reactions have been mostly positive. But the impact that this seems to have is pretty surprising to me.

(the mypy_primer output and incoming links show JAX, AstroPy, SciPy, scikit-image and Colour from just this PR)

The mypy_primer output looks pretty scary at first glance, but I'm pretty sure that the new mypy_errors that this will introduce for jax and colour, are justified; i.e. that they're either true positives. And in case of xarray, related to the arange(start=) change. See #30147 (comment) for some details.

If you consider the reported line numbers in the mypy_primer diff, you'll see that we're talking about 5 lines of code here; so most what you're seeing is just irrelevant noise :)

so this seems to be in need of a prominent entry in the release notes at least.

That's also what I was thinking, yea. I wasn't sure yet what the best way to go about that was. Open a separate PR for the release note (highlight?) and link that PR, or use the PR number of one these actual PR's?

@jorenham
Copy link
Member Author

so this seems to be in need of a prominent entry in the release notes at least.

#30208 :)

rgommers added a commit to rgommers/numpy that referenced this pull request Nov 16, 2025
This reverts a very small part of numpygh-30147, because it broken Pythran
(see pythran#2368). While a new Pythran 0.18.1 release is available,
that doesn't help for the SciPy 1.15.x releases which allow
numpy<2.5 and pythran<0.18, see
https://github.com/scipy/scipy/blob/maintenance/1.15.x/pyproject.toml#L30-L41
So introducing this signature change only for 2.5.0 is much better.

In addition, doing so doesn't leave us only a very narrow window with
compatible NumPy and Pythran releases which would otherwise bite distro
packagers as well.
cakedev0 pushed a commit to cakedev0/numpy that referenced this pull request Dec 5, 2025
This reverts a very small part of numpygh-30147, because it broken Pythran
(see pythran#2368). While a new Pythran 0.18.1 release is available,
that doesn't help for the SciPy 1.15.x releases which allow
numpy<2.5 and pythran<0.18, see
https://github.com/scipy/scipy/blob/maintenance/1.15.x/pyproject.toml#L30-L41
So introducing this signature change only for 2.5.0 is much better.

In addition, doing so doesn't leave us only a very narrow window with
compatible NumPy and Pythran releases which would otherwise bite distro
packagers as well.
cakedev0 pushed a commit to cakedev0/numpy that referenced this pull request Dec 5, 2025
cakedev0 pushed a commit to cakedev0/numpy that referenced this pull request Dec 5, 2025
This fixes `inspect.signature` for

- `np.random.BitGenerator`
- `np.random.Generator`
- `np.random.MT19937`
- `np.random.PCG64`
- `np.random.PCG64DXSM`
- `np.random.Philox`
- `np.random.RandomState`
- `np.random.SFC64`
- `np.random.SeedSequence`
- `np.random.bit_generator.SeedlessSeedSequence`

This also fixes a typo in `bit_generator.pxd` that accidentally defined an empty unused class `np.random.bit_generator.SeedlessSequence`.

Related to numpy#30104, numpy#30114, numpy#30121, numpy#30124, numpy#30126, numpy#30137, numpy#30138, numpy#30140, numpy#30143, numpy#30146, numpy#30147, and numpy#30155
IndifferentArea pushed a commit to IndifferentArea/numpy that referenced this pull request Dec 7, 2025
IndifferentArea pushed a commit to IndifferentArea/numpy that referenced this pull request Dec 7, 2025
This fixes `inspect.signature` for

- `np.random.BitGenerator`
- `np.random.Generator`
- `np.random.MT19937`
- `np.random.PCG64`
- `np.random.PCG64DXSM`
- `np.random.Philox`
- `np.random.RandomState`
- `np.random.SFC64`
- `np.random.SeedSequence`
- `np.random.bit_generator.SeedlessSeedSequence`

This also fixes a typo in `bit_generator.pxd` that accidentally defined an empty unused class `np.random.bit_generator.SeedlessSequence`.

Related to numpy#30104, numpy#30114, numpy#30121, numpy#30124, numpy#30126, numpy#30137, numpy#30138, numpy#30140, numpy#30143, numpy#30146, numpy#30147, and numpy#30155
IndifferentArea pushed a commit to IndifferentArea/numpy that referenced this pull request Dec 7, 2025
This reverts a very small part of numpygh-30147, because it broken Pythran
(see pythran#2368). While a new Pythran 0.18.1 release is available,
that doesn't help for the SciPy 1.15.x releases which allow
numpy<2.5 and pythran<0.18, see
https://github.com/scipy/scipy/blob/maintenance/1.15.x/pyproject.toml#L30-L41
So introducing this signature change only for 2.5.0 is much better.

In addition, doing so doesn't leave us only a very narrow window with
compatible NumPy and Pythran releases which would otherwise bite distro
packagers as well.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants