DEP: Deprecate setting the shape attribute of a numpy array #29536

eendebakpt · 2025-08-10T20:43:22Z

Addresses part of #28800. Also see the discussion in #29523.

seberg

Nice, I much prefer this, I think! I don't care if we even consider _set_shape() semi-public. It's the users problem to know that they are mutating a fresh array if they insist, although in general, I hope there isn't really a reason to ever do so.

(__array_finalize__ is hopefully one of the very few ones.)

seberg · 2025-08-11T12:57:42Z

numpy/_core/src/multiarray/getset.c

        PyErr_SetString(PyExc_AttributeError,
                "Cannot delete array shape");
        return -1;
    }


I guess we can delete the "cannot delete" path here.

Yes. This is supposes to be called from the python side or via array_shape_set (which checks on val), so val cannot be null. I will add an assert just in case.

numpy/_core/tests/test_umath.py

seberg · 2025-08-11T13:04:26Z

doc/release/upcoming_changes/29492.deprecation.rst

+Setting the shape attribute is now deprecated since mutating
+an array is unsafe if an array is shared, especially by multiple
+threads.  As an alternative, you can create a new view via
+`np.reshape` or `array.reshape`. For example: `x = np.arange(15); x = np.reshape(x, (3, 5))`.


We should point out the option to use reshape(..., copy=False) to ensure an error (which is one reason why some places used the shape attribute setting).

We could actually roll that out in the places where things were changed here, although overall, I am not really worried about accidental copies here.

I feel we should mention _set_shape() somewhere maybe. Probably just here? I.e. say that if there is no alternative and you know that you are the only owner (for example within __array_finalize__) setting the shape is still possible via _set_shape()). This is mostly used by matrices although we discourage the behavior.

I added a mention to _set_shape without further explanation (to discourage usage).

seberg · 2025-08-11T13:08:10Z

numpy/_core/src/multiarray/getset.c

+    /* Deprecated NumPy 2.4, 2025-07-29 */
+    if (DEPRECATE("Setting the shape on a NumPy array has been deprecated"
+                  " in NumPy 2.4.\nAs an alternative, you can create a new"
+                  " view using np.reshape."


Could also mention copy=False here.

Specifically, maybe add " using np.reshape (with copy=False if needed)."

Co-authored-by: Sebastian Berg <[email protected]>

… deprecate_shape_v3

seberg

FWIW, I am happy to try this, if others agree mildly let's give it a shot. Might apply the suggestiong or do a minor thing, but I think this should be ready rally.

doc/release/upcoming_changes/29492.deprecation.rst

Co-authored-by: Sebastian Berg <[email protected]>

eendebakpt · 2025-08-28T09:49:05Z

Scipy and astropy are using the shape setting. I creates issues to keep track scipy/scipy#23522, astropy/astropy#18561.

mhvk

I think this all looks good! I really have only very minor comments -- to be honest hardly worth running CI again, but maybe just...

numpy/_core/tests/test_umath.py

mhvk · 2025-08-28T16:07:53Z

numpy/lib/_function_base_impl.py

        # switch first and second axis
-        output[0].shape = (1, -1) + s0[2:]
-        output[1].shape = (-1, 1) + s0[2:]
+        output[0] = output[0].reshape((1, -1) + s0[2:])


I'd tend to instead generate output separately for this case, but obviously no big deal.

Still think if makes more sense to do,

if indexing == 'xy' and ndim > 1: output = [np.asanyarray(x).reshape((1, -1) + s0[2:] for x in xi] else: output = [np.asanyarray(x).reshape(s0[:i] + (-1,) + s0[i + 1:]) for i, x in enumerate(xi)]

But I can also see that you want to minimize changes...

numpy/linalg/_linalg.py

numpy/ma/core.py

numpy/random/_generator.pyx

numpy/random/mtrand.pyx

mhvk · 2025-08-28T16:24:57Z

doc/release/upcoming_changes/29492.deprecation.rst

+`np.reshape` or `np.ndarray.reshape`. For example: ``x = np.arange(15); x = np.reshape(x, (3, 5))``.
+To ensure no copy is made from the data, one can use ``np.reshape(..., copy = False)``.
+
+Directly setting the shape on an array is possible with the private method `np.ndarray._set_shape`.


I'd add ", though is discouraged" (on a new line).

mhvk · 2025-08-28T16:28:30Z

numpy/_core/src/multiarray/getset.c

+    /* Deprecated NumPy 2.4, 2025-07-29 */
+    if (DEPRECATE("Setting the shape on a NumPy array has been deprecated"
+                  " in NumPy 2.4.\nAs an alternative, you can create a new"
+                  " view using np.reshape."


Specifically, maybe add " using np.reshape (with copy=False if needed)."

mhvk · 2025-08-28T17:09:18Z

p.s. I just saw that #29523 is still alive. Is that still being considered, or are we just going for it here? (I'm fine with that.)

eendebakpt · 2025-08-28T17:53:16Z

p.s. I just saw that #29523 is still alive. Is that still being considered, or are we just going for it here? (I'm fine with that.)

No decision has been made on which version to use iirc, but I am fine with this one.

mhvk

Basically ready! My only real comment is about the changelog snippet. The rest is mostly attempts to make usage more uniform.

doc/release/upcoming_changes/29492.deprecation.rst

mhvk · 2025-10-18T23:47:48Z

numpy/_core/tests/test_regression.py

            b.shape = (10,)

-        assert_raises(AttributeError, rs)
+        with warnings.catch_warnings():  # gh-28901


Since python 3.11, one can just do with warnings.catch_warnings(action="ignore", category=DeprecationWarning).

Though I think just using pytest.warns(DeprecationWarning), as you do below, is cleaner, and would suggest you use that everywhere.

doc/release/upcoming_changes/29492.deprecation.rst

mhvk · 2025-10-18T23:53:29Z

numpy/lib/_function_base_impl.py

        # switch first and second axis
-        output[0].shape = (1, -1) + s0[2:]
-        output[1].shape = (-1, 1) + s0[2:]
+        output[0] = output[0].reshape((1, -1) + s0[2:])


Still think if makes more sense to do,

if indexing == 'xy' and ndim > 1: output = [np.asanyarray(x).reshape((1, -1) + s0[2:] for x in xi] else: output = [np.asanyarray(x).reshape(s0[:i] + (-1,) + s0[i + 1:]) for i, x in enumerate(xi)]

But I can also see that you want to minimize changes...

mhvk · 2025-10-18T23:54:30Z

numpy/linalg/tests/test_linalg.py

            if np.asarray(b).ndim == 1:
-                expect_resids.shape = (1,)
+                expect_resids = expect_resids.reshape((1,))
                assert_equal(residuals.shape, expect_resids.shape)


Surely, this line should be outside the if...

Yes I agree, changed.

numpy/ma/core.py

doc/release/upcoming_changes/29536.deprecation.rst

mhvk

Looks all good to me, thanks!

seberg · 2025-11-04T10:43:18Z

I think we even checked and fixed larger downstream packages for this, right? Because in that case I think we can try to do this (can always revert if there is larger fallout).
If not, maybe let's wait until after branching since it may be a bit noisier (although I have to say that a deprecation doesn't break an end-user workflow, worst case it may slow it a bit).

eendebakpt · 2025-11-07T20:37:46Z

If I recall correctly astropy, h5py and scipy have been addressed and there are PRs open for yt-project. scikit-learn still has a couple of deprecated operations I think.

seberg · 2025-11-08T14:33:33Z

Thanks, I am tempted to wait for 2.5 then. If we are worried about fix, I think there is far more disruption to worry about here.

eendebakpt · 2025-12-15T23:44:37Z

Thanks, I am tempted to wait for 2.5 then. If we are worried about fix, I think there is far more disruption to worry about here.

@seberg Is this ready to go in?

eendebakpt added 14 commits August 6, 2025 23:46

[DEP] Deprecate setting the shape attribute of a numpy array

98264cb

add release note

27ef5f2

update tests

b7842a8

more conversions

9af1e83

convert more tests

1695917

ma

1d71fc5

fix generator and mrecords

e8cbd57

review comments

354a04e

remove debugging code

bea4884

remove examples

5c8a65e

restore examples wiht only get of shape

914462d

remove doc with deprecated shape setting

7e06ce2

update release note

2e80bf0

warn for all subclasses of ndarray

d49ab1e

github-actions bot added the 07 - Deprecation label Aug 10, 2025

eendebakpt added 2 commits August 10, 2025 22:55

convert one more shape set

37d1b42

add deprecation test

b888253

seberg reviewed Aug 11, 2025

View reviewed changes

eendebakpt and others added 3 commits August 15, 2025 15:04

Update numpy/_core/tests/test_umath.py

5576aa2

Co-authored-by: Sebastian Berg <[email protected]>

review comments

5de80b9

Merge branch 'deprecate_shape_v3' of github.com:eendebakpt/numpy into…

0ff8bb0

… deprecate_shape_v3

seberg approved these changes Aug 19, 2025

View reviewed changes

doc/release/upcoming_changes/29492.deprecation.rst Outdated Show resolved Hide resolved

doc/release/upcoming_changes/29492.deprecation.rst Outdated Show resolved Hide resolved

eendebakpt and others added 3 commits August 24, 2025 21:27

Apply suggestions from code review

f096794

Co-authored-by: Sebastian Berg <[email protected]>

Merge branch 'main' into deprecate_shape_v3

1a237cf

Merge branch 'main' into deprecate_shape_v3

198e62a

eendebakpt mentioned this pull request Aug 28, 2025

MAINT: Replace deprecated usage of numpy array shape, dtype and strides setting scipy/scipy#23522

Open

mhvk approved these changes Aug 28, 2025

View reviewed changes

review comments

f739d54

eendebakpt added 2 commits September 4, 2025 20:56

Merge branch 'main' into deprecate_shape_v3

87a45e7

Merge branch 'main' into deprecate_shape_v3

e51276d

mhvk mentioned this pull request Oct 7, 2025

TSK: deprecate setting strides, dtype, and shape on arrays. #28800

Open

4 tasks

eendebakpt mentioned this pull request Oct 18, 2025

DEP: Deprecate setting the shape attribute of an exact numpy array #29523

Closed

eendebakpt requested review from mhvk and seberg October 18, 2025 18:12

mhvk reviewed Oct 19, 2025

View reviewed changes

This was referenced Oct 19, 2025

TST: avoid mutating ndarray.shape attributes yt-project/unyt#597

Open

BUG: avoid soon-to-be-deprecated direct mutations of ndarray.shape attributes (1/2) yt-project/yt#5307

Merged

RFC: avoid direct mutations to ndarray.shape attributes h5py/h5py#2728

Merged

review comments

f45274e

tylerjereddy reviewed Oct 19, 2025

View reviewed changes

doc/release/upcoming_changes/29536.deprecation.rst Show resolved Hide resolved

update pr number

96e029b

eendebakpt requested a review from mhvk October 29, 2025 22:12

mhvk approved these changes Nov 4, 2025

View reviewed changes

seberg added this to the 2.5.0 Release milestone Nov 8, 2025

jorenham mentioned this pull request Nov 22, 2025

DEP, TYP: ndarray.shape setter pending deprecation #30282

Merged

Merge branch 'main' into deprecate_shape_v3

48cdec4

Uh oh!

DEP: Deprecate setting the shape attribute of a numpy array #29536

Are you sure you want to change the base?

DEP: Deprecate setting the shape attribute of a numpy array #29536

Conversation

eendebakpt commented Aug 10, 2025

Uh oh!

seberg left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seberg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

eendebakpt commented Aug 28, 2025

Uh oh!

mhvk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mhvk commented Aug 28, 2025

Uh oh!

eendebakpt commented Aug 28, 2025

Uh oh!

mhvk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

mhvk left a comment

Choose a reason for hiding this comment

Uh oh!

seberg commented Nov 4, 2025

Uh oh!

eendebakpt commented Nov 7, 2025

Uh oh!

seberg commented Nov 8, 2025

Uh oh!

eendebakpt commented Dec 15, 2025

Uh oh!

Reviewers

Assignees