Skip to content

Conversation

@madphysicist
Copy link
Contributor

@madphysicist madphysicist commented Feb 10, 2021

As per @charris's request, I am resurrecting #7804.

This likely also closes #12336.

Discussed in the mailing list forever ago: https://mail.python.org/pipermail/numpy-discussion/2016-July/075722.html
Discussed in the mailing list now: https://mail.python.org/pipermail/numpy-discussion/2021-February/081476.html

I did manage to implement np.atleast_3d in terms of np.atleast_nd.

@seberg seberg added 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes 62 - Python API Changes or additions to the Python API. Mailing list should usually be notified. and removed 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes labels Feb 10, 2021
@seberg
Copy link
Member

seberg commented Feb 10, 2021

I am pretty neutral on this addition (I honestly don't understand where its useful, but it also doesn't seem like a big feature creep – i.e. this is only useful when you don't know the input). If the old mailing list discussion was super positive, I might have overlooked it.

But I think this has to hit the mailing list again with the concrete current proposal.

I guess the old point is. If I just want to add dimensions on the left (usually not necessary), there are many options like np.broadcast_to(arr, (1,)*ndim). If I know the input dimensions (i.e. it is fixed), I can use np.expand_dims. Of course this function is probably easier on the eye, so maybe worth it even in those cases. Otherwise, I am mainly wondering if the multiple vs. one array choice and the pos kwarg choice need discussion still (could make that kwarg only possibly).

@madphysicist
Copy link
Contributor Author

madphysicist commented Feb 10, 2021

@seberg I am actually in a similar boat. There are a couple of cases where this would seem like a nice-to-have at best. I thought my original PR was dead and was prepared to leave it that way. The original feedback was pretty ambivalent, although I did have a few people ask me about it later. This is here pretty much because @charris asked me to revive it.

@madphysicist
Copy link
Contributor Author

@BvB93 It's pretty clear that the remaining errors are something I did in the typing tests, but I can't seem to figure it out. Could you take a look when you have the chance? I'm especially confused about the error saying that numpy does not have an attribute atleast_nd.

@madphysicist
Copy link
Contributor Author

I've squashed all the commits now that the last bug is fixed. At this point, code is good to go unless someone has something to say on the new mailing list post.

@madphysicist
Copy link
Contributor Author

Looks like the Azure failtures are not related to anything I did.

Base automatically changed from master to main March 4, 2021 02:05


@array_function_dispatch(_atleast_nd_dispatcher)
def atleast_nd(ary, ndim, pos=0):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In numpy.array you can pre-pend ones to the array-shape by giving a 'ndmin' parameter.
Therefore, I think the variable name 'ndim' should be renamed to 'ndmin' to make it consistent with numpy.array.

Copy link
Contributor

@pbrod pbrod Oct 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the atleast_nd should allow multiple inputs in the same way as atleast_1d, atleast_2d and atleast_3d do. This will make it easier to use since then the call syntax and output will be the same as the atleast_XXXd functions. Any written code with multiple uses of the function will be shorter like this:

x1, x2, x3 = numpy.atleast_nd(x1,x2,x3, ndmin=3)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pos argument is the axis index to insert the new dimensions. What about renaming 'pos' to 'axis'? That will make it more consistent with the normalize_axis_index function.

@dg-pb
Copy link

dg-pb commented Nov 13, 2023

I hope this will be merged. Otherwise, could anyone provide a simple way to do what this functionality provides?

I.e. if my input is a 2d array - leave it as it is, otherwise if it is a 1d array, append one dimension to the end.

Neither of above suggestions seem to do the trick.

@jonas-eschle
Copy link

Hi, what's the status of this? Glad to help if anything is stoping it from going forward.

If I know the input dimensions (i.e. it is fixed), I can use np.expand_dims

@seberg I think that's the whole point of atleast_1d etc: if you know the shape, you can convert them by hand, but the point is that you don't need to know and check, and all you want is to have a certain shape, no matter what's the input.
The only difference in this PR is that you don't want the additional dimension to be always in the first place, therefore I would argue, whoever needs atleast_nd needs this PR (except if you're lucky and your case coincides with pos=0)

For the use-case, I find the 2D shape of (event, dimensions) very common in many applications and therefore the desire to have one dimensional arrays made to 2D with shape (event, 1)

Looking forward to have this in!

@dg-pb
Copy link

dg-pb commented Apr 18, 2024

I have been using this for a fair while now. Extremely useful. I don't even use the other ones atleast_1d|2d|3d anymore, only this one.

  1. I would like to propose simpler implementation. The one proposed here is elegant, but a bit too clever (at least for my taste) - takes me few seconds to get my head round every time I look at it again.
def atleast_nd(ary, ndim, pos=0):
    ary = array(ary, copy=False, subok=True)
    if ary.ndim > ndim:
        pos = normalize_axis_index(pos, ary.ndim + 1)
        shape = ary.shape
        new_shape = shape[:pos] + (1,) * extra + shape[pos:]
        return reshape(a, new_shape)
    return ary
  1. is operator.index(ndim) needed here? Are there objects that represent number of dimensions in this manner?

  2. What I also have found useful is having one more flag, which raises error if input array number of dimensions is higher than requested. I named it exact (in contrast with at least).

def atleast_nd(ary, ndim, pos=0, exact=False):
    ary = array(ary, copy=False, subok=True)
    extra = ary.ndim > ndim
    if extra > 0:
        pos = normalize_axis_index(pos, ary.ndim + 1)
        shape = ary.shape
        new_shape = shape[:pos] + (1,) * extra + shape[pos:]
        return reshape(a, new_shape)
    elif exact and extra != 0:
        raise ValueError(f'{ndim=} > {nd=}')
    return ary

@madphysicist
Copy link
Contributor Author

madphysicist commented Apr 18, 2024 via email

@dg-pb
Copy link

dg-pb commented Apr 18, 2024

I'm happy to revisit the implementation.

By the way, the version above is 100ns slower than yours.

I chose to use this one for being slightly simpler, but I would be happy with either one merged into numpy.

@mattip
Copy link
Member

mattip commented Nov 9, 2025

This PR got quite close to the finish line, got bogged down in review, and then bit-rotted. I think we should get it across the finish line following the array API. Note we have numpy._core.shape_base._atleast_nd since #11991, and as far as I can tell it should be sufficient to make that function public.

I am not sure why the interface is so complicated. The Array API does not support multiple input arrays i.e. atleast_nd(a, b, c, d, ndims=10) so I don't think we need to either. We could extend the Array API to include a pos argument, but I would prefer not to. The Array API is pretty clear that this function only prepends dimensions. We can leave atleast_3d (which I would not touch in this PR) for those use cases, like creating 3d images, that want a post-appended dimension.

@jonas-eschle
Copy link

The Array API is pretty clear that this function only prepends dimensions.

Well, that's a good argument. I think sometimes it's still okay to deviate and expand functionality, in the end, the standard needs to be continuously developed.

The use-case I see is simply to fill dimensions where they may have been omitted (the may is important here since it's the atleast, and not a prepend function. So If I know, either the array is in dimension x, y, or only x, I want it to be (x, y) or (x, 1).
But if we can't specify a pos argument, atleast_* only solves the case of prepending and the above example is not solved at all; we also cannot use it to get (1, x) and then swap the axis, because if it was already (x, y) before, we don't wanna swap. It has to be one function (or split into a ndims_missing with an insert_dims or something).

But maybe I am missing the point: how would you solve the above? (In an ndim case)

@mattip
Copy link
Member

mattip commented Nov 10, 2025

@jonas-eschle are you asking because you have a use case or because it would be nice to have a generic swiss-army-knife function? We do try to stick to the array API standard unless there is a really good reason to deviate. And for a specific problem, I think it would be more straight-forward to write a one-line padded = a[:, :, None] rather than adapt atleast to all the different padding options.

@jonas-eschle
Copy link

you have a use case

Yes, but I am also curious to know general use-cases for atleast_nd, my personal impression is that prepending is just one out of many use-cases. Simple example, you have a feature vector. Both shapes, (nfeat) as well as (nfeat, 1) or(with nfeatures, nevents dimensions for example). That's a simple use-case where we want to append, not prepend.

more straight-forward to write a one-line padded = a[:, :, None]

This does not work. We will need to check first if it's one or two dimensional. But that's the whole point (afaiu) of atleast_*: check the dimension, pad until the dimensionality is as high as requested. The problem is, the current function only allows prepending, and not padding.

Maybe I am missing the point that makes prepending a more prominent use-case than just general padding?

@madphysicist
Copy link
Contributor Author

madphysicist commented Nov 11, 2025 via email

@jonas-eschle
Copy link

The original usecase for this function was to both append and prepend dimensions. The function in the array API is not sufficient for that.

This would only support a pos argument I think, a natural generalization of 0, -1 basically.

Supporting the array API should, IMHO, not restrict improvements to functionality. There is a slim chance that the array API would add the same argument name for a different behavior, but that seems unlikely; to me, it seems more likely that the array API would follow the pos approach.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

01 - Enhancement 62 - Python API Changes or additions to the Python API. Mailing list should usually be notified.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

atleast_2d axis argument option

8 participants