Skip to content

Release 6.1 when numpy 2.0 is released? #16200

Closed

Description

@mhvk

What is the problem this feature will solve?

Numpy 2.0 is a sufficiently breaking change that it feels like trying to support it in a bugfix release in 6.0 is not really sensible any more: some things are just going to break and that is unexpected for a bugfix release.

But it would be very good to have an official astropy release that does work with numpy 2.0. Hence, @taldcroft and I wondered whether it might make sense to aim for a somewhat earlier than usual 6.1. It is not completely clear how much earlier, but the beta version of numpy 2.0 has been released and the expectation is that the final version will be about a month from now. Does this make sense?

My own feeling is that 6.1 in itself is looking like a rather minor release, with not too much that is new (which is not illogical, since bigger things have to wait for 7.0 anyway). So, from that respect, perhaps it is not so crazy to do feature-freeze in about a week?

Alternative

As noted by @pllim below, we can just add a big fat warning in 6.0.1 that the numpy 2.0 changes are sufficiently large that we cannot guarantee that astropy will run the same way with numpy 2.0 as it did with 1.x in all cases.

The obvious advantage is that in that case we can just stick to our own schedule.

The disadvantage is that people may not read very carefully and those just trying to use the latest versions of everything may get hurt.

Activity

  1. added this to the v6.1.0 milestone on Mar 15, 2024
  2. astrofrog commented on Mar 15, 2024

    @astrofrog
    Member

    I'd be happy with this, but which PRs are important to include for Numpy 2.0 support?

  3. neutrinoceros commented on Mar 15, 2024

    @neutrinoceros
    Contributor

    @astrofrog see #16167 (comment) for a list of PRs that triggered this discussion.

  4. astrofrog commented on Mar 15, 2024

    @astrofrog
    Member

    Thanks!

  5. astrofrog commented on Mar 15, 2024

    @astrofrog
    Member

    So just to be clear, should we release 6.1 just before or just after numpy 2.0 final is released ideally?

  6. neutrinoceros commented on Mar 15, 2024

    @neutrinoceros
    Contributor

    ABI compatibility requires we make a release (not necessarily 6.1) at some point between numpy 2.0rc1 and 2.0 final.

  7. pllim commented on Mar 15, 2024

    @pllim
    Member

    I haven't been following closely so I am confused. I thought #16142 already took care of "copy" in a backward compatible way and all the extra stuff are "nice to have"?

  8. mhvk commented on Mar 15, 2024

    @mhvk
    ContributorAuthor

    @pllim - I think the conclusion is dawning on us that numpy 2.0 is enough of a break (mostly because of the changed meaning of copy) that making all code work consistently between numpy 1 and numpy 2 is not going to work. Instead, people have to adjust. Since that means that astrop 6.0.x will work somewhat differently with numpy 1.x and 2.0 (if one passes in copy arguments), it seems cleanest just to say that for 6.0, numpy 2.0 is not supported, and for 6.1 to explain that one has to be careful.

    But perhaps we are being too careful? After all, most code will just work fine, and most breakage gives a clear exception.

  9. pllim commented on Mar 15, 2024

    @pllim
    Member

    That would push v6.1 ahead for 2-3 months. I am not sure if I am comfortable with that. 🤔 🤪

    Is it an option to issue a memo that "yes you can use v6.0.1 with numpy 2.0 but..." and then they can wait a bit more for v6.1 without us changing our release schedule? Or is that bad because we are changing the behavior of this package between v6.0.1 and v6.1? But then again, v6.1 has other non-backward compatible changes too so maybe that is ok?

    TBH it is going to take the scientific python world months to catch up anyway. At that time, v6.1 is due to release regardless.

  10. mhvk commented on Mar 15, 2024

    @mhvk
    ContributorAuthor

    @pllim - your comments make me waver in the opposite direction again. So far, it has certainly been the plan to just have numpy 2.0 be supported in 6.0.1... Let me put this on top as an alternative.

  11. saimn commented on Mar 15, 2024

    @saimn
    Contributor

    That would push v6.1 ahead for 2-3 months.

    A bit less than a month actually, v6.1 Feature Freeze was planed for 2024-04-19.
    Given the potentially breaking change that Numpy 2.0 will be for people in general, I agree with the initial proposal, adding support in Astropy 6.1 and moving forward a bit the release. I think that will make things more clear for downstream packages and users.

  12. neutrinoceros commented on Mar 18, 2024

    @neutrinoceros
    Contributor

    We have another reason to go straight to 6.1 for numpy 2 compat: string arrays now support addition, which implies a change in table aggregation rules.
    See #16211 (comment)

  13. mhvk commented on Mar 20, 2024

    @mhvk
    ContributorAuthor
  14. astrofrog commented on Mar 20, 2024

    @astrofrog
    Member

    Given that it might be a bit soon to do feature freeze this week, would doing it Friday next week be soon enough? Could someone confirm when Numpy 2.0 is scheduled for?

  15. pllim commented on Mar 20, 2024

    @pllim
    Member

    I still think v6.0.x is compatible enough as it is... 👀

  16. neutrinoceros commented on Mar 20, 2024

    @neutrinoceros
    Contributor

    Could someone confirm when Numpy 2.0 is scheduled for?

    Here's the latest estimate numpy/numpy#24300 (comment)
    Currently, the pessimistic timeline puts numpy 2.0.0rc1 in 3 days from now, but there was no public statement for 11 days so I supposed it could still be delayed.

  17. mhvk commented on Mar 20, 2024

    @mhvk
    ContributorAuthor

    I've also not seen much else either, but rc1 seems to be getting close. I suspect a final release in about a month from now. To me, the main benefit of having a separate astropy release at the same time is that people can adjust both their numpy and astropy pieces to deal with the numpy changes (probably mostly the copy keyword, but also the changes in representation of scalars for doctests, etc.). E.g., I don't think we'd want to backport the astropy doc changes (#15065) to 6.0.x, yet it would be nice if we had docs that used the newest numpy.

    Anyway, maybe the right time to send out a message to astropy-dev to ask if a Friday 29th freeze would really inconvenience anyone? My sense so far is that this release has been mostly about refactoring - the what's-new only has that io.ascii now uses 64-bit by default (which is really a numpy 2.0 driven change). So, in that respect little to stop us from releasing a bit earlier.

  18. pllim commented on Mar 20, 2024

    @pllim
    Member

    Well... Our devdeps is currently failing, so it won't make sense for any release until #16211 is resolved anyway.

  19. mhvk commented on Mar 20, 2024

    @mhvk
    ContributorAuthor

    That one is being fixed upstream; I don't think there is much doubt that numpy will release fairly soon, and we might as well get ready for that. Unless there are other things we really want to have in 6.1 -- among the 80 open ones, I don't see anything of great interest, but that large number suggests that a few weeks of triaging those would be good regardless!

  20. pllim commented on Mar 20, 2024

    @pllim
    Member

    That one is being fixed upstream

    numpy/numpy#26079 was abandoned. I hope they still plan to fix it...

  21. mhvk commented on Mar 20, 2024

    @mhvk
    ContributorAuthor

    numpy/numpy#26079 was abandoned.

    Indeed, because a better way was found in discussion!

  22. neutrinoceros commented on Mar 25, 2024

    @neutrinoceros
    Contributor

    I see that @saimn just opened #16230, so it looks like we're going for a bugfix release after all ?
    In any case, if 6.0.x is supposed to have numpy 2.0 compat, please consider backporting #16229 first

  23. saimn commented on Mar 25, 2024

    @saimn
    Contributor

    #16231: Numpy 2.0 support will come in v6.1.0, as discussed here.

  24. astrofrog commented on Mar 25, 2024

    @astrofrog
    Member

    I've emailed astropy-dev notifying people of the upcoming feature freeze this Friday.

  25. pllim commented on May 6, 2024

    @pllim
    Member

    astropy v6.1.0 is released and pins to numpy 2.0rc for build.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions