BLD: set a temporary upper limit on runtime-requirement for numpy - #15206

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

• Do the proposed changes actually accomplish desired goals?
• Do the proposed changes follow the Astropy coding guidelines?
• Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
• Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
• Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
• Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
• Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
• Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
• Is a milestone set? Milestone must be set but we cannot check for it on Actions; do not let the green checkmark fool you.
• At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

it's not clear to me why this patch wouldn't be acceptable in the main branch

Because a lot of downstream packages test against dev versions of dependencies (e.g., numpy-dev + astropy-dev). If we pin too strictly in the dev (main) branch, it might force numpy-dev to silently downgrade to a stable numpy, which is not desirable. I have had that happened so many time in combo of other packages that I am really tired of it.

Thanks for opening a follow-up PR as requested. I appreciate it!

If we pin too strictly in the dev (main) branch, it might force numpy-dev to silently downgrade to a stable numpy, which is not desirable.

I'll fully admit I don't understand everything about pip's dependency resolver, so I'm not excluding this possibility, and before this fires back, I'd like to disclose that I'm not advocating for using upper boundaries lightly, but numpy 2.0 is a set to be a major exception in the scientific Python landscape's history, so there's that. Should I redirect the PR to a release branch instead, and if so, which one ?

I understand your concerns, @neutrinoceros . We already have multiple patches to main trying to catch up with numpy 2.0.dev here, so I can imagine the headache downstream where someone does not have time to update their packages the same way. Let's wait a little and see if other devs have any inputs. I will also put this on the agenda of our next infrastructure tag-up.

works for me. Thanks !

this capping doesn't prevent future-compatibility testing using nightlies numpy builds, because the dev version currently used by numpy (2.0.0dev0) compares as < 2.0.

While this reflects how packaging.version.Version works:

>>> from packaging.version import Version
>>> Version("2.0.0dev0") < Version("2.0")
True

Empically, I find that pip does not follow the same rules, and will indeed downgrade numpy because of the upper limit, as @pllim suspected. I now agree that it's best not to merge this on the main branch, but maybe this still has value on a backport branch ? If not, feel free to just close this

but maybe this still has value on a backport branch ?

I can see the value in that. I have added this as an agenda for the next infrastructure tag-up on 2023-08-29. But if we don't get quorum that day (due to vacation etc), then the next one.

I thought about this more and I still don't think we should do this in main. Since there are conflicts now, I am just going to close this and move the discussion to #15393 . Thank you for your understanding!