Astropy should endorse SPEC 0

@bsipocz

As suggested by @bsipocz in astropy/astropy#20184 (comment)

SPEC main page: https://scientific-python.org/specs/
SPEC 0 is at https://scientific-python.org/specs/spec-0000/ and is mentioned in https://github.com/astropy/astropy-APEs/blob/main/APE18.rst
SPEC has a mention of Astropy at https://scientific-python.org/specs/core-projects/astropy/

Thanks for linking to the APE; we clearly have had these same discussions multiple times and it would be nice if we could agree on a scheme that does not depend on subjective ideas about difficulty of maintenance.

Crucially, we need to agree on what "endorsement" means here. I've typically been on the opposition side of SPEC0-focused discussions, though in my own view I could argue I still endorse the SPEC; I just think that what compliance forbids is much more valuable than what it allows. In my opinion, the SPEC shouldn't be a reason to shed off backward compatibility layers, and not used as an argument in this direction. I do agree these discussions need to happen within the margins defined by the SPEC, but I don't think it's wise to try and settle them before they happen through policy. In essence, when the SPEC allows for it, we can start discussing the maintenance tradeoffs associated with a backward compatible layer, but we shouldn't assume the cost of dropping to be null.

I do note that my interpretation of the SPEC is both in minority and possibly at odds with the authors' intentions.

I guess this is where predictable comes in... I'd really like (1) not to revisit the same discussion over and over; (2) the decision to be made at a fixed point in time (ideally, ahead of a development cycle, i.e., just after a release, so there is clarity for the next half year) and should involve looking up things, not discussion. Note that I'm OK with giving some leeway, like "we drop whatever SPEC 0 allows to drop at the start/middle of a development cycle" instead of what it will allow to drop at release time. Similarly, I could imagine we update optional dependencies only for major releases. Heck, we could even do that for required. Really, the main thing to me is to have a scheme that does not need further thought and discussion.

I hold the opposite belief that these discussions are essential 😅

In hindsight, I think the main reason I want these discussions to happen is that the pace that SPEC0 allows is too fast for my taste. If we can find a slightly slower one that satisfies every party, I would agree that case by case discussions are not needed.

The issue is that SPEC 0 basically leaves it up to us to discuss dropping versions every time. What would be good is to have a system that means we don't have to discuss it every time. For example we could have a new APE that says that eg 25% or 50% more time than specified by SPEC 0 is the maximum time window size while SPEC 0 is the minimum, then dropping dependencies and Python versions at the e.g. 50% extra boundary would not be up for discussion.

sounds like a good middle ground to me.

One of my hopes for PlasmaPy has been to automate SPEC 0 dependency updates via monthly pull requests to bump the minimum supported versiosn of package dependencies. Given the amount of discussion about this in the astropythoniverse, automation would probably save a lot of of developer time. 🌠

I submitted astral-sh/uv#7515 to request that the ability to perform SPEC 0 updates be implemented in uv.

I also started working on a package to automatically bump-minimum-dependencies as per a user-defined schedule, though I haven't gotten it to work yet for complicated version specifiers. I think others have worked on automating these updates too. 🤔

automation would probably save a lot of of developer time.

With all due respect I think it's a really bad idea. We need to own these decisions and their consequences. Automation completely removes the cognitive friction that these decisions should cost.

We need to own these decisions and their consequences. Automation completely removes the cognitive friction that these decisions should cost.

While the creation of the PR would be automated, the PR itself would undergo code review before being merged, so the developers would still own the decisions and consequences. The goal would be to streamline the cognitive friction, not eliminate it. These PRs could be thought of as suggested quarterly SPEC 0 updates to be reviewed and then discussed at a community meeting before deciding whether to accept the changes.

It'd be a lot like dependabot.

Note

It turns out that there is such a workflow at scientific-python/spec0-action. It's not very customizable yet so I don't think it'll fit Astropy's needs, unless the decision is to strictly follow the SPEC 0 drop schedule. Here's a sample PR it created: PlasmaPy/PlasmaPy#3363

That's still problematic to me, as these changes would be suggested based on nothing but timing, instead of coming up organically as need arises.

+1000 on developing and agreeing upon a scheme that we can apply going forward without this recurring and never-converging discussion. It's both frustrating and a cost to the project.

Whatever we end up doing, I think the most important thing is clear communication to stakeholders (especially big pipelines that tend to be more rigid than research code) and give them ample time to object/adapt, so that means no bumping version, say, in the last month (?) before feature freeze.

I also agree that we can treat different tiers of dependencies slightly differently, from major to minor, something as follows:

python version
numpy version
pyyaml, scipy, matplotlib
other runtime dependencies that are not expected to change constantly, so that would exclude things like astropy-iers-data and certifi
test dependencies
docs dependencies

I think it is okay to bump (5) and (6) even in bugfix releases, so we can backport compatibility patches for release branches to keep testing and generating docs. The rest I leave up to you, as long as the communication requirement is satisfied.

Hope this helps!

@neutrinoceros

I am very much in favour of a mindless scheme, with this discussion not being needed again. The only question to me is how closely we stick to the shortest period. I think there are two basic options:

At start of development cycle, drop what will be droppable by release time (strict SPEC 0);
At start of development cycle, drop what is droppable at that point (i.e., we will typically trail SPEC 0 a little at release time);

This way one can develop against a known, definite state, which moves forward at a know, reliable pace.

In principle, one could also do the same for major releases only, but I worry that this still leads to unnecessary discussion, now about whether or not to have a major release, e.g., if a PR author finds during the development cycle that SPEC 0 compliant newer dependencies would really help. If we were to go that route, I would advocate to making yearly major releases automatic.

Note that I'm not against someone checking at release time whether it is possible to continue support beyond SPEC 0, i.e., if @neutrinoceros or someone else who cares sufficiently is willing to try reverting in the release branch the commit that changed minimum versions and removed work-around code, and finds that after resolving the conflicts everything still builds correctly, then I'd be fine for that to be released.

@taldcroft

I wonder if the timescale of research code and facilities is different enough that I'm not sure any of the discussion about SPEC 0 really matters for facilities. Let me lay it out.

A research facility fixes a version of Astropy, e.g. 7.0. We at astropy don't do major breaking changes in the dot releases so they should be fine to naturally bump up to 7.1, 7.2, etc, but we allow for breaking changes on the major releases, so 8.0 might require real work to support. If they want to fix a code environment then 7.x it is. This is effectively a ~1 year guarantee for being current with Astropy, after which they are just pinning to an old version of Astropy. Bugfixes are done for that year and after are made on a good-faith effort, but really it's better if we review PRs made by the people who need the bug fixed. Facilities won't go from 7.x to 8.x if they update every 2-3 years, they'll go right to 9.x/10.x, with all the attendant breaking changes.
Spec 0 basically drops EoL code every year, on pretty much the same cadence as Astropy allows for breaking changes. Moreover all the other Spec 0 adoptees are dropping explicit support on some similar cadence (+/- six months)

So the development timelines align relatively closely, and its facilities that can lag years behind. We have a bimodal distribution of desired support windows!
To me, this says that it's not the development timescale that's a problem and we should be free to adopt Spec 0 (+ 6 months). The real underlying issue it's our ability to backport bug fixes for our slow-cadence users (facilities)!

It seems like tackling the bug-fix issue would solve the actual community problem, as I understand it. What can we do?

LTS? Oh no? But maybe?
- partner with institutions to do bug-fixes
- this is maybe what TideLift is good for?
Accept that as part of Astropy the maintainers should do more backports? ugh.
Very controversial: semantic back ports done by AI and checked by a maintainer. I've been doing this in my own code — e.g. GalacticDynamics/unxt@a76cddd — where I want to backport something but the code has drifted and the backport bot can't do it automatically. I just pass the PR I want to backport and the backport bot's instructions to Claude and ask for a semantic backport and explanation of all required changes. Then I review. It's fast because it's basically a backport. With this pattern I backport basically ALL bug fixes in ALL my codes.

This was a long-winded way of saying that I think Astropy's current timescales basically already aligns with Spec 0. We should adopt it in some reasonable form — I'm also fine with @taldcroft and others' timeline. We should also figure out how to serve our significant and other important community, which operates on longer timescales.

Facilities won't go from 7.x to 8.x if they update every 2-3 years, they'll go right to 9.x/10.x

This reads like we are assuming that we do breaking releases about as frequently as our policy allows for. I don't understand what justifies this assumption.

Accept that as part of Astropy the maintainers should do more backports? ugh.

You know, there's a very minor (albeit a huge one-time diff) that we could do to make future backports easier; as I've been handling a good share of manual backports I noticed that many merge conflicts, blocking automatic backports, are actually logically trivial and only affect import statements. Now, if we renounced the current compact style where multiple import statements are allowed on a single line, I think we we could get a much better success rate for automated backports. The formatters we already use support this strategy, FTR. Just a thought.

Now, if we renounced the current compact style where multiple import statements are allowed on a single line, I think we we could get a much better success rate for automated backports. The formatters we already use support this strategy, FTR. Just a thought.

I'm a fan of that.

This reads like we are assuming that we do breaking releases about as frequently as our policy allows for.

I think NOT assuming this is possible is dangerous since it's asking us to freeze our API. We have a major release cycle that is publicly advertised. We can break things in major releases. We often don't, but that's not a guarantee anyone should count on since the explicit guarantee is that major release can have changes. If people want a longer major release cycle then we should have a longer release cycle.

But major releases can have major changes.

Of course ! All I'm saying is that we could very easily have less than 1/3 of our feature-releases be majors.

we could very easily have less than 1/3 of our feature-releases be majors.

That's arguing for Astropy having a longer major release cycle. I for one vote 👎 . It's painful enough already trying to keep astropy abreast with the rapidly changing computing landscape. Major releases every 1-1.5 years is the same cadence as Python itself.

I do agree major releases can be further apart, since astropy API is no longer changing fast, but I do not think our minor releases should have fixed minimum supported versions of dependencies (except if they make a major release). I guess basically I'd be quite happy to move more to a numpy cycle, where major releases are fairly rare (but python minimum support levels are changed following SPEC 0; for reference, numpy-dev now requires python >= 3.12).

I missed the word "major" in the comment I was replying to. Yes. That is indeed what I'm arguing. One year between two majors is the minimum we guarantee, but I don't think our policy specifically requires we always go for short cycles ?

I do not think our minor releases should have fixed minimum supported versions of dependencies

If you mean fixes throughout a given minor series here, I agree for hard dependencies, but not for optional ones.

I guess basically I'd be quite happy to move more to a numpy cycle, where major releases are fairly rare

Numpy isn't following semver as closely as astropy; deprecation cycles are frequently concluded in minor releases, which would be majors in astropy.

I note that this whole thread was all triggered by astropy/astropy#20184, where I hoped to raise to python >=3.12 as that would make my C-based PRs much easier. That is of course about a hard dependency, so are you happy to change it then? As I noted, numpy has changed, so if one is stuck on python 3.11, one will be in a situation one could potentially go for astropy 9.0 but not numpy 2.6.

Note that I personally would include not just hard but also recommended dependencies. I care less about optional ones, though I don't really see a benefit of not including them (automation is easier...).

The reason I don't think we should aggressively bump optional deps is that these requirements are effectively not enforceable; specifically, they do not compose well with imperative environment installation workflows, which I'm sure everyone here wants to support and keep supporting.

For instance, say a user installs astropy in a fresh env, without extras, via pip. They try to use the library before realizing they need matplotlib too. If they install it via another pip invoke, the resolver has no way to know it should take lower bounds from astropy into account. Now if for any reason (say, pip is configured to prefer cached packages over fresh downloads, or matplotlib is pinned via tox.ini, requirements.txt or pylock.toml) they end up with an older version of matplotlib than astropy supports, they just got a broken install in two commands, and it's very hard to understand why from that standpoint. In this situation, as the user, I would immediately conclude that astropy is broken, and as a maintainer, if I somehow managed to understand what happened, I would draw the same conclusion.

This scenario is even more likely to happen via tox, which normally pieces together a test env via successive imperative commands.

That is of course about a hard dependency, so are you happy to change it then?

Yes. As a sign of good faith, I'll open that PR myself today.

Hmm, your logic about the optional dependencies makes a lot of sense. It may be good to treat these separately in our discussion; to the extent possible, maybe they should just not have version requirements at all.

For the recommended ones, we're back in the usual bind not of our making, that really we'd happily include those in the strong dependencies by default if only there were a nice & easy way not to install them (i.e., https://peps.python.org/pep-0771/).

Just a side note about PEP 771 - I'm hoping to put it to a decision by the new Python Packaging Committee once it is in place, so hopefully we should know one way or another later this year.

With respect to automating updates to the minimum allowed versions of dependencies, I did a beta release of bump-minimum-dependencies, which uses uv add to update pyproject.toml. Dependency groups and optional dependencies can be updated altogether or individually. It's possible to specify both a drop period (which defaults to 24 months per SPEC 0) and a cooldown period (which defaults to 12 months), and to skip certain dependencies from the updates.

It's not production ready since I need to add more test pyproject.toml files (like Astropy's!) and address a few issues and edge cases (like markers).

To use it with uvx, run:

uvx bump-minimum-dependencies

or alternatively

uvx bump-minimum-dependencies --drop-months=24 --cooldown-months=18 --all-extras --all-groups

I'm hoping to automate pull requests with these updates for PlasmaPy after I get back from visiting family over the coming week. Speaking of, I should probably leave for the airport... 😅

Note

With Astropy's pyproject.toml, uv<=0.12.3 shifts the position of an in-line comment, but that's been fixed for uv's next release.

Update: I've now done a v0.1.0 release of bump-minimum-dependencies, to be followed up shortly with v0.1.1. 😅 I'm using Astropy's pyproject.toml for two test cases.

Navigation Menu

Repository navigation

Astropy should endorse SPEC 0 #539

6 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Astropy should endorse SPEC 0 #539

Description

Activity

mhvk commented on Aug 4, 2026

neutrinoceros commented on Aug 4, 2026

mhvk commented on Aug 4, 2026

neutrinoceros commented on Aug 4, 2026

neutrinoceros commented on Aug 5, 2026

astrofrog commented on Aug 5, 2026

neutrinoceros commented on Aug 5, 2026

namurphy commented on Aug 5, 2026

neutrinoceros commented on Aug 5, 2026

namurphy commented on Aug 5, 2026

neutrinoceros commented on Aug 6, 2026

taldcroft commented on Aug 6, 2026

pllim commented on Aug 6, 2026

mhvk commented on Aug 6, 2026

6 remaining items

nstarman commented on Aug 7, 2026

neutrinoceros commented on Aug 7, 2026

nstarman commented on Aug 7, 2026

neutrinoceros commented on Aug 7, 2026

nstarman commented on Aug 7, 2026

mhvk commented on Aug 7, 2026

neutrinoceros commented on Aug 7, 2026

neutrinoceros commented on Aug 7, 2026

neutrinoceros commented on Aug 7, 2026

mhvk commented on Aug 7, 2026

neutrinoceros commented on Aug 8, 2026

neutrinoceros commented on Aug 8, 2026

mhvk commented on Aug 8, 2026

astrofrog commented on Aug 8, 2026

namurphy commented on Aug 10, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions