Repository navigation
[proposal] Switch from tox to Nox for running continuous integration environments #16412
Description
Astropy uses tox to run the environments for its continuous integration checks. While tox is in most ways a fantastic tool, its main drawback is that the configuration file (tox.ini) is an INI file. I personally find tox.ini files to be quite difficult to understand, modify, and maintain.
Nox is a project that is similar to tox, except that the configuration file is written in Python. 🐍 I'm in the process of switching PlasmaPy's tox environments to Nox sessions. I've been finding it significantly easier to set up and understand sessions in noxfile.py compared to tox.ini. I propose that Astropy switches its test runner from tox to Nox.
For some background, I highly recommend Hynek Schlawack's blog post on "Why I Like Nox", and trying out Nox for new or personal projects.
Example noxfile.py
import nox
@nox.session
def tests(session):
"""Run tests with pytest."""
session.install('pytest')
session.run('pytest')This file defines the tests session, which can be run with nox -s tests. (Vocabulary difference: a tox environment corresponds to a Nox session.)
Tradeoffs ⚖️
Here are some of the pros and cons of making the switch.
✅ Reasons to switch to Nox
- The configuration file would be written in Python! 🎉🎂💖🎆🪩🥦🐍
- Sessions would be more maintainable in the long term because the configuration file would be written in the language that Astropy contributors are most likely to understand. 🌌
- Sessions could make use of Python functionality. 🐍
- Sessions could be generated dynamically. 🌠
- Astropy contributors would no longer need to understand the INI format, since the repository would contain no more
.inifiles. ✨ - Nox is being actively developed.
- Writing a Nox session tends to be quicker than writing a tox environment, in my experience. ⏱️
- Switching to Nox would likely save developer time in the long run while improving developer experience. ⌛
❌ Reasons to keep using tox
- Switching to Nox would require perhaps a week of someone's time. This would include converting tox environments to Nox sessions, updating GitHub Actions, revising the relevant sections of the development guide, and resolving any unexpected issues that may arise.
- Switching to Nox would require having to fully understand the contents of
tox.ini. 😐 - tox is currently doing its job satisfactorily, with enough Astropy contributors able to maintain it. 👥
- tox is an actively developed, well-established, and widely used project.
- Switching to Nox could disrupt some user workflows.
- Switching to Nox would lead to differences with packages whose architecture is inspired by Astropy's.
Suggested process for switching
I recommend taking an incremental process rather than doing a monolithic PR:
- Convert a subset of existing tox environments into Nox sessions in one PR (e.g., just the docs environments).
- Update GitHub workflows to use the Nox sessions in a follow-up PR
- Repeat.
There is a tool called tox-to-nox that can help convert tox environments to Nox sessions, though I haven't tried it yet.
Activity
namurphy commentedon May 7, 2024
One moderately nonsensical follow-up point: I'd say that nox is more astrophysically relevant than tox because$\mathrm{NO}_x$ chemicals have been observed in the ISM! 🌌 Nitric oxide was even astromolecule of the month in July 2007.
neutrinoceros commentedon May 8, 2024
Any ideas in particular ?
My understanding of tox is that is already allows defining sessions by composition, which could be argued is the only kind of dynamism that we need. I'm personally worried that introducing more dynamism would inflate the maintenance burden, not reduce it.
This seems like a very important point. I don't think we want to alienate the ecosystem that bloomed from astropy, and there's been a consistent effort to keep workflows as similar as possible between repos. Moving astropy away from tox would likely mean that we'd need to help coordinated and affiliated packages do the same, and it sounds like a sizable effort.
astrofrog commentedon May 8, 2024
I'm against this for a couple of reasons 👎
setup.pyto putting as much as possible insetup.cfgand thenpyproject.tomlbecause it is much nicer and more maintainable to define this in a declarative file with a very specific syntax. It limits feature creep and also makes it possible for the configuration to be machine-readable (for instance if we ever want to do any automated updates across the astropy ecosystem). I occasionally come across packages that still have large and complexsetup.pyfiles and it is a bit of a nightmare to navigate these - I fear the same would happen here.neutrinoceros commentedon May 8, 2024
That's also what I meant in my first point, and I fully agree with your reasoning. Thanks for articulating it more clearly than I did 😄
namurphy commentedon May 8, 2024
Thank you for the replies — they are very helpful and insightful, and definitely aspects we need to consider!
For PlasmaPy, I'm considering dynamically creating nox sessions that run the tests for each subpackage. For example, doing
nox -s particleswould let me run the tests and doctests forplasmapy.particles.I'm currently switching over to a nox session that regenerates pinned requirements files used in CI. It's really helpful to be able to use a
forloop! I'm also adding a mildly helpful function to generates the names of the requirements files, which I'm likely to reuse in testing environments. This is in PlasmaPy/PlasmaPy#2664.I've previously considered dynamically extracting the minimum requirements from
pyproject.tomlto run tests against minimum requirements, eliminating the need to define them in bothpyproject.tomlandtox.ini. (This is no longer necessary, asuvhas a--resolution=lowest-directflag to do this for us.)I suspect that the amount that this effect would increase the maintenance burden would be more than offset by the reduction in maintenance burden by switching from an INI format to a Python API. I'm curious how the maintenance burden was impacted in other projects that have made the switch from tox → nox.
Agreed; to me this is the most compelling of the reasons not to switch. As with everything in software engineering, there are tradeoffs! ⚖️ It's difficult to find a balance between consistency & stability of package infrastructure, and adoption of new tools that can improve developer experience.
Thank you again!
eerovaher commentedon May 9, 2024
There is a high enough probability that Astral releases something in the near future that we might want to use instead of
tox, so I view the idea of replacingtoxwithnoxright now as a needless distraction.astrofrog commentedon May 9, 2024
FYI this currently works with tox by doing e.g.
which will run all tests in astropy.io.fits.
I would say our maintenance burden for using tox.ini is very little right now - I haven't seen any real complaints about it in the last couple of years or things we have spent significant time to work around, but I might be wrong. Tox has also got a lot better with tox 4 as
tox --developis now nice and speedy.neutrinoceros commentedon May 9, 2024
Were there any hint specifically in that direction ?
eerovaher commentedon May 9, 2024
toxis one of the tools they mention in a blog post. That's not saying very much, but it does qualify as a hint.nstarman commentedon May 19, 2024
I'm also on the nox bandwagon. I switched recently when I started using https://github.com/scientific-python/cookie to set up my projects. We don't bake a cookie ourselves anymore, and I think this is the one we do / should recommend people use. Anyway, https://github.com/scientific-python/cookie uses
noxand IMO it's much more pleasant than tox thus far.So yes, we would have differences with packages whose architecture was inspired by Astropy's, but we would a) hopefully inspire them to move towards the https://github.com/scientific-python package layout and b) Astropy would be more similar to packages whose architecture is inspired by the scientific python community.
[-][proposal] Switch from tox to nox for running continuous integration environments[/-][+][proposal] Switch from tox to Nox for running continuous integration environments[/+]astrofrog commentedon May 20, 2024
Just to check, does anyone here have concrete examples of issues with the current tox set-up that would actually be solved by switching to nox? How often do people even have to delve into the current tox config?
bsipocz commentedon Aug 20, 2025
What about closing this one? If I count correctly, this idea hasn't gather enough support or in fact any strong reasons for the switch from the currently working approach.
astrofrog commentedon Aug 20, 2025
Agree it makes sense to close this
pllim commentedon Aug 22, 2025
Thanks for the suggestions anyway, @namurphy ! It was much appreciated that you took the time to write up such details and brought in your expertise from plasmapy. We can always reopen in the future if interest changes.
nstarman commentedon Aug 22, 2025
@namurphy do you know if nox supports running tests in parallel yet? I recall this being a blocker.