Remove setup.py file - #14361

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

• Do the proposed changes actually accomplish desired goals?
• Do the proposed changes follow the Astropy coding guidelines?
• Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
• Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
• Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see "When to rebase and squash commits".
• Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
• Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
• Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
• Is a milestone set? Milestone must be set but we cannot check for it on Actions; do not let the green checkmark fool you.
• At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

👋 Thank you for your draft pull request! Do you know that you can use [ci skip] or [skip ci] in your commit messages to skip running continuous integration tests until you are ready?

If this works, it would be great. In personal projects I only have a pyproject.toml unless I need mypyc compilation. Modernizing and simplifying the Astropy infrastructure is always a +1 from me!

Certainly a long time since I used it. My sense would be to make this a totally minimal setup.py which emits a deprecation warning (for 5.3 and perhaps also for the next LTS, 6.0), and then we remove it after.

Did you comb the docs and remove the defunct python setup.py build_ext --inplace ?

Not yet, I was waiting to see if there were any objections to the idea.

(I can try and determine the first version that worked if we think we might want to go ahead).

We had a similar issue with one of our packages lately, and I had a look. It should be 21.3 (released 2021-10).

Maybe should sent a memo about this out to astropy-dev list, just in case. Also cc @olebole in case this affects Debian.

@astrofrog how difficult would it be to update extension-helpers so that it can read the configuration from the pyproject.toml file directly rather than from setup.cfg? I think it would be nice if we could get rid of setup.cfg completely.

@plim thanks to ask me! In the moment, setup.py-less packages are a bit new for us; however we will be in a freeze very soon (Feb 12) and therefore take 5.2.1 for the next Debian release (unless there is an RC issue with that). Once we released that (summer), we will have enough time to to sort out any difficulties. So, I am fine with that change, as long as it is not backported to 5.2.x (but even then I could probably revert that change if needed).

@WilliamJamieson yes should be doable - want to try and open a PR to extension-helpers? 😀

So it turns out there is already a deprecation warning currently when one tries to use setup.py:

/Users/tom/python/dev/lib/python3.11/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.

This warning has been visible since setuptools 58.3 (Oct 2021), however that is for install and not build_ext which doesn't show a deprecation warning yet.

The 3.11 CI failure will be fixed by a rebase.

Losing access to setup.py build_ext would be annoying so I'm -1 on that. build_ext has options that are sometimes useful (--debug, --cython-gdb) and pip doesn't have a way to pass those options to build_ext. Also when developing a C/Cython extensions, using build_ext is much faster than pip (which recreates the isolated env), and build_ext works without an internet connection whereas pip needs to ping pypi even if packages are in cache.
Also setup.py install is deprecated but other subcommands are not (there is no alternative yet, not sure what the long term plan is).

Losing access to setup.py build_ext would be annoying so I'm -1 on that. build_ext has options that are sometimes useful (--debug, --cython-gdb) and pip doesn't have a way to pass those options to build_ext.

Out of curiosity, is this not supported by build's --config-setting flag?

Also when developing a C/Cython extensions, using build_ext is much faster than pip (which recreates the isolated env), and build_ext works without an internet connection whereas pip needs to ping pypi even if packages are in cache. Also setup.py install is deprecated but other subcommands are not (there is no alternative yet, not sure what the long term plan is).

When I do:

python -m build --no-isolation -x -w

the rebuilds are fast. I wonder if this can reasonably closely resemble your current workflow?

If you specifically want the traditional pip install -e . developer mode, I'm not sure whether there's a way to easily update the binaries, since the model is changing (in many/all? tools) to one where build products live outside of the source tree.

E.g., from the setuptools manual:

The editable term is used to refer only to Python modules inside the package directories. Non-Python files, external (data) files, executable script files, binary extensions, headers and metadata may be exposed as a snapshot of the version they were at the moment of the installation.

Would be wonderful to have the various quicker routes to building documented somewhere! I had not heard of build before (but obviously that was just an apt-get away).

Would be wonderful to have the various quicker routes to building documented somewhere! I had not heard of build before (but obviously that was just an apt-get away).

I agree! This is new, and I'm also still learning the incantations. I suspect @lwasser and team at PyOpenSci will have this written up as part of their packaging guide, but I'll let her confirm.

I should also mention that this is the prime incentive behind https://github.com/scientific-python/devpy — to make these commands discoverable, as is appropriate for the project.

Would be wonderful to have the various quicker routes to building documented somewhere! I had not heard of build before (but obviously that was just an apt-get away).

I agree! This is new, and I'm also still learning the incantations. I suspect @lwasser and team at PyOpenSci will have this written up as part of their packaging guide, but I'll let her confirm.

Yes we are working on a packaging guide and i'm not sure if i explicitly mention build there yet aside from a short example of using setuptools. But also we are working on instructions in this repo which will have tutorials eventually for a single workflow. The example that i linked to shows using pdm and meson_python (it is NOT fully fleshed out yet) but look at the readme. Build just provides a front end interface to call a backend. perhaps we also include devpy in the future too @stefanv ?!

i will try to flesh out the setuptools readme file in the near future. But if there are specific things that we can add sooner rather than later to help with this transition away from setup.py (as it makes sense) please feel free to tell us what would be most helpful!

And i'll try to beef up the section that talks about the setuptools/ build interface as well.

Thanks! I should speak mostly for myself, but what would be really useful as a developer is to have short-cuts that avoid having to create the whole package on repeat if one is just working on a single python or C file. Currently, since in astropy we use tox I tend to do something like

tox -e test --develop --notest
source .tox/test/bin/activate
pytest ...

but this is good for changes to python files only.

I guess bottom line is to try to make it easy and understandable both for end-users who need one-off installs, and developers who need a quick edit-test cycle.

p.s. Of course, way worse than code is editing documentation -- sphinx runs are just painful on larger projects...

If you are willing to go that route, I can recommend Meson. We switched scikit-image (just did our 0.20 release), and so far it's been working very well.

I'd be happy to lend a hand to get you going.

Yes, I saw numpy is switching to meson too. From the bit of a look I had at it, it seems quite nice & logical.

Yeah, this is not gonna happen in 3 days, moving milestone.

It looks like the current recommendation is either to keep a minimal setup.py (with most metadata in pyproject.toml), or to switch to an entirely new build system like Meson.

Meson supports having most info in the pyproject.toml, right 🤞, so either way we should continue migrating?

Indeed, meson-python post-dates PEP 621 and never implemented its own unique metadata format. :)

We discussed this briefly again at infrastructure tag-up on 2023-04-25. @saimn , @tomdonaldson , and I are of the thought that maybe we can make incremental progress on this by first doing the setup.cfg -> pyproject.toml without changing anything else. And then this can build on that work at a later time.

Except for flake8 which probably would need its own .flake8 without setup.cfg. There were objections in hiding flake8 settings in tox.ini earlier.

@astrofrog , are you interested to pursue this?

I like everything about that suggestion except .flake8. Now that ruff supersedes all flake8 rules I think we should get rid of it. Short of that, for people who still want flake8 help in their code editors, we can hide the configuration somewhere. It's vestigial.

.flake8 is exactly how you "hide the configuration somewhere". It's a dotfile that doesn't clutter up an unrelated file like setup.cfg or tox.ini.

I do agree that if one commits to using ruff then flake8 seems altogether redundant, unless you're using flake8 plugins that ruff doesn't happen to implement.

I have flake8 hooked up to Sublime, VS Code, and Emacs on Linux and Windows. Dunno how to do that with ruff.

@pllim I only use VS Code, which has builtin support, so I haven't experimented with setting it up, but https://beta.ruff.rs/docs/editor-integrations/#language-server-protocol-official should work.

I still think it would be less controversial to just live with a .flake8 for now. We can debate ruff vs flake8 in a separate issue, since that is completely unrelated to the build machinery anyway.

Agreed with .flake8 - I do have ruff & black working under emacs; hope to update the emacs documentation at some point...

Agreed with .flake8 - I do have ruff & black working under emacs; hope to update the emacs documentation at some point...

(use-package elpy
  :commands elpy-enable
  :config
  (setq elpy-syntax-check-command "pychecker.sh")
  (setq python-check-command "pychecker.sh")
  (setq elpy-rpc-virtualenv-path 'current)
)

Then put whatever you like in pychecker.sh. Ironically, I now see that my pychecker.sh has commands from... astropy!

We use pre-commit on many projects; I know some developers can't stand it, but in my experience it's been entirely pain-free. (SciPy devs didn't like it, so @eli-schwartz and I implemented a more secure pre-commit lookalike).

FWIW the pre-commit.ci bot has been invaluable to us in workflows particularly if you have beginner contributors. you can just run a linting step prior to merging a PR and save them the hassle of local setup. i'd definitely consider creating a linting discussion around this to talk about your needs. move the discussion to a separate PR / issue item!

We typically just run pre-commit as a step before the test suite. Is there an advantage to the bot?

So that valuable discussion don't get lost in an unrelated PR, can you please open new issue w.r.t. pre-commit? Thanks!

The bot is already configured for this repository, it just has autofix-prs set to "false", which is an opinion that some projects have. ;)

Back to possible alternatives, there is also https://github.com/scikit-build/scikit-build-core with cmake . cc @henryiii

Hi humans 👋 - this pull request hasn't had any new commits for approximately 4 months. I plan to close this in 30 days if the pull request doesn't have any new commits by then.

In lieu of a stalled pull request, please consider closing this and open an issue instead if a reminder is needed to revisit in the future. Maintainers may also choose to add keep-open label to keep this PR open but it is discouraged unless absolutely necessary.

If this PR still needs to be reviewed, as an author, you can rebase it to reset the clock.

If you believe I commented on this pull request incorrectly, please report this here.

@astrofrog how difficult would it be to update extension-helpers so that it can read the configuration from the pyproject.toml file directly rather than from setup.cfg? I think it would be nice if we could get rid of setup.cfg completely.

@WilliamJamieson - extension-helpers v1.1.0 is now out with your addition for being able to read the option from pyproject.toml

I'm going to close this pull request as per my previous message. If you think what is being added/fixed here is still important, please remember to open an issue to keep track of it. Thanks!

If this is the first time I am commenting on this issue, or if you believe I closed this issue incorrectly, please report this here.