You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There is a PEP proposed for deprecating distutils. Meson uses distutils in the python module for, if I am correct, building extension modules, so this PEP may be of interest to you.
Note that in #9288 we added an additional reliance on distutils (in the same exact introspection command that already imports distutils). In addition to detecting whether to link against libpython, we now also use distutils to detect whether sysconfig is completely crippled by the vendor and unable to tell the truth about install paths.
Handling libpython linking could be done by a try/except that reacts to an ImportError by saying "clearly this must be sys.version_info >= (3, 8) so the answer is don't link".
Handling debian paths cannot be resolved other than by waiting on debian to fix things.
As we get closer to the final removal of distutils in 3.12 we will need to both consider that try/except for libpython, and keep an eye on the debian situation.
$ meson setup build --prefix=/home/tcaswell/.virtualenvs/bleeding/lib/python3.12/site-packages
The Meson build system
Version: 0.64.0
Source dir: /home/tcaswell/source/p/numpy/numpy
Build dir: /home/tcaswell/source/p/numpy/numpy/build
Build type: native build
Project name: test
Project version: 0
C compiler for the host machine: ccache cc (gcc 12.2.0 "cc (GCC) 12.2.0")
C linker for the host machine: cc ld.bfd 2.39.0
Host machine cpu family: x86_64
Host machine cpu: x86_64
meson.build:13:0: ERROR: <PythonExternalProgram 'python3' -> ['/home/tcaswell/.virtualenvs/bleeding/bin/python3']> is not a valid python or it is missing distutils
A full log can be found at /home/tcaswell/source/p/numpy/numpy/build/meson-logs/meson-log.txt
FYI, there's been a migration among packaging tools to use the standalone, modern sysconfig stdlib module over distutils' equivalent, which mesonpy's @FFY00 has led the charge on. Perhaps he has some insightful comments on that here?
We already use sysconfig for most things. There is:
one thing that sysconfig doesn't currently provide -- I've been meaning to open a CPython ticket to have it added (I mentioned it in passing in a meson-python ticket somewhere) which I will try doing soon
one thing that sysconfig does provide, but Debian has frustratingly broken, which I need to investigate whether it works now.
I'll try to see if I can get news on both counts in the near future. Because one of them is totally dependent on Debian's timeframe and they already know about the problem but have been flatly ignoring it for years, I've been letting the general issue slide, hoping that if I check in on it closer to the cutoff point it will finally resolve itself. But I didn't think about people using the alphas for 3.12 a year before the cutoff point. :)
I see, thanks. In that case, I'd suggest opening a CPython issue for the former as soon as possible (and the latter, if there's anything needed on the upstream end), to allow plenty of time before the beta freeze in less than 6 months for discussion, implementation, review and merging, and preferably allow you, distros and others to test with it.
I've done some more archeology. The sysconfig patch is in the repository for the Debian's Python package since a long time, but it is actually installed only starting with Python 3.10, in Debian testing. The only thing that can be done is to check the Python version ad fall back to distutils for Python versions earlier than 3.10.
I've done some more archeology. The sysconfig patch is in the repository for the Debian's Python package since a long time, but it is actually installed only starting with Python 3.10, in Debian testing. The only thing that can be done is to check the Python version ad fall back to distutils for Python versions earlier than 3.10.
Activity
lazka commentedon Sep 6, 2020
distutils is only used to decide if extensions should link against libpython to follow the platform defaults (Debian patched to not link, default was to link). With Python 3.8 upstream Python default changed to not link to libpython by default, so if distutils ever gets removed, meson can just default to not linking always: https://docs.python.org/3/whatsnew/3.8.html#debug-build-uses-the-same-abi-as-release-build
eli-schwartz commentedon Oct 3, 2021
Note that in #9288 we added an additional reliance on distutils (in the same exact introspection command that already imports distutils). In addition to detecting whether to link against libpython, we now also use distutils to detect whether sysconfig is completely crippled by the vendor and unable to tell the truth about install paths.
Handling libpython linking could be done by a try/except that reacts to an ImportError by saying "clearly this must be
sys.version_info >= (3, 8)so the answer is don't link".Handling debian paths cannot be resolved other than by waiting on debian to fix things.
As we get closer to the final removal of distutils in 3.12 we will need to both consider that try/except for libpython, and keep an eye on the debian situation.
tacaswell commentedon Nov 14, 2022
distutilshas now been pulled out of the main branch of CPython and meson is failing to detect it as a valid Python:adapted from numpy's meson branch:
CAM-Gerlach commentedon Nov 15, 2022
FYI, there's been a migration among packaging tools to use the standalone, modern
sysconfigstdlib module over distutils' equivalent, which mesonpy's @FFY00 has led the charge on. Perhaps he has some insightful comments on that here?eli-schwartz commentedon Nov 15, 2022
We already use sysconfig for most things. There is:
I'll try to see if I can get news on both counts in the near future. Because one of them is totally dependent on Debian's timeframe and they already know about the problem but have been flatly ignoring it for years, I've been letting the general issue slide, hoping that if I check in on it closer to the cutoff point it will finally resolve itself. But I didn't think about people using the alphas for 3.12 a year before the cutoff point. :)
CAM-Gerlach commentedon Nov 16, 2022
I see, thanks. In that case, I'd suggest opening a CPython issue for the former as soon as possible (and the latter, if there's anything needed on the upstream end), to allow plenty of time before the beta freeze in less than 6 months for discussion, implementation, review and merging, and preferably allow you, distros and others to test with it.
tacaswell commentedon Nov 16, 2022
The suggestion of installing a setuptools 60+ worked for getting meson to work with py312.
dnicolodi commentedon Nov 16, 2022
It seems that recent Debian patches
sysconfigtoo now https://salsa.debian.org/cpython-team/python3/-/blob/master/debian/patches/sysconfig-debian-schemes.diff I'll have a look at when this was introduced. If it is old enough (how old?) we can probably rely on it. Otherwise we will need to make the checks conditional. I'll see what can be done.Edit: it looks like the
sysconfigpatching goes way back to Python 3.7 https://salsa.debian.org/cpython-team/python3/-/blob/python3.7/debian/patches/sysconfig-debian-schemes.diff I think we can probably safely rely on it. I'll prepare a PR and see what the CI tests think about it.dnicolodi commentedon Nov 16, 2022
It would be great to have some input from the Debian Python maintainers on this.
dnicolodi commentedon Nov 16, 2022
Probably worth checking https://www.debian.org/doc/packaging-manuals/python-policy/
dnicolodi commentedon Nov 16, 2022
I've done some more archeology. The
sysconfigpatch is in the repository for the Debian's Python package since a long time, but it is actually installed only starting with Python 3.10, in Debian testing. The only thing that can be done is to check the Python version ad fall back todistutilsfor Python versions earlier than 3.10.The other
distutilsis used for is this:meson/mesonbuild/modules/python.py
Lines 366 to 370 in 47d1a55
FFY00 commentedon Nov 17, 2022
See https://ffy00.github.io/blog/02-python-debian-and-the-install-locations/ if you want some more archeology 😛
python/cpython#88611 is the proposed fix.
dnicolodi commentedon Nov 17, 2022
IIUC, the linked issue is about where to find the headers, not about whether to link or not with libpython. Aren't these separate issues?
FFY00 commentedon Nov 17, 2022
Yes, sorry, I misread it.
22 remaining items
python module: stop using distutils "link to libpython" probe on rece…
Add setuptools as a host dep to make Meson happy
Add setuptools as a host dep to make Meson happy
Add setuptools as a host dep to make Meson happy
python module: stop using distutils schemes on sufficiently new Debian
python module: stop using distutils "link to libpython" probe on rece…
python module: stop using distutils schemes on sufficiently new Debian
python module: stop using distutils "link to libpython" probe on rece…
python module: stop using distutils schemes on sufficiently new Debian
python module: stop using distutils "link to libpython" probe on rece…
python module: stop using distutils schemes on sufficiently new Debian
python module: stop using distutils "link to libpython" probe on rece…
python module: stop using distutils schemes on sufficiently new Debian
python module: stop using distutils "link to libpython" probe on rece…