Repository navigation
REGR: cross-compilation fails for osx-arm64 with meson 1.5.0 #13403
Description
Describe the bug
A previously (with meson 1.4.2) working build for osx-arm64 broke with the release of meson 1.5.0, apparently trying to run sanitycheckc_cross.exe with a wrong architecture
The Meson build system
Version: 1.5.0
Source dir: /Users/runner/miniforge3/conda-bld/matplotlib-suite_1720666814590/work
Build dir: /Users/runner/miniforge3/conda-bld/matplotlib-suite_1720666814590/work/builddir
Build type: cross build
Program python3 found: YES (/Users/runner/miniforge3/conda-bld/matplotlib-suite_1720666814590/_build_env/bin/python3)
Project name: matplotlib
Project version: 3.9.1
meson.build:1:0: ERROR: Could not invoke sanity test executable: [Errno 86] Bad CPU type in executable: '/Users/runner/miniforge3/conda-bld/matplotlib-suite_1720666814590/work/builddir/meson-private/sanitycheckc_cross.exe'.
Note that we're cross-compiling three different architectures (the other two being linux-aarch64 and linux-ppc64le, both compiled from linux-64), and osx-arm64 is the only one that broke.
More specifically, the cleaned-up build invocation looks as follows
mkdir builddir
$PREFIX/bin/python -m mesonbuild.mesonmain setup builddir \
--buildtype release --prefix=$PREFIX -Dlibdir=lib \
--cross-file $BUILD_PREFIX/meson_cross_file.txt \
--pkg-config-path=$PREFIX/lib/pkgconfig \
-Dsystem-freetype=true -Dsystem-qhull=true
and the contents of meson_cross_file.txt are:
[host_machine]
system = 'darwin'
cpu = 'arm64'
cpu_family = 'aarch64'
endian = 'little'
[binaries]
cmake = '$BUILD_PREFIX/bin/cmake'
pkg-config = '$BUILD_PREFIX/bin/pkg-config'To Reproduce
Build the recipe in https://github.com/conda-forge/matplotlib-feedstock
Reverting back to meson <1.5 causes the build to succeed.
Expected behavior
Build succeeds
system parameters
- Is this a cross build or just a plain native build (for the same computer)?
- cross build
- what operating system (e.g. MacOS Catalina, Windows 10, CentOS 8.0, Ubuntu 18.04, etc.)
- build environment is macOS-12 image (x64) from azure, target is macOS>=11.0 (arm64)
- what Python version are you using e.g. 3.8.0
- 3.9-3.12
- what
meson --version- 1.5.0
- what
ninja --versionif it's a Ninja build- 1.12.1
Activity
eli-schwartz commentedon Jul 11, 2024
Would you be able to bisect this by installing different git commits of meson into that feedstock? Nothing obviously jumps out at me as related to this, except maybe c107624 (although that commit does look suspiciously related ;)).
h-vetinari commentedon Jul 11, 2024
Your suspicion was spot on. I tried this in conda-forge/matplotlib-feedstock#398:
[-]REGR: cross-compilation fails on osx-arm64 with meson 1.5.0[/-][+]REGR: cross-compilation fails for osx-arm64 with meson 1.5.0[/+]eli-schwartz commentedon Jul 11, 2024
Thanks for verifying that. @xclaesse as the author of that commit, can you take a look at this?
(It is bedtime now so that's all the debugging I will be doing for now.)
h-vetinari commentedon Jul 11, 2024
Originally I worried that this would have a huge blast radius for conda-forge (well, for those packages being built by meson at least), as that's one of our main architectures, and we only cross-compile it (due to lack of native agents in azure).
I restarted conda-forge/scipy-feedstock#269 1, and to my surprise, this doesn't run into the same issue:
One difference that stands out is
--native-file, though I don't understand under what conditions this is created; it's not part of the build script, and it doesn't seem to be created in scipy itself either (though there's across-compiledsetting in__config__.py.in). CC @rgommersconda-forge/scikit-learn-feedstock#270 similarly passes (and sets
--native-file). So it doesn't seem to affect everything, but I don't understand the conditions under which failure happens or not.Footnotes
originally restricted to windows as it was meant only for debugging add initial support for llvm-flang #13323 ↩
rgommers commentedon Jul 11, 2024
--native-fileis the setup argument coming frommeson-python, to ensure that the native Python interpreter Meson picks is actually the right one (i.e., the one in the environment for whichmeson-pythonitself is installed). It's always passed when you build an sdist or wheel withmeson-python.rgommers commentedon Jul 11, 2024
@h-vetinari this line is wrong: https://github.com/conda-forge/matplotlib-feedstock/blob/414fc00e6cde964337d84b5234673a647cc22c00/recipe/build_base.sh (both because
meson-pythonmay redo the configure stage, and because invokingpython -m mesonbuild.mesonmainisn't supported by Meson AFAIK). If you adapt it to what's used in scipy-feedstock, that may help.h-vetinari commentedon Jul 11, 2024
Thanks a lot @rgommers; I fixed the double-execution in conda-forge/matplotlib-feedstock#398 (now just going through meson-python as it should), but the issue persists with 1.5.0, and in fact the same result for the specific commits as above (links to CI logs have been updated below):
Revert various cross compile changes
lazka commentedon Jul 11, 2024
Here is a potential fix: #13411
h-vetinari commentedon Jul 11, 2024
Thanks a lot @lazka! 🙏
I've tested your PR in conda-forge/matplotlib-feedstock#398 and it works again! 🥳
Revert various cross compile changes
h-vetinari commentedon Jul 21, 2024
Is there any approximate timeline for a 1.5.1 release?
eli-schwartz commentedon Jul 21, 2024
There are two more bugs open on the 1.5.1 milestone, one of them is going to be investigated and hopefully fixed tomorrow. I'd like to get either or both of them fixed in the next couple of days, then release 1.5.1.
Revert various cross compile changes
lazka commentedon Jul 27, 2024
@h-vetinari 1.5.1 is out
h-vetinari commentedon Jul 28, 2024
I saw, thanks a lot for everyone's effort!
Revert various cross compile changes
Revert various cross compile changes