Thanks. The compiler ID is a class variable, it's not derived automatically from the class name. So we should probably do that.

This will close gh-8113.

Review comments on gh-9850 (a previous, simpler, attempt to add OneAPI compiler support) suggested using a compiler ID of oneapi or intel-oneapi (rather than IntelLLVM), and also adding -fp-mode=precise as the default.

I'm personally not so sure adding default flags like that is a good idea. I agree that it'd have been much better for Intel to not default to fastmath-like behavior but instead have defaults compatible with GCC and Clang behavior. However, for developers that do want the Intel compiler's default, it's not great if a build system overrides it.

I added a compiler id of intel-llvm. I did not make it intel-oneapi because oneapi is a brand, and marketing can change branding from time to time. The compiler id will be in end user files and not easy to change.

I will change it to intel-oneapi if there is a strong preference for a oneapi name.

Are there restrictions on the Linux distros on which the compilers work? I tried this PR, and the Meson configure stage is not happy with my Arch Linux provided libm. I can reproduce it with this simple check:

$ cat sanitycheckc.c 
int main(void) { int class=0; return class; }

$ icx sanitycheckc.c -o sanitycheckc.exe
/home/rgommers/mambaforge/envs/scipy-dev/bin/ld: cannot find crtbegin.o: No such file or directory
/home/rgommers/mambaforge/envs/scipy-dev/bin/ld: /usr/lib/libm.so.6: unknown type [0x13] section `.relr.dyn'
/home/rgommers/mambaforge/envs/scipy-dev/bin/ld: skipping incompatible /usr/lib/libm.so.6 when searching for /usr/lib/libm.so.6
/home/rgommers/mambaforge/envs/scipy-dev/bin/ld: cannot find /usr/lib/libm.so.6
/home/rgommers/mambaforge/envs/scipy-dev/bin/ld: /usr/lib/libm.so.6: unknown type [0x13] section `.relr.dyn'
/home/rgommers/mambaforge/envs/scipy-dev/bin/ld: skipping incompatible /usr/lib/libm.so.6 when searching for /usr/lib/libm.so.6
/home/rgommers/mambaforge/envs/scipy-dev/bin/ld: /usr/lib/libmvec.so.1: unknown type [0x13] section `.relr.dyn'
/home/rgommers/mambaforge/envs/scipy-dev/bin/ld: skipping incompatible /usr/lib/libmvec.so.1 when searching for /usr/lib/libmvec.so.1
/home/rgommers/mambaforge/envs/scipy-dev/bin/ld: cannot find /usr/lib/libmvec.so.1
/home/rgommers/mambaforge/envs/scipy-dev/bin/ld: /usr/lib/libmvec.so.1: unknown type [0x13] section `.relr.dyn'
/home/rgommers/mambaforge/envs/scipy-dev/bin/ld: skipping incompatible /usr/lib/libmvec.so.1 when searching for /usr/lib/libmvec.so.1
icx: error: linker command failed with exit code 1 (use -v to see invocation)

icc has no issue.

The only supported linux distro's are listed here: https://www.intel.com/content/www/us/en/developer/articles/system-requirements/intel-oneapi-base-toolkit-system-requirements.html

However, many distro's work anyway. This works for me:

docker run --rm -it -v /opt/intel:/opt/intel archlinux:base-devel
[root@d4ec90d51f79 /]# cat > test.c
int main(void) { int class=0; return class; }
[root@d4ec90d51f79 /]# gcc test.c
[root@d4ec90d51f79 /]# source /opt/intel/oneapi/setvars.sh

:: initializing oneAPI environment ...
   bash: BASH_VERSION = 5.1.16(1)-release
   args: Using "$@" for setvars.sh arguments:
:: advisor -- latest
:: ccl -- latest
:: clck -- latest
:: compiler -- latest
:: dal -- latest
:: debugger -- latest
:: dev-utilities -- latest
:: dnnl -- latest
:: dpcpp-ct -- latest
:: dpl -- latest
:: inspector -- latest
:: intelpython -- latest
/opt/intel/oneapi/intelpython/latest/bin/python: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory
:: ipp -- latest
:: ippcp -- latest
:: itac -- latest
:: mkl -- latest
:: mpi -- latest
:: tbb -- latest
:: vpl -- latest
:: vtune -- latest
:: oneAPI environment initialized ::

[root@d4ec90d51f79 /]# icx test.c
[root@d4ec90d51f79 /]#

Thanks @rscohn2, that works for me too. The problem seems to be due to working in a conda environment - it ships a linker that's being very unhelpful here; I haven't seen that issue before. I'll see if I can convince Spack to build SciPy with the oneAPI compilers.

Okay, Spack is being more helpful here. To get past the configure stage of the SciPy build with this PR, the following change is needed:

diff --git a/mesonbuild/compilers/cpp.py b/mesonbuild/compilers/cpp.py
index 9159f5677..ca22b152f 100644
--- a/mesonbuild/compilers/cpp.py
+++ b/mesonbuild/compilers/cpp.py
@@ -155,7 +155,7 @@ class CPPCompiler(CLikeCompiler, Compiler):
         }
 
         # Currently, remapping is only supported for Clang, Elbrus and GCC
-        assert self.id in frozenset(['clang', 'lcc', 'gcc', 'emscripten', 'armltdclang'])
+        assert self.id in frozenset(['clang', 'lcc', 'gcc', 'emscripten', 'armltdclang', 'intel-llvm'])
 
         if cpp_std not in CPP_FALLBACKS:
             # 'c++03' and 'c++98' don't have fallback types

Then the build proceeds until it hits this:

[242/1579] Compiling Fortran object scipy/linalg/_flinalg.cpython-39-x86_64-linux-gnu.so.p/src_det.f.o
FAILED: scipy/linalg/_flinalg.cpython-39-x86_64-linux-gnu.so.p/src_det.f.o 
/home/rgommers/code/spack/lib/spack/env/oneapi/ifx -Iscipy/linalg/_flinalg.cpython-39-x86_64-linux-gnu.so.p -Iscipy/linalg -I../scipy/linalg -I/home/rgommers/code/spack/opt/spack/linux-endeavourosrolling-skylake_avx512/oneapi-2022.2.0/py-numpy-1.22.4-gowlw6cmt7kijoaybvnc5etgedf2ergq/lib/python3.9/site-packages/numpy/core/include -I/home/rgommers/code/spack/opt/spack/linux-endeavourosrolling-skylake_avx512/oneapi-2022.2.0/py-numpy-1.22.4-gowlw6cmt7kijoaybvnc5etgedf2ergq/lib/python3.9/site-packages/numpy/f2py/src -I/home/rgommers/code/spack/opt/spack/linux-endeavourosrolling-skylake_avx512/oneapi-2022.2.0/python-3.9.13-mn4ovd4dkbyg62orinqikrnvaxq6r5tf/include/python3.9 -I/opt/intel/oneapi/mkl/latest/lib/pkgconfig/../../include -I/opt/intel/oneapi/compiler/latest/lib/pkgconfig/../../linux/compiler/include -fvisibility=hidden -D_FILE_OFFSET_BITS=64 -warn general -warn truncated_source -stand=none -g -traceback -O2 -fPIC -DMKL_ILP64 -module scipy/linalg/_flinalg.cpython-39-x86_64-linux-gnu.so.p -gen-dep=scipy/linalg/_flinalg.cpython-39-x86_64-linux-gnu.so.p/src_det.f.o -gen-depformat=make -o scipy/linalg/_flinalg.cpython-39-x86_64-linux-gnu.so.p/src_det.f.o -c ../scipy/linalg/src/det.f
xfortcom: Unknown command line argument '-fvisibility'.  Try: '/opt/intel/oneapi/compiler/2022.2.0/linux/bin-llvm/xfortcom --help'
xfortcom: Did you mean '--ddg-simplify'?
xfortcom: Unknown command line argument 'hidden'.  Try: '/opt/intel/oneapi/compiler/2022.2.0/linux/bin-llvm/xfortcom --help'
compilation aborted for ../scipy/linalg/src/det.f (code 1)
[243/1579] Compiling Fortran object scipy/linalg/_flinalg.cpython-39-x86_64-linux-gnu.so.p/src_lu.f.o
FAILED: scipy/linalg/_flinalg.cpython-39-x86_64-linux-gnu.so.p/src_lu.f.o 
/home/rgommers/code/spack/lib/spack/env/oneapi/ifx -Iscipy/linalg/_flinalg.cpython-39-x86_64-linux-gnu.so.p -Iscipy/linalg -I../scipy/linalg -I/home/rgommers/code/spack/opt/spack/linux-endeavourosrolling-skylake_avx512/oneapi-2022.2.0/py-numpy-1.22.4-gowlw6cmt7kijoaybvnc5etgedf2ergq/lib/python3.9/site-packages/numpy/core/include -I/home/rgommers/code/spack/opt/spack/linux-endeavourosrolling-skylake_avx512/oneapi-2022.2.0/py-numpy-1.22.4-gowlw6cmt7kijoaybvnc5etgedf2ergq/lib/python3.9/site-packages/numpy/f2py/src -I/home/rgommers/code/spack/opt/spack/linux-endeavourosrolling-skylake_avx512/oneapi-2022.2.0/python-3.9.13-mn4ovd4dkbyg62orinqikrnvaxq6r5tf/include/python3.9 -I/opt/intel/oneapi/mkl/latest/lib/pkgconfig/../../include -I/opt/intel/oneapi/compiler/latest/lib/pkgconfig/../../linux/compiler/include -fvisibility=hidden -D_FILE_OFFSET_BITS=64 -warn general -warn truncated_source -stand=none -g -traceback -O2 -fPIC -DMKL_ILP64 -module scipy/linalg/_flinalg.cpython-39-x86_64-linux-gnu.so.p -gen-dep=scipy/linalg/_flinalg.cpython-39-x86_64-linux-gnu.so.p/src_lu.f.o -gen-depformat=make -o scipy/linalg/_flinalg.cpython-39-x86_64-linux-gnu.so.p/src_lu.f.o -c ../scipy/linalg/src/lu.f
xfortcom: Unknown command line argument '-fvisibility'.  Try: '/opt/intel/oneapi/compiler/2022.2.0/linux/bin-llvm/xfortcom --help'
xfortcom: Did you mean '--ddg-simplify'?
xfortcom: Unknown command line argument 'hidden'.  Try: '/opt/intel/oneapi/compiler/2022.2.0/linux/bin-llvm/xfortcom --help'
compilation aborted for ../scipy/linalg/src/lu.f (code 1)

-fvisibility=hidden comes from compilers/mixins/gnu.py. If ifx supports a visibility flag like that, I guess that needs to be wired in here.

I also tried adding the intel-llvm compilers to the detection logic:

diff --git a/mesonbuild/compilers/detect.py b/mesonbuild/compilers/detect.py
index 197fdb1c3..4d17fe6a1 100644
--- a/mesonbuild/compilers/detect.py
+++ b/mesonbuild/compilers/detect.py
@@ -68,11 +68,11 @@ else:
         defaults['objc'] = ['clang']
         defaults['objcpp'] = ['clang++']
     else:
-        defaults['c'] = ['cc', 'gcc', 'clang', 'nvc', 'pgcc', 'icc']
-        defaults['cpp'] = ['c++', 'g++', 'clang++', 'nvc++', 'pgc++', 'icpc']
+        defaults['c'] = ['cc', 'gcc', 'clang', 'nvc', 'pgcc', 'icc', 'icx']
+        defaults['cpp'] = ['c++', 'g++', 'clang++', 'nvc++', 'pgc++', 'icpc', 'icpx']
         defaults['objc'] = ['cc', 'gcc', 'clang']
         defaults['objcpp'] = ['c++', 'g++', 'clang++']
-    defaults['fortran'] = ['gfortran', 'flang', 'nvfortran', 'pgfortran', 'ifort', 'g95']
+    defaults['fortran'] = ['gfortran', 'flang', 'nvfortran', 'pgfortran', 'ifort', 'ifx', 'g95']
     defaults['cs'] = ['mcs', 'csc']
 defaults['d'] = ['ldc2', 'ldc', 'gdc', 'dmd']
 defaults['java'] = ['javac']

That didn't fix the -fvisibility issue, but seem needed anyway.

The ifx help says it support -fvisibility=hidden, but it does not work. I will check with a maintainer.

I added the suggested changes here and to the backport for spack: https://github.com/rscohn2/meson/tree/oneapi-backport

I filed a ticket about the -fvisibility issue and am still waiting to hear back about it.

Codecov Report

Merging #10909 (5ac0883) into master (942aea2) will decrease coverage by 2.34%.
The diff coverage is n/a.

❗ Current head 5ac0883 differs from pull request most recent head 14a9a36. Consider uploading reports for the commit 14a9a36 to get more accurate results

@@            Coverage Diff             @@
##           master   #10909      +/-   ##
==========================================
- Coverage   68.65%   66.30%   -2.35%     
==========================================
  Files         412      207     -205     
  Lines       87652    44858   -42794     
  Branches    19343     9261   -10082     
==========================================
- Hits        60175    29742   -30433     
+ Misses      22973    12806   -10167     
+ Partials     4504     2310    -2194     

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

I filed a ticket about the -fvisibility issue and am still waiting to hear back about it.

They are looking into it, but it is too late for the upcoming release. Will this block building scipy?

They are looking into it, but it is too late for the upcoming release. Will this block building scipy?

Yes, it looks like this will block building and Python extension modules with ifx, not just SciPy. Meson adds -fvisibility arguments to Python extensions, and icx/icpx seem happy with that while ifx is not. SciPy is by far the most prominent Python project which includes Fortran code though.

I wouldn't consider that blocking for this PR though - the changes here are still helpful.

When an internal build is available with the fix, I will try to continue testing. We built most of an e4s stack with ifx so the compiler looks solid.

@dcbaker : You are listed as a reviewer. Is there anything else I need to do?

This PR does not include installing and running tests for this compiler in CI. That's not necessarily a problem, as there's a bunch of compilers we don't test, particularly proprietary ones, and in general cross platform issues and per language Meson features are most important to test.

Indeed, as it happens in CI we really just test the default compilers per platform, plus a subset testing GCC vs. clang. It's always a question how much to test vs. how long the already lengthy CI should run... although I am minded that I recentlyish updated the azure pipelines jobs to install Intel ifort simply because we weren't testing fortran compilers on Windows at all.

In general for compiler support I think it's usually okay to expect the submitter to have tested and verified correctness. The most important thing for that, then, would be to run the testsuite with the appropriate compilers selected via e.g. CC=icx CXX=icpx FC=ifx ./run_tests.sh and check whether any test projects fail to work.

...

Now that I mention this, I realize that there's one more thing at least that should be updated, which is in run_project_tests.py, line 1058: skip_fortran = not(shutil.which('xxx') or shutil.which('yyy') or ...) can be updated to make sure the fortran test category gets run if oneapi is installed but e.g. gfortran is not.

On the topic of fortran and the -fvisibility flag, I have vague memories of discussing this for SciPy before, @rgommers, and basically getting the impression that fortran the language doesn't really support visibility attributes so as a result SciPy can't always avoid some pretty leaky symbol tables for compiled python modules.

Which makes me wonder, why this argument is passed to any fortran compiler even if said compiler allows it without complaint. Does it actually do anything?

Maybe Meson should just not do that?

This PR does not include installing and running tests for this compiler in CI.

I could add oneapi compiler here: https://github.com/mesonbuild/meson/blob/master/ci/ciimage/ubuntu-rolling/install.sh
and here:
https://github.com/mesonbuild/meson/blob/master/.github/workflows/os_comp.yml

My only concern is that the compiler on-disk space is 3Gbytes and whether that will be a problem. I usually run without a container and use github actions caching for the installed compiler. It takes < 1 minute to install from cache 2-3 minutes if not cached, and is only installed on the runner invocation that tests the intel compiler.

How would you want the intel compiler integrated into CI?

I ran test test system in the container. There are some failures.

• Rust does not compile. That is a known issue and likely a just something that requires upstream changes. Usually build scripts need some updates for an unrecognized compiler
• -fvisibility not supported
• ../test cases/frameworks/25 hdf5/main.f90(1): error #7013: This module file was not generated by any release of this compiler. [HDF5] use hdf5 Usually caused by improper mixing of gfortran/ifx modules

I don't see anything that looks like an issue with meson support for oneapi.

not sure if this is intended to address the issue on windows at all but just noting that I still get this error with your branch:

#8113 (comment)

I just tried with icx Version 2022.2.0 Build 20220730 (latest OneAPI base toolkit)

I still get this error with your branch

I only attempted to add linux support because I only do development on Linux. I tried it out on windows and it looks like I only had to update the compiler detector. Please try again with the fix I just pushed.

@rscohn2 it worked now (both the compiler and linker got detected after setting CXX to icx) thanks!

Does someone still have issues they want changed or is this ready to merge?

No more changes coming from me.

This PR LGTM

Which makes me wonder, why this argument is passed to any fortran compiler even if said compiler allows it without complaint. Does it actually do anything?

Maybe Meson should just not do that?

That previous discussion was scipy/scipy#15996 (comment). I think we should indeed remove -fvisibility for Fortran, and that will then make ifx work as well.

I added the intel-llvm-cl id for windows and squashed into 1 commit. Please merge if you are ok with it. Thanks.

Minor linting issues need fixing.

Also this now conflicts.

I corrected the lint errors and undid padding the reference table.

Test failure is unrelated.