Skip to content

Numpy: Ensure Spack finds Numpy's headers.#22475

Closed
RemiLacroix-IDRIS wants to merge 1 commit intospack:developfrom
RemiLacroix-IDRIS:py-numpy
Closed

Numpy: Ensure Spack finds Numpy's headers.#22475
RemiLacroix-IDRIS wants to merge 1 commit intospack:developfrom
RemiLacroix-IDRIS:py-numpy

Conversation

@RemiLacroix-IDRIS
Copy link
Copy Markdown
Contributor

The mpi4py package does the same thing.

@adamjstewart
Copy link
Copy Markdown
Member

@becker33 @scheibelp I wonder if we want to do this globally for all Python packages. The headers are almost always installed in prefix/lib/pythonX.Y/site-packages, not in prefix/include.

@becker33
Copy link
Copy Markdown
Member

@adamjstewart would it make more sense to use find_headers with that directory name, or this method using find_all_headers?

@adamjstewart
Copy link
Copy Markdown
Member

The problem is we need to recursively search both prefix.lib and prefix.lib64.

@RemiLacroix-IDRIS
Copy link
Copy Markdown
Contributor Author

@adamjstewart : Any news about this? Was it fixed globally?

@adamjstewart adamjstewart self-assigned this Jan 20, 2022
@adamjstewart
Copy link
Copy Markdown
Member

This also relates to #14513. This hasn't yet been fixed globally but I think it should be relatively easy to do after the recent Python refactor I did. I'll try to submit a PR sometime this week to do that.

@rgommers
Copy link
Copy Markdown
Contributor

The headers are almost always installed in prefix/lib/pythonX.Y/site-packages, not in prefix/include.

This is actually a bad habit for Python packages. NumPy has multiple headers and library directories where it hides static libraries. SciPy uses all of them, here is what it needs to do:

# NumPy include directory - needed in all submodules
incdir_numpy = run_command(py3,
  [
    '-c',
    'import os; os.chdir(".."); import numpy; print(numpy.get_include())'
  ],
  check: true
).stdout().strip()

inc_np = include_directories(incdir_numpy)

incdir_f2py = incdir_numpy / '..' / '..' / 'f2py' / 'src'
inc_f2py = include_directories(incdir_f2py)
fortranobject_c = incdir_f2py / 'fortranobject.c'

cc = meson.get_compiler('c')
npymath_path = incdir_numpy / '..' / 'lib'
npymath_lib = cc.find_library('npymath', dirs: npymath_path)
npyrandom_path = incdir_numpy / '..' / '..' / 'random' / 'lib'
npyrandom_lib = cc.find_library('npyrandom', dirs: npyrandom_path)

I'd say that the best solution would be for NumPy to install into the correct/standard library and include directories, and/or add proper pkg-config support. Also build systems can (and will) support NumPy as a custom dependency. Spack support for custom within-sitepackages locations doesn't make as much sense to me, because if another package relies on that it will not work anywhere else (PyPI, conda-forge, etc.) so it only makes sense for end user code that's only used with Spack. And end users probably cargo cult solutions from either the NumPy docs or from the build files of other packages.

@adamjstewart
Copy link
Copy Markdown
Member

@rgommers you may be interested in #28527 which should hopefully solve this issue for all Python packages, not just numpy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants