-
Notifications
You must be signed in to change notification settings - Fork 2.4k
Installation issue: python+ctypes #29478
Description
Steps to reproduce the issue
$ spack spec -I python
Input spec
--------------------------------
- python
Concretized
--------------------------------
[+] [email protected]%[email protected]+bz2+ctypes+dbm~debug+ensurepip+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93,4c24573,f2fd060 arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected]~debug~pic+shared arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected] arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected] libs=shared,static arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected]+libbsd arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected] arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected] arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected] arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected] arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected]~symlinks+termlib abi=none arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected] arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected]+bzip2+curses+git~libunistring+libxml2+tar+xz arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected]~python arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected]~pic libs=shared,static arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected]+optimize+pic+shared arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected] arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected] arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected]~docs certs=system arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected]+cpanm+shared+threads arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected]+cxx~docs+stl patches=b231fcc arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected]+column_metadata+dynamic_extensions+fts~functions+rtree arch=linux-centos7-x86_64
[+] ^[email protected]%[email protected] arch=linux-centos7-x86_64Error message
Python appears to install successfully, but downstream packages fail, e.g. py-setuptools:
Error message
```
==> py-setuptools: Executing phase: 'install'
==> [2022-03-12-10:39:09.181041] '/projects/spack/opt/spack/gcc-8.5.0/python/6z6bbfg/bin/python3.9' '-m' 'pip' '-vvv' '--no-input' '--no-cache-dir' '--disable-pip-version-check' 'install' '--no-deps' '--ignore-installed' '--no-build-isolation' '--no-warn-script-location' '--no-index' '--prefix=/projects/spack/opt/spack/gcc-8.5.0/py-setuptools/26gyn5x' '.'
Using pip 21.3.1 from /projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip (python 3.9)
Non-user install due to --prefix or --target option
Ignoring indexes: https://pypi.org/simple
Created temporary directory: /tmp/pip-ephem-wheel-cache-p5vr3e0t
Created temporary directory: /tmp/pip-req-tracker-iv7wygbd
Initialized build tracking at /tmp/pip-req-tracker-iv7wygbd
Created build tracker: /tmp/pip-req-tracker-iv7wygbd
Entered build tracker: /tmp/pip-req-tracker-iv7wygbd
Created temporary directory: /tmp/pip-install-2zg9v9l0
Processing /tmp/s3j/spack-stage/spack-stage-py-setuptools-59.4.0-26gyn5xre2zssmqsn2qqurxbbooxgssz/spack-src
Added file:///tmp/s3j/spack-stage/spack-stage-py-setuptools-59.4.0-26gyn5xre2zssmqsn2qqurxbbooxgssz/spack-src to build tracker '/tmp/pip-req-tracker-iv7wygbd'
Created temporary directory: /tmp/pip-modern-metadata-n1k48pa1
Preparing metadata (pyproject.toml): started
Running command /projects/spack/opt/spack/gcc-8.5.0/python/6z6bbfg/bin/python3.9 /projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpte_wdk3q
Preparing metadata (pyproject.toml): finished with status 'done'
ERROR: Exception:
Traceback (most recent call last):
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 164, in exc_logging_wrapper
status = run_func(*args)
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
return func(self, options, args)
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 338, in run
requirement_set = resolver.resolve(
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 73, in resolve
collected = self.factory.collect_root_requirements(root_reqs)
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 468, in collect_root_requirements
req = self._make_requirement_from_install_req(
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 430, in _make_requirement_from_install_req
cand = self._make_candidate_from_link(
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 201, in _make_candidate_from_link
self._link_candidate_cache[link] = LinkCandidate(
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 281, in __init__
super().__init__(
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
self.dist = self._prepare()
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare
dist = self._prepare_distribution()
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 292, in _prepare_distribution
return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 482, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 546, in _prepare_linked_requirement
dist = _get_prepared_distribution(
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 58, in _get_prepared_distribution
abstract_dist.prepare_distribution_metadata(finder, build_isolation)
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_internal/distributions/sdist.py", line 49, in prepare_distribution_metadata
self.req.prepare_metadata()
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_internal/req/req_install.py", line 556, in prepare_metadata
self.metadata_directory = generate_metadata(
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_internal/operations/build/metadata.py", line 28, in generate_metadata
distinfo_dir = backend.prepare_metadata_for_build_wheel(metadata_dir)
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_vendor/pep517/wrappers.py", line 188, in prepare_metadata_for_build_wheel
return self._call_hook('prepare_metadata_for_build_wheel', {
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_vendor/pep517/wrappers.py", line 332, in _call_hook
raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
File "/projects/spack/opt/spack/gcc-8.5.0/py-pip/m2bllhv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 89, in _build_backend
obj = import_module(mod_path)
File "/projects/spack/opt/spack/gcc-8.5.0/python/6z6bbfg/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 972, in _find_and_load_unlocked
File "", line 228, in _call_with_frames_removed
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/tmp/s3j/spack-stage/spack-stage-py-setuptools-59.4.0-26gyn5xre2zssmqsn2qqurxbbooxgssz/spack-src/setuptools/__init__.py", line 18, in
from setuptools.dist import Distribution
File "/tmp/s3j/spack-stage/spack-stage-py-setuptools-59.4.0-26gyn5xre2zssmqsn2qqurxbbooxgssz/spack-src/setuptools/dist.py", line 38, in
from setuptools import windows_support
File "/tmp/s3j/spack-stage/spack-stage-py-setuptools-59.4.0-26gyn5xre2zssmqsn2qqurxbbooxgssz/spack-src/setuptools/windows_support.py", line 2, in
import ctypes
File "/projects/spack/opt/spack/gcc-8.5.0/python/6z6bbfg/lib/python3.9/ctypes/__init__.py", line 8, in
from _ctypes import Union, Structure, Array
ModuleNotFoundError: No module named '_ctypes'
Looking back at the python build logs, it silently hid the failure of the ctypes module:
Error message
*** WARNING: renaming "_ctypes" since importing it failed: build/lib.linux-x86_64-3.9/_ctypes.cpython-39-x86_64-linux-gnu.so: undefined symbol: ffi_prep_cifPython build finished successfully!
The necessary bits to build these optional modules were not found:
_bz2 _curses _curses_panel
_lzma _sqlite3 _tkinter
readline
To find the necessary bits, look in setup.py in detect_modules() for the module's name.The following modules found by detect_modules() in setup.py, have been
built by the Makefile instead, as configured by the Setup files:
_abc atexit pwd
timeFollowing modules built successfully but were removed because they could not be imported:
_ctypes
Information on your system
- Spack: 0.17.1-1532-8adc6b7e8e
- Python: 3.9.10
- Platform: linux-centos7-zen2
- Concretizer: clingo
Additional information
@adamjstewart @scheibelp @skosukhin @varioustoxins
A recent build on an almost identical configuration passed, so I suspect this is due to the libffi update in #28970 .
spack-build-out.txt
General information
- I have run
spack debug reportand reported the version of Spack/Python/Platform - I have run
spack maintainers <name-of-the-package>and @mentioned any maintainers - I have uploaded the build log and environment files
- I have searched the issues of this repo and believe this is not a duplicate