Repository navigation
Closed
Labels
C: editableEditable installationsEditable installationsPEP implementationInvolves some PEPInvolves some PEPtype: bugA confirmed bug or unintended behaviorA confirmed bug or unintended behavior
Milestone
Description
Description
Our CI is failing due to version 21.3 release, in the pip wheel subprocess call during installation, with seemingly random packages being the culprit.
E.g.
subprocess.CalledProcessError: Command '['/opt/hostedtoolcache/Python/3.7.12/x64/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpiq0bznxg', '--quiet', 'cryptography<3.5,>=3.4.1']' returned non-zero exit status 1.
Full Log attached below
The specific dependency it fails for seems completely random, in 5 subsequent runs it's been pathspec, cryptography, tabulate, rdflib and pyasn1.
I can reproduce the issue locally as well with Arch Linux, 21.3 fails, 21.2.4 works.
This only happens in editable mode (using -e flag), without it everything install fine.
Expected behavior
Renku Package to be installed without problem
pip version
21.3
Python version
3.7.8
OS
Arch Linux/Github Actions(Ubuntu)
How to Reproduce
- clone https://github.com/SwissDataScienceCenter/renku-python
- upgrade to pip 21.3
- do
pip install -e .[all]
Output
$ python -m pip install -e .[all]
/home/user/.pyenv/versions/renku-python/lib/python3.7/site-packages/secretstorage/dhcrypto.py:16: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
from cryptography.utils import int_from_bytes
/home/user/.pyenv/versions/renku-python/lib/python3.7/site-packages/secretstorage/util.py:25: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead
from cryptography.utils import int_from_bytes
Obtaining file:///home/user/DEV/ETH/renku-python
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build editable: started
Getting requirements to build editable: finished with status 'done'
Preparing editable metadata (pyproject.toml): started
Preparing editable metadata (pyproject.toml): finished with status 'done'
Build backend does not support editables, falling back to setup.py egg_info.
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /home/user/.pyenv/versions/renku-python/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/user/DEV/ETH/renku-python/setup.py'"'"'; __file__='"'"'/home/user/DEV/ETH/renku-python/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-0jfl5x91
cwd: /home/user/DEV/ETH/renku-python/
Complete output (31 lines):
/home/user/.pyenv/versions/renku-python/bin/python: No module named pip
Traceback (most recent call last):
File "/tmp/pip-build-env-gh6wm4a4/overlay/lib/python3.7/site-packages/setuptools/installer.py", line 75, in fetch_build_egg
subprocess.check_call(cmd)
File "/home/user/.pyenv/versions/3.7.8/lib/python3.7/subprocess.py", line 363, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/user/.pyenv/versions/renku-python/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpppr5o_gk', '--quiet', 'zc.relation<1.2,>=1.1']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/user/DEV/ETH/renku-python/setup.py", line 266, in <module>
"Development Status :: 4 - Beta",
File "/tmp/pip-build-env-gh6wm4a4/overlay/lib/python3.7/site-packages/setuptools/__init__.py", line 152, in setup
_install_setup_requires(attrs)
File "/tmp/pip-build-env-gh6wm4a4/overlay/lib/python3.7/site-packages/setuptools/__init__.py", line 147, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/tmp/pip-build-env-gh6wm4a4/overlay/lib/python3.7/site-packages/setuptools/dist.py", line 809, in fetch_build_eggs
replace_conflicting=True,
File "/tmp/pip-build-env-gh6wm4a4/overlay/lib/python3.7/site-packages/pkg_resources/__init__.py", line 768, in resolve
replace_conflicting=replace_conflicting
File "/tmp/pip-build-env-gh6wm4a4/overlay/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1051, in best_match
return self.obtain(req, installer)
File "/tmp/pip-build-env-gh6wm4a4/overlay/lib/python3.7/site-packages/pkg_resources/__init__.py", line 1063, in obtain
return installer(requirement)
File "/tmp/pip-build-env-gh6wm4a4/overlay/lib/python3.7/site-packages/setuptools/dist.py", line 877, in fetch_build_egg
return fetch_build_egg(self, req)
File "/tmp/pip-build-env-gh6wm4a4/overlay/lib/python3.7/site-packages/setuptools/installer.py", line 77, in fetch_build_egg
raise DistutilsError(str(e)) from e
distutils.errors.DistutilsError: Command '['/home/user/.pyenv/versions/renku-python/bin/python', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmpppr5o_gk', '--quiet', 'zc.relation<1.2,>=1.1']' returned non-zero exit status 1.
----------------------------------------
WARNING: Discarding file:///home/user/DEV/ETH/renku-python. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.Code of Conduct
- I agree to follow the PSF Code of Conduct.
Reacted by teije01, Tilman Möller, Joe Gagliano, Chris Withers, Tyler Seader, Max Jones, Pierre Augier, Noah Kantrowitz, Matthieu Rigal, marcinbaranowski5 and 3 more
Reacted by Tilman Möller, Ben Mares, Guillaume Tassery, Yolan Honoré-Rougé and Anatoly Makarevich
Metadata
Metadata
Assignees
Labels
C: editableEditable installationsEditable installationsPEP implementationInvolves some PEPInvolves some PEPtype: bugA confirmed bug or unintended behaviorA confirmed bug or unintended behavior
Activity
teije01 commentedon Oct 12, 2021
Experiencing the same problems, also
Same with other python versions. Downgrading pip resolved our problem
sbidoul commentedon Oct 12, 2021
Ok, this seems to be related to the PEP 660 implementation.
Since this project has a pyproject.toml without build-system table, it seems that pip<21.3 did first prepare the metadata using the
setuptools.build_meta:__legacy__build backend, then calledsetup.py develop, so doing a mix of PEP 517 and legacy installation.Now pip first tries a PEP 660 editable metadata preparation and, since the build backend does not support PEP 660, it falls back to a legacy metadata preparation (
setup.py egg_info), which should in theory be equivalent but, for some reason I don't fully understand, does not work for your project.[-]21.3 breaking installation of package from source with random error on subprocess call to wheel[/-][+]21.3 breaking editable installation with `No module named pip` error[/+]sbidoul commentedon Oct 12, 2021
@Panaetius can you test if #10577 fixes the issue ?
You can install it with
pip install -U "pip @ git+https://github.com/pypa/pip@refs/pull/10577/head"sbidoul commentedon Oct 12, 2021
Note I tentatively added this to a 21.3.1 bugfix milestone, since it is a regression, but this does not mean doing an actual bugfix release has been decided yet.
Panaetius commentedon Oct 12, 2021
It gets a lot further in the install process but then unfortunately has the same error when trying to install things:
https://gist.github.com/Panaetius/5d9b29b3694b310e567bc803fb9878a8
Thank you for looking into this!
sbidoul commentedon Oct 12, 2021
@Panaetius Could you retry in a fresh environment ? Also removing any
.eggsand.egg-infodirectories that might have been created by previous attempts.ceelian commentedon Oct 12, 2021
Having a similar problem on our CI System since the new release. Our CI does a fresh setup of the environment in each build. So IMO it looks like the "freshness" of the environment isn't the cause. Using the release versions pip-21.3 setuptools-58.2.0.
Our workaround for now: pinning versions to pip==21.2.4 setuptools==57.4.0
sbidoul commentedon Oct 12, 2021
@ceelian my comment about retrying in a fresh environment was specifically for @Panaetius who is testing the linked PR.
If you'd like to test #10577 too, this is welcome.
37 remaining items
layday commentedon Oct 18, 2021
Try dropping editable installs
Handle legacy nested datasets (with tests) (#850)
Fix problem with pip 21.3 that prevents installation from source (#2344)
Pin pip==21.2.4
try no-use-pep517
ci: pip install drop -e
Bump pip from 21.3 to 21.3.1 (PR #3370)
ci: pin to pip version 21.2.4
ci: pin to pip version 21.2.4
ci: pin to pip version 21.2.4
Bump pip from 21.3 to 21.3.1 in /.github/workflows (#67)
Bump pip from 21.0.1 to 21.3.1 (PR #3429)