Repository navigation
pip install -U . fails when there's a UNIX domain socket in the current directory #5306
Description
- Pip version: 10.0.1
- Python version: 3.6
- Operating system: macOS Sierra
Description:
It seems pip 10.0.1, when installing from a directory, makes a copy of that directory before doing anything else (?). This causes problems for mypy developers. We have a daemon process that creates a UNIX domain socket named dmypy.sock in the current directory. When the mypy daemon is running in the mypy directory, and we try to install from there, this copy fails with the following error:
$ pip3 install -U .
Processing /Users/guido/src/mypy
Could not install packages due to an EnvironmentError: [('/Users/guido/src/mypy/dmypy.sock', '/private/var/folders/63/czkyq6090dd0t157zhx54xvhrdlybt/T/pip-req-build-yucol_r3/dmypy.sock', "[Errno 102] Operation not supported on socket: '/Users/guido/src/mypy/dmypy.sock'")]
Using -v we get the following traceback:
$ pip3 install -U . -v
Created temporary directory: /private/var/folders/63/czkyq6090dd0t157zhx54xvhrdlybt/T/pip-ephem-wheel-cache-z05l_dty
Created temporary directory: /private/var/folders/63/czkyq6090dd0t157zhx54xvhrdlybt/T/pip-install-pdgnid0m
Processing /Users/guido/src/mypy
Created temporary directory: /private/var/folders/63/czkyq6090dd0t157zhx54xvhrdlybt/T/pip-req-build-cykzr90d
Could not install packages due to an EnvironmentError.
Traceback (most recent call last):
File "/Users/guido/v36/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 291, in run
resolver.resolve(requirement_set)
File "/Users/guido/v36/lib/python3.6/site-packages/pip/_internal/resolve.py", line 103, in resolve
self._resolve_one(requirement_set, req)
File "/Users/guido/v36/lib/python3.6/site-packages/pip/_internal/resolve.py", line 257, in _resolve_one
abstract_dist = self._get_abstract_dist_for(req_to_install)
File "/Users/guido/v36/lib/python3.6/site-packages/pip/_internal/resolve.py", line 210, in _get_abstract_dist_for
self.require_hashes
File "/Users/guido/v36/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 310, in prepare_linked_requirement
progress_bar=self.progress_bar
File "/Users/guido/v36/lib/python3.6/site-packages/pip/_internal/download.py", line 824, in unpack_url
unpack_file_url(link, location, download_dir, hashes=hashes)
File "/Users/guido/v36/lib/python3.6/site-packages/pip/_internal/download.py", line 700, in unpack_file_url
shutil.copytree(link_path, location, symlinks=True)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/shutil.py", line 359, in copytree
raise Error(errors)
shutil.Error: [('/Users/guido/src/mypy/dmypy.sock', '/private/var/folders/63/czkyq6090dd0t157zhx54xvhrdlybt/T/pip-req-build-cykzr90d/dmypy.sock', "[Errno 102] Operation not supported on socket: '/Users/guido/src/mypy/dmypy.sock'")]
Cleaning up...
A simple workaround is to stop the daemon (or if it's been killed, rm dmypy.sock).
(First reported for mypy: python/mypy#4945, for @ilevkivsky.)
Activity
pradyunsg commentedon Apr 21, 2018
A fix for this would be that pip tries to be smarter about what it's copying.
Honestly though, I have a feeling that adding support for a
.pipignorewill let people work around edge cases and that would be cleaner than adding support for all kinds of edge cases in pip.Unrelated: pip will always reinstall from a local directory --
pip install .should just work for you. :)gvanrossum commentedon Apr 22, 2018
It's also arguably a bug in
shutil.copytree(), which contains this comment:gvanrossum commentedon Apr 22, 2018
(And thanks for the tip about not needing
-U! I learned something. :-)piotr-dobrogost commentedon Apr 23, 2018
For some background see issue #2195 – pip install of a directory is super slow.
Problem with source directory containing sockets was raised in issue #2974 – Wheel command does not work if the directory contains a socket.
pradyunsg commentedon Aug 21, 2019
Thanks to @chrahunt and @cjerdonek, there's a Python-3-only fix for this which skips copying socket files when copying the source directory. 🎉