Windows PyPy3 wheel builds crash due to unclosed file #3545
Description
- I am on the latest Poetry version.
- I have searched the issues of this repo and believe that this is not a duplicate.
- If an exception occurs when executing a command, I executed it again in debug mode (
-vvvoption).
- OS version and name: Windows 10 Home Version 20H2 OS Build 19042.685
- Python version: PyPy3.6 version 7.3.3, and PyPy3.7 version 7.3.3
- Poetry version: 1.1.4
- Link of a Gist with the contents of your pyproject.toml file: feedparser pyproject.toml @ 727ee7f
Issue
Poetry crashes when building wheels on Windows when using PyPy3.6 and PyPy3.7. The root cause is an unclosed file in poetry/core/masonry/builders/wheel.py, which causes shutil.move() to fail with a PermissionError. Here are the steps to reproduce. Note that I am using Powershell, so some commands may require syntax changes if you're reproducing in cmd.exe.
- Download either PyPy3.6 or PyPy3.7 from pypy.org. Extract somewhere. In my case, I extracted to
C:\Program Files (x86)\pypy37\. - Clone feedparser from git:
git clone https://github.com/kurtmckee/feedparser.git - Change directories:
cd feedparser - Checkout 727ee7f to ensure reproducibility:
git checkout 727ee7f - Prepare a virtual environment:
& 'C:\Program Files (x86)\pypy37\pypy3.exe' -m venv venv-pypy37 - Activate the virtual environment:
.\venv-pypy37\Scripts\Activate.ps1 - Update pip:
pypy3 -m pip install --upgrade pip - Install poetry:
pypy3 -m pip install poetry - Copy
pypy3.exetopython.exeso Poetry can run "python":copy .\venv-pypy37\Scripts\pypy3.exe .\venv-pypy37\Scripts\python.exe - Use poetry to install dependencies:
poetry update - Build feedparser:
poetry build
The output that I get is a traceback:
(venv-pypy37) C:\Users\Kurt\Documents\dev\demo-poetry-issue\feedparser [(727ee7f...) +1 ~0 -0 !]> poetry build
Building feedparser (6.0.2)
- Building sdist
- Built feedparser-6.0.2.tar.gz
- Building wheel
PermissionError
[WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\Kurt\\AppData\\Local\\Temp\\tmpuv5ngrf4.whl'
at C:\Program Files (x86)\pypy37\lib-python\3\shutil.py:581 in move
577│ symlinks=True)
578│ rmtree(src)
579│ else:
580│ copy_function(src, real_dst)
Traceback (most recent call last):
File "C:\Program Files (x86)\pypy37\lib-python\3\shutil.py", line 566, in move
os.rename(src, real_dst)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\Kurt\\AppData\\Local\\Temp\\tmpuv5ngrf4.whl' -> 'C:\\Users\\Kurt\\Documents\\dev\\demo-poetry-issue\\feedparser\\dist\\feedparser-6.0.2-py3-none-any.whl'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:\users\kurt\documents\dev\demo-poetry-issue\feedparser\venv-pypy37\site-packages\clikit\console_application.py", line 131, in run
status_code = command.handle(parsed_args, io)
File "c:\users\kurt\documents\dev\demo-poetry-issue\feedparser\venv-pypy37\site-packages\clikit\api\command\command.py", line 120, in handle
status_code = self._do_handle(args, io)
File "c:\users\kurt\documents\dev\demo-poetry-issue\feedparser\venv-pypy37\site-packages\clikit\api\command\command.py", line 171, in _do_handle
return getattr(handler, handler_method)(args, io, self)
File "c:\users\kurt\documents\dev\demo-poetry-issue\feedparser\venv-pypy37\site-packages\cleo\commands\command.py", line 92, in wrap_handle
return self.handle()
File "c:\users\kurt\documents\dev\demo-poetry-issue\feedparser\venv-pypy37\site-packages\poetry\console\commands\build.py", line 36, in handle
builder.build(fmt, executable=self.env.python)
File "c:\users\kurt\documents\dev\demo-poetry-issue\feedparser\venv-pypy37\site-packages\poetry\core\masonry\builder.py", line 30, in build
builder(self._poetry, executable=executable).build()
File "c:\users\kurt\documents\dev\demo-poetry-issue\feedparser\venv-pypy37\site-packages\poetry\core\masonry\builders\wheel.py", line 97, in build
shutil.move(temp_path, str(wheel_path))
File "C:\Program Files (x86)\pypy37\lib-python\3\shutil.py", line 581, in move
os.unlink(src)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\Kurt\\AppData\\Local\\Temp\\tmpuv5ngrf4.whl'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files (x86)\pypy37\lib-python\3\runpy.py", line 196, in _run_module_as_main
"__main__", mod_spec)
File "C:\Program Files (x86)\pypy37\lib-python\3\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\Kurt\Documents\dev\demo-poetry-issue\feedparser\venv-pypy37\Scripts\poetry.exe\__main__.py", line 7, in <module>
sys.exit(main())
File "c:\users\kurt\documents\dev\demo-poetry-issue\feedparser\venv-pypy37\site-packages\poetry\console\__init__.py", line 5, in main
return Application().run()
File "c:\users\kurt\documents\dev\demo-poetry-issue\feedparser\venv-pypy37\site-packages\clikit\console_application.py", line 142, in run
trace.render(io, simple=isinstance(e, CliKitException))
File "c:\users\kurt\documents\dev\demo-poetry-issue\feedparser\venv-pypy37\site-packages\clikit\ui\components\exception_trace.py", line 232, in render
return self._render_exception(io, self._exception)
File "c:\users\kurt\documents\dev\demo-poetry-issue\feedparser\venv-pypy37\site-packages\clikit\ui\components\exception_trace.py", line 269, in _render_exception
self._render_snippet(io, current_frame)
File "c:\users\kurt\documents\dev\demo-poetry-issue\feedparser\venv-pypy37\site-packages\clikit\ui\components\exception_trace.py", line 289, in _render_snippet
self._render_line(io, code_line)
File "c:\users\kurt\documents\dev\demo-poetry-issue\feedparser\venv-pypy37\site-packages\clikit\ui\components\exception_trace.py", line 402, in _render_line
io.write_line("{}{}".format(indent * " ", line))
File "c:\users\kurt\documents\dev\demo-poetry-issue\feedparser\venv-pypy37\site-packages\cleo\io\io_mixin.py", line 65, in write_line
super(IOMixin, self).write_line(string, flags)
File "c:\users\kurt\documents\dev\demo-poetry-issue\feedparser\venv-pypy37\site-packages\clikit\api\io\io.py", line 66, in write_line
self._output.write_line(string, flags=flags)
File "c:\users\kurt\documents\dev\demo-poetry-issue\feedparser\venv-pypy37\site-packages\clikit\api\io\output.py", line 69, in write_line
self.write(string, flags=flags, new_line=True)
File "c:\users\kurt\documents\dev\demo-poetry-issue\feedparser\venv-pypy37\site-packages\clikit\api\io\output.py", line 61, in write
self._stream.write(to_str(formatted))
File "c:\users\kurt\documents\dev\demo-poetry-issue\feedparser\venv-pypy37\site-packages\clikit\io\output_stream\stream_output_stream.py", line 24, in write
self._stream.write(string)
File "C:\Program Files (x86)\pypy37\lib-python\3\encodings\cp437.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u2192' in position 11: character maps to <undefined>
(venv-pypy37) C:\Users\Kurt\Documents\dev\demo-poetry-issue\feedparser [(727ee7f...) +1 ~0 -0 !]>
The root cause is an unmanaged os.fdopen() call in poetry/core/masonry/builders/wheel.py that keeps the zip file open when shutil.move() is called.
I've already tested a fix and will submit a PR shortly.