Poetry build does not install build dependencies (affects both install and build) #2789
Description
- I am on the latest Poetry version.
- I have searched the issues of this repo and believe that this is not a duplicate.
- If an exception occurs when executing a command, I executed it again in debug mode (
-vvvoption).
- OS version and name: OS X 10.15.5 (also tested in docker on ubuntu 16.04)
- Poetry version: 1.1.0b2
- Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/mdgoldberg/96a9955c8c823f2127dbc471a954b898
Issue
I created a repo with a minimal reproduction: https://github.com/mdgoldberg/poetry-cython-example
The problem here is that Cython isn't installed prior to building/installing the root package, even though Cython is specified in build-system.requires in the pyproject.toml. Here's how I am reproducing the issues, for both poetry install and poetry build:
$ git clone [email protected]:mdgoldberg/poetry-cython-example.git
$ cd poetry-cython-example
$ poetry install -vvv
Creating virtualenv poetry-cython-example in /Users/mattgoldberg/poetry-cython-example/.venv
Using virtualenv: /Users/mattgoldberg/poetry-cython-example/.venv
Installing dependencies from lock file
Finding the necessary packages for the current system
Installing the current project: poetry-cython-example (0.1.0)
- Building package poetry-cython-example in editable mode
- Executing build script: build.py
Traceback (most recent call last):
File "/Users/mattgoldberg/poetry-cython-example/build.py", line 6, in <module>
from Cython.Build import cythonize
ModuleNotFoundError: No module named 'Cython'
- Adding poetry_cython_example.pth to /Users/mattgoldberg/poetry-cython-example/.venv/lib/python3.6/site-packages for /Users/mattgoldberg/poetry-cython-example
- Adding the poetry_cython_example-0.1.0.dist-info directory to /Users/mattgoldberg/poetry-cython-example/.venv/lib/python3.6/site-packages
$
$ poetry build -f wheel -vvv
Building poetry-cython-example (0.1.0)
- Building wheel
Traceback (most recent call last):
File "build.py", line 6, in <module>
from Cython.Build import cythonize
ModuleNotFoundError: No module named 'Cython'
CalledProcessError
Command '['/Users/mattgoldberg/poetry-cython-example/.venv/bin/python', 'build.py']' returned non-zero exit status 1.
at ~/.pyenv/versions/3.6.10/lib/python3.6/subprocess.py:311 in check_call
307│ if retcode:
308│ cmd = kwargs.get("args")
309│ if cmd is None:
310│ cmd = popenargs[0]
→ 311│ raise CalledProcessError(retcode, cmd)
312│ return 0
313│
314│
315│ def check_output(*popenargs, timeout=None, **kwargs):
$ # now install Cython and they work
$ pip install Cython
Collecting Cython
Using cached Cython-0.29.21-cp36-cp36m-macosx_10_9_x86_64.whl (1.9 MB)
Installing collected packages: Cython
Successfully installed Cython-0.29.21
$ poetry install -vvv
Using virtualenv: /Users/mattgoldberg/poetry-cython-example/.venv
Installing dependencies from lock file
Finding the necessary packages for the current system
Installing the current project: poetry-cython-example (0.1.0)
- Building package poetry-cython-example in editable mode
- Executing build script: build.py
- Adding poetry_cython_example.pth to /Users/mattgoldberg/poetry-cython-example/.venv/lib/python3.6/site-packages for /Users/mattgoldberg/poetry-cython-example
- Adding the poetry_cython_example-0.1.0.dist-info directory to /Users/mattgoldberg/poetry-cython-example/.venv/lib/python3.6/site-packages
$ poetry build -f wheel -vvv
Building poetry-cython-example (0.1.0)
- Building wheel
- Built poetry_cython_example-0.1.0-cp36-cp36m-macosx_10_15_x86_64.whl
Note that for the fix, all that matters is whether Cython is installed in the "current" Python environment. If I install Cython globally but then activate the virtual env (which doesn't have Cython), then poetry install and poetry build still fail. Conversely, if I install Cython in the virtualenv and try to poetry install or poetry build from the directory but outside the virtualenv, it also fails.
Please, let me know if I'm misusing build.py or any other features, I know some of them may not be stabilized. For now, I'm just installing Cython before running poetry build or poetry install` in my Dockerfile, but it really seems like I shouldn't need to, per PEP 518. Let me know if I'm misunderstanding, and thank you for developing Poetry!