cannot install tensorflow and opencv-python at the same time due to wrongly detected numpy requirement #4451
Description
- I am on the latest Poetry version.
- I have searched the issues of this repo and believe that this is not a duplicate.
- If an exception occurs when executing a command, I executed it again in debug mode (
-vvvoption).
- OS version and name: Ubuntu 20.10
- Poetry version: 1.1.8
- Link of a Gist with the contents of your pyproject.toml file:
Issue
I am trying to create pyproject.toml that would specify both tensorflow and opencv-python as dependencies.
This lead to `SolverProblemError
Because no versions of tensorflow match >2.5.0,<2.5.1 || >2.5.1,<2.6.0
and tensorflow (2.5.0) depends on numpy (>=1.19.2,<1.20.0), tensorflow (>=2.5.0,<2.5.1 || >2.5.1,<2.6.0) requires numpy (>=1.19.2,<1.20.0).
And because tensorflow (2.5.1) depends on numpy (>=1.19.2,<1.20.0), tensorflow (>=2.5.0,<2.6.0) requires numpy (>=1.19.2,<1.20.0).
Because no versions of opencv-python match >=4.5.3,<4.5.3.56 || >4.5.3.56,<5.0.0
and opencv-python (4.5.3.56) depends on numpy (>=1.21.0), opencv-python (>=4.5.3,<5.0.0) requires numpy (>=1.21.0).
Thus, opencv-python (>=4.5.3,<5.0.0) is incompatible with tensorflow (>=2.5.0,<2.6.0).
So, because poetry-opencv-tensorflow depends on both tensorflow (~2.5.0) and opencv-python (^4.5.3), version solving failed.
So, as tensorflow has very strict numpy requirement this must be 1.19. Assuming here this would python 3.8 the opencv would require minimum numpy 1.17.3 (so 1.19 would be fine).
For some reason poetry thinks it requires 1.21 which is incorrect according to:
- setup.py of opencv-python, see github
- metadata of downloaded wheel
$ pkginfo -f requires_dist opencv_python-4.5.3.56-cp38-cp38-win_amd64.whl
requires_dist: ['numpy (>=1.17.3)']
- fact that it works fine with pip
$ pip freeze | grep -P "numpy|tensorflow"
numpy==1.19.5
tensorflow==2.5.0
tensorflow-estimator==2.5.0
$ pip install opencv-python
Collecting opencv-python
Using cached opencv_python-4.5.3.56-cp38-cp38-manylinux2014_x86_64.whl (49.9 MB)
Requirement already satisfied: numpy>=1.17.3 in /home/rskolasinski/miniconda3/envs/test/lib/python3.8/site-packages (from opencv-python) (1.19.5)
Installing collected packages: opencv-python
Successfully installed opencv-python-4.5.3.56
To reproduce:
First prepare environment from linked gist file
$ poetry install
Creating virtualenv poetry-opencv-tensorflow-qJ4q1krR-py3.8 in /home/rskolasinski/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies... (2.1s)
Writing lock file
Package operations: 36 installs, 0 updates, 0 removals
• Installing certifi (2021.5.30)
• Installing charset-normalizer (2.0.4)
• Installing idna (3.2)
• Installing pyasn1 (0.4.8)
• Installing urllib3 (1.26.6)
• Installing cachetools (4.2.2)
• Installing oauthlib (3.1.1)
• Installing pyasn1-modules (0.2.8)
• Installing requests (2.26.0)
• Installing rsa (4.7.2)
• Installing six (1.15.0)
• Installing google-auth (1.35.0)
• Installing requests-oauthlib (1.3.0)
• Installing absl-py (0.13.0)
• Installing grpcio (1.34.1)
• Installing google-auth-oauthlib (0.4.5)
• Installing markdown (3.3.4)
• Installing numpy (1.19.5)
• Installing protobuf (3.17.3)
• Installing tensorboard-plugin-wit (1.8.0)
• Installing tensorboard-data-server (0.6.1)
• Installing werkzeug (2.0.1)
• Installing astunparse (1.6.3)
• Installing flatbuffers (1.12)
• Installing gast (0.4.0)
• Installing google-pasta (0.2.0)
• Installing h5py (3.1.0)
• Installing keras-nightly (2.5.0.dev2021032900)
• Installing keras-preprocessing (1.1.2)
• Installing opt-einsum (3.3.0)
• Installing tensorboard (2.6.0)
• Installing tensorflow-estimator (2.5.0)
• Installing termcolor (1.1.0)
• Installing typing-extensions (3.7.4.3)
• Installing wrapt (1.12.1)
• Installing tensorflow (2.5.1)and now try to add opencv-python package
$ poetry add opencv-python -vvv
Using virtualenv: /home/rskolasinski/.cache/pypoetry/virtualenvs/poetry-opencv-tensorflow-qJ4q1krR-py3.8
PyPI: No release information found for opencv-python-3.1.0, skipping
PyPI: 53 packages found for opencv-python *
Using version ^4.5.3 for opencv-python
Updating dependencies
Resolving dependencies...
1: fact: poetry-opencv-tensorflow is 0.1.0
1: derived: poetry-opencv-tensorflow
1: fact: poetry-opencv-tensorflow depends on tensorflow (~2.5.0)
1: fact: poetry-opencv-tensorflow depends on opencv-python (^4.5.3)
1: selecting poetry-opencv-tensorflow (0.1.0)
1: derived: opencv-python (>=4.5.3,<5.0.0)
1: derived: tensorflow (>=2.5.0,<2.6.0)
PyPI: No release information found for opencv-python-3.1.0, skipping
PyPI: 1 packages found for opencv-python >=4.5.3,<5.0.0
1: fact: opencv-python (4.5.3.56) depends on numpy (>=1.21.0)
1: selecting opencv-python (4.5.3.56)
1: derived: numpy (>=1.21.0)
PyPI: No release information found for numpy-0.9.6, skipping
PyPI: No release information found for numpy-0.9.8, skipping
PyPI: No release information found for numpy-1.0.3, skipping
PyPI: No release information found for numpy-1.0.4, skipping
PyPI: No release information found for numpy-1.0b1, skipping
PyPI: No release information found for numpy-1.0b4, skipping
PyPI: No release information found for numpy-1.0b5, skipping
PyPI: No release information found for numpy-1.0rc1, skipping
PyPI: No release information found for numpy-1.0rc2, skipping
PyPI: No release information found for numpy-1.0rc3, skipping
PyPI: No release information found for numpy-1.1.1, skipping
PyPI: No release information found for numpy-1.2.0, skipping
PyPI: No release information found for numpy-1.2.1, skipping
PyPI: No release information found for numpy-1.4.0, skipping
PyPI: 3 packages found for numpy >=1.21.0
1: fact: tensorflow (2.5.1) depends on numpy (>=1.19.2,<1.20.0)
1: fact: tensorflow (2.5.1) depends on absl-py (>=0.10,<1.0)
1: fact: tensorflow (2.5.1) depends on astunparse (>=1.6.3,<1.7.0)
1: fact: tensorflow (2.5.1) depends on flatbuffers (>=1.12.0,<1.13.0)
1: fact: tensorflow (2.5.1) depends on google-pasta (>=0.2,<1.0)
1: fact: tensorflow (2.5.1) depends on h5py (>=3.1.0,<3.2.0)
1: fact: tensorflow (2.5.1) depends on keras-preprocessing (>=1.1.2,<1.2.0)
1: fact: tensorflow (2.5.1) depends on opt-einsum (>=3.3.0,<3.4.0)
1: fact: tensorflow (2.5.1) depends on protobuf (>=3.9.2)
1: fact: tensorflow (2.5.1) depends on six (>=1.15.0,<1.16.0)
1: fact: tensorflow (2.5.1) depends on termcolor (>=1.1.0,<1.2.0)
1: fact: tensorflow (2.5.1) depends on typing-extensions (>=3.7.4,<3.8.0)
1: fact: tensorflow (2.5.1) depends on wrapt (>=1.12.1,<1.13.0)
1: fact: tensorflow (2.5.1) depends on gast (0.4.0)
1: fact: tensorflow (2.5.1) depends on tensorboard (>=2.5,<3.0)
1: fact: tensorflow (2.5.1) depends on tensorflow-estimator (>=2.5.0,<2.6.0)
1: fact: tensorflow (2.5.1) depends on keras-nightly (>=2.5.0.dev,<2.6.0)
1: fact: tensorflow (2.5.1) depends on grpcio (>=1.34.0,<1.35.0)
1: derived: not tensorflow (==2.5.1)
PyPI: 1 packages found for tensorflow >=2.5.0,<2.5.1 || >2.5.1,<2.6.0
1: fact: tensorflow (2.5.0) depends on numpy (>=1.19.2,<1.20.0)
1: fact: tensorflow (2.5.0) depends on absl-py (>=0.10,<1.0)
1: fact: tensorflow (2.5.0) depends on astunparse (>=1.6.3,<1.7.0)
1: fact: tensorflow (2.5.0) depends on flatbuffers (>=1.12.0,<1.13.0)
1: fact: tensorflow (2.5.0) depends on google-pasta (>=0.2,<1.0)
1: fact: tensorflow (2.5.0) depends on h5py (>=3.1.0,<3.2.0)
1: fact: tensorflow (2.5.0) depends on keras-preprocessing (>=1.1.2,<1.2.0)
1: fact: tensorflow (2.5.0) depends on opt-einsum (>=3.3.0,<3.4.0)
1: fact: tensorflow (2.5.0) depends on protobuf (>=3.9.2)
1: fact: tensorflow (2.5.0) depends on six (>=1.15.0,<1.16.0)
1: fact: tensorflow (2.5.0) depends on termcolor (>=1.1.0,<1.2.0)
1: fact: tensorflow (2.5.0) depends on typing-extensions (>=3.7.4,<3.8.0)
1: fact: tensorflow (2.5.0) depends on wrapt (>=1.12.1,<1.13.0)
1: fact: tensorflow (2.5.0) depends on gast (0.4.0)
1: fact: tensorflow (2.5.0) depends on tensorboard (>=2.5,<3.0)
1: fact: tensorflow (2.5.0) depends on tensorflow-estimator (>=2.5.0rc0,<2.6.0)
1: fact: tensorflow (2.5.0) depends on keras-nightly (>=2.5.0.dev,<2.6.0)
1: fact: tensorflow (2.5.0) depends on grpcio (>=1.34.0,<1.35.0)
1: derived: not tensorflow (==2.5.0)
1: fact: no versions of tensorflow match >2.5.0,<2.5.1 || >2.5.1,<2.6.0
1: conflict: no versions of tensorflow match >2.5.0,<2.5.1 || >2.5.1,<2.6.0
1: ! tensorflow (>2.5.0,<2.5.1 || >2.5.1,<2.6.0) is partially satisfied by not tensorflow (2.5.0)
1: ! which is caused by "tensorflow (2.5.0) depends on numpy (>=1.19.2,<1.20.0)"
1: ! thus: tensorflow (>=2.5.0,<2.5.1 || >2.5.1,<2.6.0) requires numpy (>=1.19.2,<1.20.0)
1: ! tensorflow (>=2.5.0,<2.5.1 || >2.5.1,<2.6.0) is partially satisfied by not tensorflow (2.5.1)
1: ! which is caused by "tensorflow (2.5.1) depends on numpy (>=1.19.2,<1.20.0)"
1: ! thus: tensorflow (>=2.5.0,<2.6.0) requires numpy (>=1.19.2,<1.20.0)
1: fact: tensorflow (>=2.5.0,<2.6.0) requires numpy (>=1.19.2,<1.20.0)
1: derived: numpy (>=1.19.2,<1.20.0)
1: derived: not opencv-python (==4.5.3.56)
1: fact: no versions of opencv-python match >=4.5.3,<4.5.3.56 || >4.5.3.56,<5.0.0
1: conflict: no versions of opencv-python match >=4.5.3,<4.5.3.56 || >4.5.3.56,<5.0.0
1: ! opencv-python (>=4.5.3,<4.5.3.56 || >4.5.3.56,<5.0.0) is partially satisfied by not opencv-python (4.5.3.56)
1: ! which is caused by "opencv-python (4.5.3.56) depends on numpy (>=1.21.0)"
1: ! thus: opencv-python (>=4.5.3,<5.0.0) requires numpy (>=1.21.0)
1: ! not numpy (>=1.21.0) is satisfied by numpy (>=1.19.2,<1.20.0)
1: ! which is caused by "tensorflow (>=2.5.0,<2.6.0) requires numpy (>=1.19.2,<1.20.0)"
1: ! thus: opencv-python (>=4.5.3,<5.0.0) is incompatible with tensorflow (>=2.5.0,<2.6.0)
1: ! tensorflow (>=2.5.0,<2.6.0) is satisfied by tensorflow (~2.5.0)
1: ! which is caused by "poetry-opencv-tensorflow depends on tensorflow (~2.5.0)"
1: ! thus: opencv-python is forbidden
1: ! opencv-python (>=4.5.3,<5.0.0) is satisfied by opencv-python (^4.5.3)
1: ! which is caused by "poetry-opencv-tensorflow depends on opencv-python (^4.5.3)"
1: ! thus: version solving failed
1: Version solving took 0.241 seconds.
1: Tried 1 solutions.
Stack trace:
8 ~/.poetry/lib/poetry/_vendor/py3.8/clikit/console_application.py:131 in run
129│ parsed_args = resolved_command.args
130│
→ 131│ status_code = command.handle(parsed_args, io)
132│ except KeyboardInterrupt:
133│ status_code = 1
7 ~/.poetry/lib/poetry/_vendor/py3.8/clikit/api/command/command.py:120 in handle
118│ def handle(self, args, io): # type: (Args, IO) -> int
119│ try:
→ 120│ status_code = self._do_handle(args, io)
121│ except KeyboardInterrupt:
122│ if io.is_debug():
6 ~/.poetry/lib/poetry/_vendor/py3.8/clikit/api/command/command.py:171 in _do_handle
169│ handler_method = self._config.handler_method
170│
→ 171│ return getattr(handler, handler_method)(args, io, self)
172│
173│ def __repr__(self): # type: () -> str
5 ~/.poetry/lib/poetry/_vendor/py3.8/cleo/commands/command.py:92 in wrap_handle
90│ self._command = command
91│
→ 92│ return self.handle()
93│
94│ def handle(self): # type: () -> Optional[int]
4 ~/.poetry/lib/poetry/console/commands/add.py:173 in handle
171│ self._installer.whitelist([r["name"] for r in requirements])
172│
→ 173│ status = self._installer.run()
174│ except BaseException:
175│ # Using BaseException here as some exceptions, eg: KeyboardInterrupt, do not inherit from Exception
3 ~/.poetry/lib/poetry/installation/installer.py:103 in run
101│ local_repo = Repository()
102│
→ 103│ return self._do_install(local_repo)
104│
105│ def dry_run(self, dry_run=True): # type: (bool) -> Installer
2 ~/.poetry/lib/poetry/installation/installer.py:235 in _do_install
233│ )
234│
→ 235│ ops = solver.solve(use_latest=self._whitelist)
236│ else:
237│ self._io.write_line("Installing dependencies from lock file")
1 ~/.poetry/lib/poetry/puzzle/solver.py:65 in solve
63│ with self._provider.progress():
64│ start = time.time()
→ 65│ packages, depths = self._solve(use_latest=use_latest)
66│ end = time.time()
67│
SolverProblemError
Because no versions of tensorflow match >2.5.0,<2.5.1 || >2.5.1,<2.6.0
and tensorflow (2.5.0) depends on numpy (>=1.19.2,<1.20.0), tensorflow (>=2.5.0,<2.5.1 || >2.5.1,<2.6.0) requires numpy (>=1.19.2,<1.20.0).
And because tensorflow (2.5.1) depends on numpy (>=1.19.2,<1.20.0), tensorflow (>=2.5.0,<2.6.0) requires numpy (>=1.19.2,<1.20.0).
Because no versions of opencv-python match >=4.5.3,<4.5.3.56 || >4.5.3.56,<5.0.0
and opencv-python (4.5.3.56) depends on numpy (>=1.21.0), opencv-python (>=4.5.3,<5.0.0) requires numpy (>=1.21.0).
Thus, opencv-python (>=4.5.3,<5.0.0) is incompatible with tensorflow (>=2.5.0,<2.6.0).
So, because poetry-opencv-tensorflow depends on both tensorflow (~2.5.0) and opencv-python (^4.5.3), version solving failed.
at ~/.poetry/lib/poetry/puzzle/solver.py:241 in _solve
237│ packages = result.packages
238│ except OverrideNeeded as e:
239│ return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
240│ except SolveFailure as e:
→ 241│ raise SolverProblemError(e)
242│
243│ results = dict(
244│ depth_first_search(
245│ PackageNode(self._package, packages), aggregate_package_nodes