pytest --collect-only fails when 'fail under' is set on a configuration #426
Description
Summary
My pyproject.toml have the following sections (among other):
[tool.pytest.ini_options]
minversion = '6.0'
addopts = '--quiet --cov'
[tool.coverage.run]
source = ['my_module']
[tool.coverage.report]
fail_under = 75And some tools (such as VSCode Python's plugin) depends on pytest --collect-only. However, when running that it fails as it doesn't actually run the tests and it displays a coverage way below the real value.
Expected vs actual result
Expected:
when running pytest --collect-only I'd expect it only to fail if for some reason it collect the list of tests, ignoring the fail under parameter.
Actual result:
pytest --collect-only fails stating the coverage didn't reach the goal.
Reproducer
git clone https://github.com/maurofaccenda/pytest-cov-issue.git
cd pytest-cov-issue/
python3 -m venv venv
source ./venv/bin/activate
pip install pip --upgrade
pip install -r requirements.txt
pytest # succeeds
pytest --collect-only # fails
echo $? # prints 1
Versions
pip list output:
Package Version
-------------- -------
attrs 20.1.0
coverage 5.2.1
iniconfig 1.0.1
more-itertools 8.4.0
packaging 20.4
pip 20.2.2
pkg-resources 0.0.0
pluggy 0.13.1
py 1.9.0
pyparsing 2.4.7
pytest 6.0.1
pytest-cov 2.10.1
setuptools 44.0.0
six 1.15.0
toml 0.10.1
$ python --version
Python 3.8.2
$ pytest --version
pytest 6.0.1
Config
[tool.pytest.ini_options]
minversion = '6.0'
addopts = '--quiet --cov'
[tool.coverage.run]
source = ['my_module']
[tool.coverage.report]
fail_under = 75