Repository navigation
Description
Moved from astropy/astropy-helpers#258
Every so often I make the mistake of running pytest (or py.test for old versions) instead of python setup.py test. It seems like this should work, but instead I get errors collecting tests that seem to be related to the astropy helpers. Even though the file doesn't appear in the traceback below, it seems to come from imports in the conftest.py from the package template.
This also appears to bite unwary users of packages that use the template: http://stackoverflow.com/questions/31410723/error-using-pytest-tutorial
$ pytest
=============================================== test session starts ================================================
platform darwin -- Python 2.7.12, pytest-3.0.3, py-1.4.31, pluggy-0.4.0
rootdir: /Users/jlong/software/webbpsf, inifile: setup.cfg
collected 0 items / 1 errors
====================================================== ERRORS ======================================================
________________________________________________ ERROR collecting _________________________________________________
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/py-1.4.31-py2.7.egg/py/_path/common.py:332: in visit
for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/py-1.4.31-py2.7.egg/py/_path/common.py:368: in gen
if p.check(dir=1) and (rec is None or rec(p))])
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/main.py:672: in _recurse
ihook = self.gethookproxy(path)
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/main.py:575: in gethookproxy
my_conftestmodules = pm._getconftestmodules(fspath)
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/config.py:314: in _getconftestmodules
mod = self._importconftest(conftestpath)
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/config.py:350: in _importconftest
self.consider_conftest(mod)
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/config.py:373: in consider_conftest
if self.register(conftestmodule, name=conftestmodule.__file__):
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/config.py:225: in register
ret = super(PytestPluginManager, self).register(plugin, name)
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:371: in register
hook._maybe_apply_history(hookimpl)
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:768: in _maybe_apply_history
res = self._hookexec(self, [method], kwargs)
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:339: in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:334: in <lambda>
_MultiCall(methods, kwargs, hook.spec_opts).execute()
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:614: in execute
res = hook_impl.function(*args)
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/astropy/tests/pytest_plugins.py:137: in pytest_configure
if not config.getoption('remote_data'):
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/config.py:1148: in getoption
raise ValueError("no option named %r" % (name,))
E ValueError: no option named u'remote_data'
============================================== pytest-warning summary ==============================================
WC1 None [pytest] section in setup.cfg files is deprecated, use [tool:pytest] instead.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
==================================== 1 pytest-warnings, 1 error in 1.62 seconds ====================================
At @pllim's request, I tried the last 2.x release of pytest as well:
$ py.test
=============================================== test session starts ================================================
platform darwin -- Python 2.7.12, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /Users/jlong/software/webbpsf, inifile: setup.cfg
collected 0 items / 1 errors
====================================================== ERRORS ======================================================
________________________________________________ ERROR collecting _________________________________________________
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/py-1.4.31-py2.7.egg/py/_path/common.py:332: in visit
for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/py-1.4.31-py2.7.egg/py/_path/common.py:368: in gen
if p.check(dir=1) and (rec is None or rec(p))])
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/main.py:647: in _recurse
ihook = self.gethookproxy(path)
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/main.py:550: in gethookproxy
my_conftestmodules = pm._getconftestmodules(fspath)
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/config.py:297: in _getconftestmodules
mod = self._importconftest(conftestpath)
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/config.py:333: in _importconftest
self.consider_conftest(mod)
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/config.py:356: in consider_conftest
if self.register(conftestmodule, name=conftestmodule.__file__):
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/config.py:208: in register
ret = super(PytestPluginManager, self).register(plugin, name)
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:370: in register
hook._maybe_apply_history(hookimpl)
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:747: in _maybe_apply_history
res = self._hookexec(self, [method], kwargs)
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:338: in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:333: in <lambda>
_MultiCall(methods, kwargs, hook.spec_opts).execute()
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py:596: in execute
res = hook_impl.function(*args)
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/astropy/tests/pytest_plugins.py:137: in pytest_configure
if not config.getoption('remote_data'):
../../homebrew/Cellar/python/2.7.12/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_pytest/config.py:1061: in getoption
raise ValueError("no option named %r" % (name,))
E ValueError: no option named u'remote_data'
============================================= 1 error in 1.47 seconds ==============================================
I also confirmed that removing the conftest.py I mentioned above resolves the issue, which implicates the from astropy.tests.pytest_plugins import * line. Hence the moved issue.
Activity
eteq commentedon Oct 12, 2016
@josePhoenix - can you clarify why you need
py.testto work? That is, is there some specific reason whypython setup.py testis insufficient, or is it just that usingpy.testfeel more natural?josePhoenix commentedon Oct 12, 2016
pytestexecutablesetup.py testFor projects I'm part of that use more-or-less "vanilla" PyTest, the instructions are "cd projdir/; pytest". If questions come up, I can refer to http://docs.pytest.org/.
For affiliated packages, the instructions are "python setup.py test". That much is equivalently simple. But as soon as you need to go beyond that, all bets are off. The command line arguments are not documented anywhere beyond the level in the
python setup.py test --helpoutput. I opened an issue with astropy-helpers because I'm using the package template and the setup.py customizations live in the helpers, and I was told it's actually in astropy. I guess there's documentation in the astropy docs (partially reproducing the pytest documentation), but I have yet to meet a casual contributor that reads all the documentation of the project they're contributing to and all the documentation of its dependencies.But really, at the end of the day, I'm quite lazy and would simply rather type
than
to accomplish the same thing.
eteq commentedon Oct 26, 2016
@josePhoenix - I'm 100% with you that it should work - I was asking to see whether there's a particular use case missing from the existing machinery that you need it for. So I'm not arguing that you shouldn't be able to use
py.testhere, but just letting you know that two of these are supported by thepython setup.py testcase to let you work around it for now:You can use
ASTROPY_USE_SYSTEM_PYTEST=1for this if you want - then it will use whatever the system's default pytest is, which you can set to whichever one you want. Definitely not as easy as giving the executable directly, but it will work in a pinch.And note that if you want to force this for your package so that it never uses the bundled
py.test, you can set that to be the default in thesetup.cfg. That's the intended way to configure all of the test stuff (whichpytestinvocations should respect... but clearly the bug here is preventing that)They should be, using the
-a/--argspython setup.py testoptions. Anything provided there gets passed directly topy.test.Just to be clear, this is subtly different from the
python setup.py testcase. The former runs the test in the source directory and the latter in the build directory. These are subtly different, and usually the latter is slightly safer. So it's not quite identical...All of that said, there is clearly a bug here that should be fixed! I just want to make it clear that the
python setup.py testmachinery is there for a reason, not just to make for longer command-line typing.eteq commentedon Oct 26, 2016
@pllim @astrofrog @embray - do any of you understand why this is happening? It seems like some of the plugin machinery is not getting recognized and its hence failing due to a lack of the remote_data markings.
I'll try to dig into it when I get a chance if no one else has any ideas...
pllim commentedon Oct 26, 2016
I can't comment much... Sorry.
josePhoenix commentedon Oct 26, 2016
Well, it's from this commit: https://github.com/astropy/astropy/pull/2044/files#diff-e602ffb35fbd67477e5543c9cc3846d4R132
And most likely because no default was specified (see http://docs.pytest.org/en/latest/writing_plugins.html?highlight=getoption#_pytest.config.Config.getoption ).
pllim commentedon Oct 26, 2016
👍 @josePhoenix , you are welcome to submit a fix to Astropy.
Fix missing default in during pytest_configure
pllim commentedon Feb 3, 2017
Is this still an issue? There are PRs to fix compatibility with pytest 3 and also to update the doc for testing.
pllim commentedon Apr 18, 2017
Doc was updated and changes to deal with pytest 3.x were merged. Closing for now. Please re-open if you run into this again.