Repository navigation
Error when generating WCS documentation #117
Description
When build the latest docs, I get the following error:
Traceback (most recent call last):wcsprm
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sphinx/ext/autodoc.py", line 321, in import_object
__import__(self.modname)
ImportError: No module named _wcs
/Users/tom/tmp/astropy/docs/wcs/api_wcsprm.rst:14: WARNING: autodoc can't import/find class 'astropy.wcs._wcs.Tabprm', it reported error: "No module named _wcs", please check your spelling and sys.path
Activity
eteq commentedon Dec 30, 2011
Odd, I do not see this error (either on Ubuntu 11.04 nor OS X 10.6 w/ py2.7) - the docs build without any problem.
astrofrog commentedon Dec 30, 2011
I'm using Sphinx 1.1.2. I just tested with a clean master, and it still occurs. The error occurs because of this autoclass statement:
which expects the built C extension. But even after running
python setup.py buildorpython setup.py installI can't get that error to go away, maybe because we now include theastropysource directory in the tree correctly? (https://github.com/astropy/astropy/blob/master/docs/conf.py#L24) - and that source directory doesn't contain the built c extensions.mdboom commentedon Dec 30, 2011
Are you installing astropy into a virtualenv, perhaps? The sphinx-build being used to build the docs must be able to import the installed astropy. What I generally do is install sphinx into my virtualenv or add my virtualenv to PYTHONPATH to get around this limitation.
astrofrog commentedon Dec 30, 2011
I'm not using a virtualenv. Just to make sure I understand, the following line in
conf.pyshould force the source directory to be used instead of the installedastropy:If I change the line to:
as it used to be (which was wrong), then the installed version is used and I don't see this error anymore. Do your PYTHONPATH edits override the above setting and place the installed astropy first in the path, in front of the source directory?
eteq commentedon Dec 30, 2011
Are you building the docs by doing
python setup.py build_sphinx, ormake htmlin the docs directory?astrofrog commentedon Dec 30, 2011
I'm using the latter
eteq commentedon Dec 30, 2011
Maybe try the former? it should do exactly the same thing as doing
python setup.py buildfollowed by themake htmlthing... but maybe its getting confused about which directory you're in?You might also try uninstalling the system-installed version (or
setup.py developlink) and seeing if you can get it built that way - if it builds there but doesn't with the install, then at least we know its got to do with the installed version instead of the actual building process...astrofrog commentedon Dec 30, 2011
If I remove any installed version, clone a clean copy of the master branch in the upstream repository, and run:
I get:
(note that the error is still there). Am I doing something wrong?
eteq commentedon Jan 2, 2012
I was able to reproduce it by starting a fresh clone, and I think I understand what's happening: when building, the docs inject
..intosys.modules, as @astrofrog pointed out... but if you start fresh and only dopython setup.py build(or anything that triggers the build), the C extensions get put into `build/lib.platformstuff/...``, and the docs only see the normal source tree, which does not include the built versions.These was working for me (and presumably @mdboom) because I use
python setup.py develop, which puts the compiled code inside the source directory... but we definitely want it to work the way you tried to do it. I think the solution is to inject../build/lib.whateverintosys.modules(after..), but I'm not sure what the best way is to figure out what thelibdirectory is. Does anyone know if there's some stdlib or setuptools function that can tell you this?astrofrog commentedon Jan 2, 2012
I think adding the
liblibrary frombuildwill be a bit messy, as the build directory can contain several lib directories (from different python versions). The cleanest way is probably just to not inject anything intosys.modules, and to just requireastropybe installed in order to build the docs. One could always consider adding..tosys.modules, but at the end, rather than at the start, so that it is only used as a last resort, and would not include compiled C extensions.eteq commentedon Jan 2, 2012
Well, my thought was that there's always only one
build/lib.whateverdirectory for any given python version/platform... and the information is all in theplatformstdlib module, so it shouldn't be that hard to make sure that only the appropriatebuild//libdirectory is added. I was just wondering if anyone knew of a "standard" way to do it (perhaps @iguananaut or @mdboom?). It might just be hidden somewhere in distutils, in which case we might as well use it.I, at least, find it frustrating to use the as-installed version for building the doc. I know some projects do this, but I sometimes want to build docs for a different version than the one I have installed (or even for a package I don't necessarily want to install at all), which is then impossible. I've also built the wrong docs many times by not doing
setup.py installfirst -- that's particularly annoying if you're doingmake htmlbecause you have to always go back to the source dir or have two consoles open. If we put..first, followed bybuild/lib.whatever, I think that's the least error-prone way.eteq commentedon Jan 3, 2012
I did a few tests after finding
distutils.util.get_platform... see eteq/astropy@4bedaf8 - that does what I was suggesting above...However, it reveals another problem: because we do
import astropyinsetup.py, it still imports the in-source version if I dopython setup.py build_sphinx, although it now works perfectly if you domake htmlinside the docs directory. We could add some extra bits to make setup.py use thebuildversion of astropy (if it's present) to get around this, or play around with imports in thebuild_sphinxcommand (which is already customized for us). Alternatively, we could do as @astrofrog suggests, and put..(and perhaps theget_platformthing I did in the above commit) at the end...phn commentedon Jan 3, 2012
I was wondering whether using something like Tox is better than attempting to add too many modifications to sys.path and other variables.
One issue with Tox is that it creates the "sdist" archive and then tries to build the entire package, after copying the "sdist" to an isolated virtualenv. This means that testing only the documentation build may not be possible.
Tox also has a
commandsoption that can run arbitrary commands. So, for example one can sayto run tests using
py.testshipped with astropy.I am not an expert, so can't think of potential problems.
taldcroft commentedon Jan 3, 2012
I haven't tested with astropy just now, but in the past I got around this problem with
That builds the compiled extensions within the source dir.
mdboom commentedon Jan 3, 2012
@phn: I haven't spent much time with Tox, but building docs without first building the code is impossible by design anyway, so I wouldn't worry about being about to build docs separately from anything else.
@eteq: It would be nice to simplify this, but there's a problem with being too magical here since we don't (currently) control how and where Sphinx was installed and building in place is such a problem with multiple branches of astropy or multiple versions of python.
The doc build works by running a Makefile which calls sphinx-build in the shell. This sphinx-build may or may not be the same Python that was invoked by
python setup.py build_sphinx. So one could build astropy in place and still fail because Sphinx expects the C extensions to be compiled for a different version of Python.I think I fall on @astrofrog's suggestion that you kind of need to know what you're doing and we should inject anything into
sys.path. The docs should just clearly say "install astropy, and make sure it is importable from the same environment as Sphinx is installed in".However, if we want something more automated, there may be a way to leverage setuptools and ensure Sphinx installed in the current Python environment, then set the SPHINXBUILD environment variable to point to that specific installation of the
sphinx-buildscript, then addbuild/platformto the path (don't rely on--inplace, it's too broken when using multiple versions of Python), and then use the makefile to build the docs.mdboom commentedon Jan 3, 2012
Scratch my comment about
build_sphinxfailing if using a different python interpreter from the one where sphinx is installed.build_sphinxdoesn't use the Makefile -- I've been seeing problems for an unrelated reason. I suppose I'm ok with addingbuild/platformtosys.pathin that case -- and we can do it from our custombuild_sphinxcommand and leave the rawmake htmlinvocation alone.eteq commentedon Jan 4, 2012
@taldcroft -
build_ext --inplaceI think does the same thing asdevelopin terms of building - so that works, but I don't think we want to do that if we want to try to be using the standard as-built version.@mdboom - just so I understand, in your second post you're suggesting have it first try the installed version, and then falling back on
build/platform, or the other way around?astrofrog commentedon Jan 4, 2012
I agree with @mdboom that we can do the magic for the
build_sphinxcommand, and that themake htmlshould just use the installed version (so we should remove thesys.pathcommand fromconf.py).mdboom commentedon Jan 4, 2012
@eteq: I wasn't clear... but I was suggesting that
build_sphinxwould use the version inbuild/platformand thatmake htmlwould do nothing magical (i.e. just use whatever Python paths the user may have already set up).eteq commentedon Jan 5, 2012
Ah, now that I get it, that sounds like a good plan to me as well.
I can go ahead and do this as part of #115, or we can have a separate pull request to address this and I can rebase #115 against that - either way is fine with me.
eteq commentedon Feb 14, 2012
This should be closed when #171 is closed.
astrofrog commentedon Apr 18, 2012
Now that #187 has been merged in, I think this issue can be closed. I no longer see the WCS error I originally reported in this issue. @eteq - can you confirm this can be closed?
eteq commentedon Apr 18, 2012
Yep, this is now fixed (at least, it's no longer the docs' fault if wcs doesn't build! :)
Merge pull request #117 from astrofrog/fix-frame-path