ipython_directive @savefig: should detect missing static html dir and not place output in source dir

The ipython_directive's @savefig currently has its default output set relative to the static html directory. If that directory is missing, it places the output outside of the source tree. The build succeeds with a warning from Sphinx that it cannot find the image—which does not help to find the source of the problem. Solution: raise an error/warning if the directive's output path is invalid.
@savefig stores its output in the source directory. A better default would be to store it in build/_static.

@stefanv

@stefanv can you test #8888? I think it fixes these things, but I'm not super experienced with this stuff.

That does not seem to work for me, but it may very well be that I am not testing correctly. The repository I am testing against is

https://github.com/statlab/permute

Where I see:

internal padding after /home/stefan/akad/projekte/permute/doc/source/user/npc/kenya.rst:30: WARNING: image file not readable: user/npc/../../../build/html/_static/test.png

(after commenting out ipython_savefig_dir from the config)

Thanks for testing, I'll use that repo to figure out what I'm not quite doing correctly.

@stefanv

@stefanv I do not seem to be able to reproduce using permute and coomenting the ipython_savefig_dir could sphinx-build have picked up a ipython savefig directive from another env ?

 git diff -U0
diff --git a/doc/source/conf.py b/doc/source/conf.py
index 4fb1ba1..a6bff13 100644
--- a/doc/source/conf.py
+++ b/doc/source/conf.py
@@ -147 +147 @@ html_static_path = ['_static']
-ipython_savefig_dir = '../build/html/_static'
+# ipython_savefig_dir = '../build/html/_static'

$ make html
sphinx-build -b html -d build/doctrees   source build/html
Running Sphinx v1.3.3
making output directory...
loading pickled environment... not yet created
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 29 source files that are out of date
updating environment: 29 added, 0 changed, 0 removed
checking for /Users/bussonniermatthias/dev/permute/doc/source/permute.bib in bibtex cache... not found
parsing bibtex file /Users/bussonniermatthias/dev/permute/doc/source/permute.bib... parsed 10 entries
reading sources... [100%] user/two-sample
/Users/bussonniermatthias/dev/permute/permute/core.py:docstring of permute.core.one_sample:37: ERROR: Undefined substitution referenced: "z_i".
/Users/bussonniermatthias/dev/permute/permute/core.py:docstring of permute.core.two_sample_conf_int:30: ERROR: Undefined substitution referenced: "F_x(t) - F_y(t)".
/Users/bussonniermatthias/dev/permute/permute/core.py:docstring of permute.core.two_sample_shift:33: ERROR: Undefined substitution referenced: "F_x(t) - F_y(t)".
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] user/two-sample
generating indices... genindex py-modindex
highlighting module code... [100%] permute.stratified
writing additional pages... search
copying images... [100%] user/npc/../../../build/plot_directive/user/npc/kenya-1.png
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded, 3 warnings.

Build finished. The HTML pages are in build/html.

Note that I needed to work around matplotlib/matplotlib#5884 while building.

@stefanv

Note that one now has to roll permute back to an earlier commit to test this, because the example @stefanv referenced has been switched to use a matplotlib directive.

I also couldn't reproduce this. I'm dropping it from the 4.1 milestone because there's nothing we can really do until someone can reproduce it.

If you can't reproduce, let's close this and see if we run into the same problem again?

Sounds reasonable. We can always reopen it if we do. Thanks Stefan.

I think I'm running into this issue on 4.0.1 and a related issue on 4.1.1

Sorry in advance for not putting together a minimum reproducible example and pointing you to my mess of code...

I'm trying to build the documentation for this project:

https://github.com/wholmgren/pvlib-python/tree/pvsystem

The figures on this rtd page will appear if I use IPython 4.0.1, but they will not appear using IPython 4.1.1.

I think this is because on 4.0.1, IPython puts the figures in my source/_static if the build directory does not exist, and then they eventually get copied over to the right spot. So it's not ideal, as described above, but at least it works. However, on 4.1.1, IPython puts the figures in the directory from which I'm running make html and they never make it to the right place.

Reopening to have look.

@saimn

Thanks for the info. Here is a build with ipython 5.1 : https://readthedocs.org/projects/xray/builds/4760089/ The build succeeds but the images are not copied correctly, very likely because of what @saimn said above. We are pinned on 4.0.3 since then, which currently suits our needs.

Create savefig_dir in Sphinx IPython directive

I think the widgets warning is unrelated to this error.

Can anyone who's having trouble with this try #10183?

@takluyver

Thanks @takluyver !

I think this is on the right track, but unfortunately RTD seems unhappy with this: https://readthedocs.org/projects/xray/builds/4916490/

Partial traceback:

(...)
 File "/home/docs/checkouts/readthedocs.org/user_builds/xray/conda/fix-docs/lib/python3.5/site-packages/IPython/sphinxext/ipython_directive.py", line 930, in run
   rgxin, rgxout, promptin, promptout = self.setup()
 File "/home/docs/checkouts/readthedocs.org/user_builds/xray/conda/fix-docs/lib/python3.5/site-packages/IPython/sphinxext/ipython_directive.py", line 898, in setup
   os.makedirs(os.path.abspath(savefig_dir))
 File "/home/docs/checkouts/readthedocs.org/user_builds/xray/conda/fix-docs/lib/python3.5/os.py", line 241, in makedirs
   mkdir(name, mode)
OSError: [Errno 30] Read-only file system: '/home/docs/checkouts/readthedocs.org/readthedocs/templates/sphinx/_static'

Exception occurred:
 File "/home/docs/checkouts/readthedocs.org/user_builds/xray/conda/fix-docs/lib/python3.5/os.py", line 241, in makedirs
   mkdir(name, mode)
OSError: [Errno 30] Read-only file system: '/home/docs/checkouts/readthedocs.org/readthedocs/templates/sphinx/_static'

I'm not familiar enough with RTD and ipython to understand whats going on here. Also, I wonder what changed between ipython4 and ipython5 so that this change would be necessary :(

Looking at the error and the code, it seems to think that outdir is a template directory, for some reason. i'm rather puzzled by that. RTD, not unreasonably, appears to put that in a read-only location, so trying to create the directory there fails.

I think your config here is being ignored, by the way, because the option appears to be called ipython_savefig_dir rather than savefig_dir.

There were some changes which went into 4.1 (#8888). Since you're pinned to 4.0.3, maybe that's what broke it for you?

There were some changes which went into 4.1 (#8888). Since you're pinned to 4.0.3, maybe that's what broke it for you?

Yes, if I recall well I tried several options and 4.0.3 was the last one to work. Now I've tried a new build with your suggested changes modified to:

# Ensure that the directory for saving figures exists
try:
    os.makedirs(os.path.abspath(savefig_dir))
except:
    pass

(fmaussion@4e49fbd)

The build completes, but as before the images just won't show up... (log: https://readthedocs.org/projects/xray/builds/4917004/)

I don't really get it, do we do anything wrong here? :-( (btw, I also just removed the obsolete option you mentioned in our conf.py to do the test...)

Your change silences the error creating the directory, but if it can't create the directory, it won't be able to save images into it, so it's not fixing the underlying issue.

We calculate the savefig directory relative to the output directory:
https://github.com/takluyver/ipython/blob/4dc7caa9ede16968301d3abe5f031120d4de00d6/IPython/sphinxext/ipython_directive.py#L853

What I don't understand is that your error indicates that the output directory is /home/docs/checkouts/readthedocs.org/readthedocs/templates/sphinx/, which doesn't make much sense.

@takluyver

Thanks @takluyver , this is very helpful. I've been able to solve the problem by setting manually the ipython_savefig_dir option in conf.py:

ipython_savefig_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)),
'_build','html','_static')

I still think something should be solved upstream, as this option should be optional.

@minrk

Using https://github.com/karthik1024/trialproject , I've resolved the issue with images not showing up. But you still see the 'image file not readable' warning, because it expects to copy images from the source directory to the build directory. I'm not sure how to work around that other than writing the images to the source directory (i.e. reverting part of #8888).

@minrk - any thoughts? As you wrote #8888.

Put savefig images back in source directory

I've made #10186 as an alternative fix, if anyone wants to try it. It moves images back into the docs source directory, which gets rid of the warnings.

Backport PR #10186: Put savefig images back in source directory

Navigation Menu

Repository navigation

`ipython_directive` @savefig: should detect missing static html dir and not place output in source dir #8733

21 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

ipython_directive @savefig: should detect missing static html dir and not place output in source dir #8733

Description

Activity

minrk commented on Oct 9, 2015

stefanv commented on Oct 16, 2015

minrk commented on Oct 19, 2015

Carreau commented on Jan 19, 2016

takluyver commented on Jan 25, 2016

stefanv commented on Jan 25, 2016

takluyver commented on Jan 25, 2016

wholmgren commented on Feb 10, 2016

Carreau commented on Feb 10, 2016

21 remaining items

fmaussion commented on Jan 18, 2017

takluyver commented on Jan 19, 2017

fmaussion commented on Jan 19, 2017

takluyver commented on Jan 19, 2017

fmaussion commented on Jan 19, 2017

takluyver commented on Jan 19, 2017

fmaussion commented on Jan 19, 2017

takluyver commented on Jan 19, 2017

takluyver commented on Jan 19, 2017

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

`ipython_directive` @savefig: should detect missing static html dir and not place output in source dir #8733