Skip to content

MRG: Add refs from docstring to backrefs - #347

Merged
choldgraf merged 9 commits into
sphinx-gallery:masterfrom
larsoner:docrefs
Jul 16, 2018
Merged

choldgraf merged 9 commits into
sphinx-gallery:masterfrom
larsoner:docrefs

Conversation

@larsoner

@larsoner larsoner commented Feb 9, 2018

Copy link
Copy Markdown
Contributor

Adds to backreferences any function/class that gets :func: or :meth:'ed in the docstring. This should be particularly useful for things like matplotlib.axes.Axes and similar classes that get instantiated only indirectly.

Closes #205.

@larsoner

larsoner commented Feb 9, 2018

Copy link
Copy Markdown
Contributor Author

@codecov-io

codecov-io commented Feb 9, 2018 •

Copy link
Copy Markdown

Codecov Report

Merging #347 into master will increase coverage by 2.17%.
The diff coverage is 97.22%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #347      +/-   ##
==========================================
+ Coverage   93.13%   95.31%   +2.17%     
==========================================
  Files          27       27              
  Lines        1953     1985      +32     
==========================================
+ Hits         1819     1892      +73     
+ Misses        134       93      -41
Impacted Files Coverage Δ
sphinx_gallery/py_source_parser.py 81.48% <ø> (ø) ⬆️
sphinx_gallery/tests/test_backreferences.py 100% <100%> (ø) ⬆️
sphinx_gallery/tests/test_full.py 100% <100%> (ø) ⬆️
sphinx_gallery/backreferences.py 96.72% <92.3%> (+7.43%) ⬆️
sphinx_gallery/gen_gallery.py 89.94% <0%> (+2.36%) ⬆️
sphinx_gallery/docs_resolv.py 83.85% <0%> (+13%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0a19f0f...901c371. Read the comment docs.

@jklymak

jklymak commented Jul 10, 2018

Copy link
Copy Markdown
Contributor

I think this solves a big problem for Matplotlib for which we've been using a hacky workaround. matplotlib/matplotlib#10974

@larsoner

Copy link
Copy Markdown
Contributor Author

I'm happy to rebase, IIRC the only remaining thing is reviews (@choldgraf ?)

@choldgraf

Copy link
Copy Markdown
Contributor

holy moly I didn't notice this one in February, but agree it would be super useful. I'll try to give it a review during one of my "I've been talking about code way too much I need to actually code a bit" breaks at SciPy :-)

@larsoner

Copy link
Copy Markdown
Contributor Author

Rebased and added fixes for our test infrastructure:

  1. CircleCI had library problems.
  2. Sphinx build needed to be done within the docutils_namespace context manager.

Example of np.sin manually placed in example docs to be in backrefs:

https://523-25860190-gh.circle-artifacts.com/0/rtd_html/index.html

New docs of feature:

https://523-25860190-gh.circle-artifacts.com/0/rtd_html/advanced_configuration.html#references-to-examples

Ready for review/merge from my end.

@lesteve

lesteve commented Jul 12, 2018 •

Copy link
Copy Markdown
Member

Example of np.sin manually placed in example docs to be in backrefs:

Looks nice! Just curious, would it be easy to make np.sin a link here to the np.sin documentation?

@larsoner

larsoner commented Jul 12, 2018 via email

Copy link
Copy Markdown
Contributor Author

@lesteve lesteve left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, a few comments/questions.

Comment thread doc/advanced_configuration.rst Outdated
The files we are including are from the ``backreferences_dir``
configuration option setup for Sphinx-Gallery.

.. _code_tesiii:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reference is not used anywhere is it?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see it added anywhere, @Titan-C what is this for?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since I added in doc/_templates/module.rst also the instruction to document classes I wanted to seed how this was affected on the line numbers. This reference was for personal use when reloading the HTML should have not been committed.
As a side issue. It appears that the readthedocs theme on a python3 build(at least mine) cannot align the line numbers with the code lines. Other sphinx themes and our py2 build on CircleCI do display correctly.

Comment thread sphinx_gallery/tests/test_full.py Outdated
buildername='html')

app.build(False, [])
# need to build with this mode for automodule and backrefs to work

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what this comment is about ... what do you mean by "this mode"? Is that related to moving app.build inside the with docutils_namespace()

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes exactly, if the app.build is not in the context manager the links are not actually made

assert 'numpy.arange.html' in lines
# assert '#module-matplotlib.pyplot' in lines
# assert 'pyplot.html' in lines
assert '#module-matplotlib.pyplot' in lines

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intersesting, I remember this was failing from time to time, but maybe now this is more stable?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah the CIs are happy so I figure we can try it, if it ends up being inconsistent again we can comment it back out

Comment thread sphinx_gallery/tests/test_full.py Outdated
mod_file = op.join(out_dir, 'gen_modules', 'sphinx_gallery.sorting.html')
with codecs.open(mod_file, 'r', 'utf-8') as fid:
lines = fid.read()
assert 'FileNameSortKey' in lines

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice test!

Maybe add a few comments before each assert to explain a bit what they are checking? Proposal (improvements more than welcome!):

  • first assert: "check that FileNameSortKey class appears in the reference API of the sphinx_gallery.sorting module"
  • second assert: "check that the backreferences work when

Bonus points if you can add a test that the "normal" (i.e. in Python code) backreferences works? (maybe there is one already I haven't checked ...)

@lesteve

lesteve commented Jul 13, 2018

Copy link
Copy Markdown
Member

Possibly but that's a separate issue (exists in master too I think) so I'd rather deal with it separately

Yep completely agree, I was asking more out of curiosity!

@larsoner

Copy link
Copy Markdown
Contributor Author

@lesteve comments addressed

Comment thread examples/plot_function_identifier.py Outdated

Calls Sphinx-Gallery identify names function to figure out which
functions are called in the script and to which module do they belong.
This demonstrates how Sphinx-Gallery identifies names function to figure out

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

names function function names ?

Comment thread examples/plot_function_identifier.py Outdated
It uses both the code itself, as well as the docstrings (such as this one),
as adding a reference to :func:`numpy.sin` and :func:`numpy.exp` will create
proper backreferences even if they are not explicitly used. This is useful
in particular when functions return classes -- if you add them to the docstring

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems from the regex below that this is true for any text block references yeah? Can we make this a bit clearer in the examples / documentation? They tend to refer to the docstring but this should also work within ####### blocks right?

@choldgraf

Copy link
Copy Markdown
Contributor

this is a clever way to support functions that aren't directly imported! Just a couple quick thoughts but in general it LGTM. +1 on a merge once my question on whether this works in more than just example docstrings is addressed

@larsoner

larsoner commented Jul 16, 2018 •

Copy link
Copy Markdown
Contributor Author

@Titan-C Titan-C left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great. +1 on my side.
I noticed a broken link in the docs, and have some irrelevant questions on the document style.

Yes Numpy and intersphinx is always a challenge, I agree just using obj because it works, other times I had to use other dark tricks. Most of the time is because this are C extensions.

Comment thread doc/advanced_configuration.rst Outdated
For more information on what files you can use, see `preparing your
repository <https://mybinder.readthedocs.io/en/latest/using.html#preparing-a-repository-for-binder>`_
in the `Binder documentation <docs.mybinder.org>`_ for more information on
in the `Binder documentation <docs.mybinder.org>`__ for more information on

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering why you need to change the underscore? Trying to figure out(still ignorant), I realized this link is broken. Without http sphinx considers it as a local link.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The _ makes it a link that can be used elsewhere, and it's named the same way so it creates a warning. The alternative would be just to not include the link in the second one, but this way makes it so that if one is removed both still work

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread doc/maintainers.rst
@@ -1,49 +1,52 @@
:orphan:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did sphinx complained this not being in any toctree? I missed that warning. But this makes me think if we should put this on our docs. Kind of reduce cognitive burden. All docs are online, but the release instructions are only in the repo.
Also why all the new white space?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did sphinx complained this not being in any toctree?

Yes I was getting rid of warnings

Also why all the new white space?

Getting rid of sphinx warnings about bad spacing (which are in master). But really these should all be part of the same bulleting indentation so it's a bugfix of sorts, too.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can add it to the docs later if someone is motivated, not sure there is much to gain (users don't care and maintainers can just look in the repo)

@choldgraf

Copy link
Copy Markdown
Contributor

I'm planning to merge this once appveyor is happy (and if github isn't broken), speak now or forever hold your peace!

@choldgraf
choldgraf merged commit 6a1810e into sphinx-gallery:master Jul 16, 2018
@choldgraf

Copy link
Copy Markdown
Contributor

actually it looks like appveyor did pass, but GitHub is just being wonky so the display isn't updating. I'm gonna merge since we've now got a bunch of +1s!

🎉 🎉 🎉 thanks @larsoner !!! 🎉 🎉 🎉

@larsoner
larsoner deleted the docrefs branch July 16, 2018 18:52
@lesteve

lesteve commented Jul 16, 2018

Copy link
Copy Markdown
Member

Nice!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants