Repository navigation
Use sphinxcontrib-globalsubs to define reStructuredText substitutions currently in docs/common_links.txt #15149
Description
Currently, Astropy uses docs/common_links.txt as a means of storing reStructuredText (reST) substitutions. In docs/conf.py, these substitutions get read in, and then put into rst_epilog which is what gets appended to every reStructuredText file (and I think every docstring too) when Sphinx builds the documentation.
We set something similar up for PlasmaPy because this practice does make writing documentation easier. I've been trying to find ways to hasten our doc build, and made a PR to remove some substitutions from our docs/common_links.rst and put the substitutions into the main documentation source file that the substitutions were used in (PlasmaPy/PlasmaPy#2272). Doing this sped up the documentation build up quite a lot (see PlasmaPy/PlasmaPy#2277). I did some test doc builds and found that removing every line from our docs/common_links.rst sped up the doc build from ∼421 to ∼266 s. My hypothesis is that the crux of the issue is that having a long rst_epilog is slowing doc builds since it gets appended to every source file.
My suggestion would be to investigate what the performance penalty is for putting numerous substitutions in docs/common_links.txt in Astropy doc builds. If it does turn out to have a significant impact on performance, then it would be worth keeping only the most useful substitutions in docs/common_links.txt or finding an alternative means for defining these substitutions (via a reST extension?). (I would have been great if there was a way to define substitutions & links via a dictionary in docs/conf.py!)
See also #11063, #11599, & #14560. Probably of interest to @nstarman.
Thanks!
Activity
namurphy commentedon Aug 8, 2023
Looks like there's a thing! sphinxcontrib-globalsubs
It was mentioned in sphinx-doc/sphinx#2173.
I'm going to rename this issue to implement substitutions via that.
I created a parallel issue in PlasmaPy (PlasmaPy/PlasmaPy#2280). It would be worth working together to implement this for both projects since we're currently doing pretty much the same thing.
[-]Investigate doc build performance penalty from putting reST substitutions in `rst_epilog`[/-][+]Use `sphinxcontrib-globalsubs` to define reStructuredText substitutions currently in `docs/common_links.txt`[/+]nstarman commentedon Aug 8, 2023
Thanks @namurphy, this looks great! I'm more than happy to participate in discussion and review PRs on this. My plate is a little full ATM to lead the Astropy-side of implementing this.
namurphy commentedon Aug 9, 2023
I'm implementing
sphinxcontrib-globalsubsin PlasmaPy/PlasmaPy#2281. We usesphinxcontrib-globalsubsby creating aglobal_substitutionsdictionary inconf.pywith the substitution name as the key and the substitution as the value.I ended up creating
docs/_global_substitutions.pyto defineglobal_substitutions, and then importedglobal_substitutionsintodocs/conf.py.The most time-consuming part was that
sphinxcontrib-globalsubsdoesn't seem to directly handle Sphinx links (e.g., likeSphinx_which was defined via a line like.. _Sphinx: https://sphinx-docs.org...). I ended up converting these links to substitutions instead, and then had to do text replacements for changing the links to substitutions throughout the documentation.Repeating the benchmark on my home computer, moving everything out of
rst_epiloglet us speed up our Sphinx build from 415s to 200 s!nstarman commentedon Aug 10, 2023
Sounds great! And a nice speed boost. I don't think Astropy's
common_linkshas too many Sphinx links, mostly subs, so hopefully implementing this for Astropy won't be too painful. Do you have any interesting in spearheading this PR?namurphy commentedon Aug 17, 2023
I might be able to do so, but I don't know if I'll have the time or capacity to do so in the next ∼2.4 months. In the meantime, if anyone else wants to take this on, please feel free to do so. Thank you!