Skip to content

Speed up doc build by factor of 2 by defining substitutions via sphinxcontrib-globalsubs instead of common_links.rst - #2281

Merged
namurphy merged 34 commits into
PlasmaPy:mainfrom
namurphy:sphinxcontrib-globalsubs
Aug 24, 2023
Merged

namurphy merged 34 commits into
PlasmaPy:mainfrom
namurphy:sphinxcontrib-globalsubs

Conversation

@namurphy

@namurphy namurphy commented Aug 8, 2023 •

Copy link
Copy Markdown
Member

This PR implements sphinxcontrib-globalsubs as a Sphinx extension to allow us to define reStructuredText substitutions that can be used throughout PlasmaPy's documentation.

The substitutions are now defined in docs/_global_substitutions.py, which defines the global_substitutions dict which is imported in docs/conf.py, and is what sphinxcontrib-globalsubs uses to define global substitutions. I moved all substitutions out of common_links.rst. Substitutions are now defined in _global_substitutions.py via a dictionary, or in the file the substitutions are used in.

Motivation and context

Prior to this pull request, we defined global reStructuredText substitutions in docs/common_links.rst. The contents of common_links.rst got added to the rst_epilog Sphinx configuration variable, which is used to append text to every reStructuredText source file. The problem with this approach is that it can add ∼2–3 minutes to an already slow documentation build. This PR lets us skip using rst_epilog and thus speeds up the documentation build.

Related issues

See #2277 and #2280 for more detail. Closes #2280. Also discussed in astropy/astropy#15149. I learned about sphinxcontrib-globalsubs in sphinx-doc/sphinx#2173.

@github-actions

github-actions Bot commented Aug 8, 2023

Copy link
Copy Markdown

Thank you for submitting a pull request (PR) to PlasmaPy! ✨ The future of the project depends on contributors like you, so we deeply appreciate it! 🌱

Our contributor guide has information on:

The bottom of this page shows several checks that are run for every PR. Don't worry if something broke! We break stuff all the time. 😺 Click on "Details" to learn why a check didn't pass. Please also feel free to ask for help. We do that all the time as well. 🌸 You can find us in our chat room or weekly community meeting & office hours. Here are some tips:

  • Try fixing CI / Python 3.11 test failures first.
  • Most pre-commit.ci - pr failures can be automagically fixed by commenting pre-commit.ci autofix below, followed by a git pull to bring the changes back to your computer.
  • If pre-commit.ci - pr says that a function is too long or complex, try breaking up that function into multiple short functions that each do one thing. See also these tips on writing clean scientific software.
  • If the CI / Documentation check ends with a cryptic error message, check out our documentation troubleshooting guide.
  • For a documentation preview, click on Details next to docs/readthedocs.org:plasmapy.

If this PR is marked as ready for review, someone should stop by to provide a code review and offer suggestions soon. ✅ If you don't get a review within a few days, please feel free to send us a reminder.

Please also use SI units within PlasmaPy, except when there is strong justification otherwise or in some examples.

We thank you once again!

@github-actions github-actions Bot added docs PlasmaPy Docs at http://docs.plasmapy.org packaging Related to packaging or distribution labels Aug 8, 2023
@github-actions github-actions Bot added the requirements Related to updating requirements label Aug 8, 2023
@codecov

codecov Bot commented Aug 8, 2023 •

Copy link
Copy Markdown

Codecov Report

Patch and project coverage have no change.

Comparison is base (1a07e49) 98.35% compared to head (b343439) 98.35%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2281   +/-   ##
=======================================
  Coverage   98.35%   98.35%           
=======================================
  Files          99       99           
  Lines        8554     8554           
=======================================
  Hits         8413     8413           
  Misses        141      141           
Files Changed Coverage Δ
plasmapy/plasma/sources/openpmd_hdf5.py 82.41% <ø> (ø)
plasmapy/utils/data/downloader.py 100.00% <ø> (ø)

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@namurphy

namurphy commented Aug 8, 2023

Copy link
Copy Markdown
Member Author
  • I needed to set ruff's configuration to allow docs/conf.py to contain unused imports. This is to allow us to import Sphinx (& Sphinx extension) configuration variables to be imported from other files.

  • One thing that tripped me up was to add the extension as sphinxcontrib.globalsubs (not sphinxcontrib_globalsubs).

  • Since we're defining global substitutions via a Python dict, we could potentially generate a contributor guide table that contains the substitutions that can be used in the docs. It could be in the documentation guide, or on its own page (with a link from the documentation guide).

@namurphy

namurphy commented Aug 8, 2023

Copy link
Copy Markdown
Member Author

From #2277:

Build of main with the full common_links.rst: 421 s
Build in #2272 which moved some of common_links.rst to the doc guide: 344 s
Build with empty docs/common_links.rst: 266 s (with a whole bunch of couple hundred warnings)

In the most recent doc build, it was 384 s, which doesn't include the changes from #2272. Better!

(I did these on my work computer using tox -e build_docs_pins.)

@namurphy

namurphy commented Aug 8, 2023 •

Copy link
Copy Markdown
Member Author

Merging in #2272 gets it down to 317 s. Much better! And that's still with a bunch of links in docs/common_links.rst that we can excise. I think we can still do better.

@github-actions github-actions Bot added the plasmapy.plasma Related to the plasmapy.plasma subpackage label Aug 8, 2023
@namurphy namurphy added the no changelog entry needed For PRs that should not have a changelog entry label Aug 9, 2023
Comment thread changelog/2281.doc.rst Resolved
Comment thread docs/common_links.rst

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously, the contents of this file were included in rst_prolog, which is a Sphinx configuration variable containing text that gets appended to every file and docstring before Sphinx processes it. We had been using it to define substitutions. However, adding this to every source file and docstring added ∼3 minutes to an already long documentation build. Replacing this file via sphinxcontrib-globalsubs was the main point of this PR.

Comment thread docs/conf.py Resolved
@namurphy

namurphy commented Aug 9, 2023 •

Copy link
Copy Markdown
Member Author

Benchmark on my home computer using tox -e build_docs_pins (and deleting docs/_build in between). This is the output enabled by enabling sphinx.ext.duration. The timing variations for the build are somewhere around ±10 s.

With #2272 and this PR, we've sped up the Sphinx build by a factor of two!

@namurphy
namurphy marked this pull request as ready for review August 9, 2023 17:10
@namurphy
namurphy requested a review from a team as a code owner August 9, 2023 17:10
@namurphy
namurphy requested review from JaydenR2305 and removed request for a team August 9, 2023 17:10

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the main file to review. Thank you!

Comment on lines +160 to +162
global_substitutions = (
plasmapy_subs | doc_subs | numpy_subs | astropy_subs | processed_links
)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Union operator for dictionaries...added in Python 3.9!

@namurphy

namurphy commented Aug 9, 2023

Copy link
Copy Markdown
Member Author

The main thing to review is docs/_global_substitutions.py.

Most of the changes in the rest of the documentation are associated with changing reStructuredText links into reStructuredText substitutions (i.e. changing the Sphinx_ link to the |Sphinx| substitution). I already double checked those, so it should be safe to skip them.

Thank you!

@namurphy namurphy changed the title Implement sphinxcontrib-globalsubs Speed up documentation build by 2d7 sphinxcontrib-globalsubs Aug 9, 2023
@namurphy namurphy changed the title Speed up documentation build by 2d7 sphinxcontrib-globalsubs Speed up documentation build by factor of 2 by implementing sphinxcontrib-globalsubs Aug 9, 2023
@namurphy namurphy changed the title Speed up documentation build by factor of 2 by implementing sphinxcontrib-globalsubs Speed up doc build by factor of 2 by defining substitutions via sphinxcontrib-globalsubs instead of common_links.rst Aug 9, 2023
@github-actions github-actions Bot added CI Related to continuous integration linters Code linters and autoformatters labels Aug 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI Related to continuous integration contributor guide docs PlasmaPy Docs at http://docs.plasmapy.org linters Code linters and autoformatters no changelog entry needed For PRs that should not have a changelog entry packaging Related to packaging or distribution plasmapy.plasma Related to the plasmapy.plasma subpackage plasmapy.utils Related to the plasmapy.utils subpackage requirements Related to updating requirements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use sphinxcontrib-globalsubs to define reST substitutions instead of rst_epilog

1 participant