Repository navigation
Extract the development docs from this repository #22866
Description
We have historically built all of our documentation from the sphinx source in the main matplotlib/matplotlib repo (this one). We would then host this by unpacking the result of the build at the top level of website + into a version specific sub-folder. We would also rely on
While this served us well for over a decade, over the past few years we have slowly started to pull things apart. The goal is to ultimately de-couple the version independent parts of the documentation (which should be re-published as soon as they are update) from the version-specific docs (which should be Steps we have already taken so far:
- move the cannonical docs to https://matplotlib.org/stable https://discourse.matplotlib.org/t/canonical-documentation-have-moved/21863
- split the third-party docs out into its own repo https://github.com/matplotlib/mpl-third-party
- have a sphinx theme for all of our websites https://github.com/matplotlib/mpl-sphinx-theme
- split the top-level index into its own repo https://github.com/matplotlib/mpl-brochure-site
- have built docs for the governance https://github.com/matplotlib/governance
- moved our ground-truth hosting to digital ocean https://github.com/matplotlib/matplotlib.org
The next thing we should pull out into its own repository is the content under https://matplotlib.org/stable/devel/index.html . We probably should copy-paste the GHA/sphinx configuration from the governance repo and update the rst in that framework.
We should do this as a second step after we finish @noatamir 's update of that content (to make it easier to review the changes at each step).
Activity
QuLogic commentedon Apr 22, 2022
Do we want to put this in
mpl-brochure-siteor a separate repository?tacaswell commentedon Apr 23, 2022
Originally I thought
mpl-brocure-site, but I now I think a separate repository. We already have a whole bunch of repos serving docs so we might as well lean into that and separate the repos by purpose.timhoffm commentedon Apr 25, 2022
Makes sense. It does not fit into
mpl-brochure-site.We may need a place to document which repositories are used for which parts of the docs. - Should this be in a separate repo as well? 😜
noatamir commentedon Apr 26, 2022
Maybe just in the docs themselves?
An overview of the structure/architecture of the docs would fit in the docs no?! it's a bit meta but makes sense in a way...
story645 commentedon Apr 26, 2022
Could maybe go in the dev guide for doc contributions since "where is this thing I want to change?" is a not infrequent new contributor question.
jklymak commentedon Apr 26, 2022
I personally think we should have as few repos as possible. i.e. one tied to releases, and the other not tied to anything. mpl_third_party is a bit different as it has its own review and build pipeline.
tacaswell commentedon Apr 26, 2022
Currently we are pulling the artifacts for our docs from 14 different repos (see
https://github.com/matplotlib/matplotlib.org/blob/e57c891aaddf8fda1c5fd06dcb5c99875e2c05bf/matplotlib.org.yml#L10-L24) so although I agree with the sentiment, I think we are well past hoping to keep the number of repositories involved to very small numbers. The very large number of repos contributing to the docs is not new, we were previously exploiting the implicit layering that github pages so it was much harder to see!
If we focus just on the core library docs we have:
I've been thinking about security for unrelated reasons recently. I am not concerned about what people we know and have given access to are going to do, however I am concerned about what someone who gets control of one of our accounts might do which is why I want to restrict write permissions on the goverance and very repos that are the source for the high-traffic parts of our website.
I do not think there is anything we really want to merge together, we are already paying the complexity cost of "more that one" at the caddy configuration level, and wanting to have different review/merge rules on https://matplotlib.org and the development guide I think we should go with (just) one more repo and include the above in the devel guide as a part of how to work on the docs.
jklymak commentedon Apr 26, 2022
OK, that makes sense. maybe just leave it open to be more than just "dev guide" in case we want other docs not tied explicitly to the release cycle?
tacaswell commentedon Apr 26, 2022
That seems correct to me, but no better term is coming to mind quickly. I would also want to move the communication guidelines out of governance to the new repo along with guides/templates on how to handle common situations with (new) contributors ("we do not assign issues, just start working", "congrats on your first PR, hope to here from you again!", "this better handled on discourse", "can you please include a minimal running example + version numbers", "your local installation is broken, uninstall everything and try using just 1 (one) package manager", etc), the triage guide, the PR review guide, etc, the meta-doc on docs, etc.
In our usual way, https://matplotlib.org/stable/devel/index.html is
develin the source tree, "Develop" in the site header, and has an H1 of "Contributing" (with a nested H2 that is also "Contributing")."contributing-and-maintaining-and-community-guide" is too long.
story645 commentedon Nov 28, 2023
Cause this is coming up in #27386, over the past year the the dev docs have been updated and gitwash stripped out (thanks @melissawm!). I'd prefer #27265 to go in before we move things (especially since it includes a 'where are docs section' I can flesh out).
Broadly speaking, the version independent docs are consolidated to:
The plan for resources was/is that it should get something like the third-party-package repo so it sort of doesn't matter, and I think a follow up PR to a migration is updating cross links using intersphinx (which there's a configuration we can use to treat it as one big doc so none of the links break so we may not even need to).
ETA: the users/project docs are also in a weird space in that for the most part I think the contents of those docs should be very difficult to change - it's mission/code of conduct/licence/history/credits - w/o major discussion. Like our CoC or mission shouldn't be changeable by way of one approval unless it's like a typo/formatting fix.
jklymak commentedon Nov 28, 2023
I don't think we need to move anything - simply point
/stable/develto/devdocs/devel, etc for other things that should be current. Or if we don't want folks popped intodevdocs, just automatically rsync the subdirectory into/stablewhenever/devdocsis updated. When this last came up, I don't think we had the possibility of server side redirects.jklymak commentedon Nov 28, 2023
Ping @QuLogic for an opinion on if either of the above are doable.
timhoffm commentedon Nov 29, 2023
I feel splitting dev instructions out into a separate repo decouples them too much, and adds extra burden (separate repo to configure and handle).
Primary requirement: The dev instructions should be updated contiuously. Typically dev happens on main and instructions how to work there are relevant, not the dev instuctions that were active during the latest release.
I see two possible ways to achieve this:
Advantages:
IMHO the additional aspects we get from a separate repo, like different permissions, are not relevant here.
QuLogic commentedon Nov 30, 2023
On the server, there are two options:
We would need to be a bit careful if
devdocsrenamed/moved a file compared tostable, though we do try to insert redirect HTML when we do. The other downside is that then all other links are todevdocs.I don't think
rsyncis viable, asdevdocsis in a separate repo from the versioned docs, and both sides are git checkouts.story645 commentedon Nov 30, 2023
I think setting a couple of the high profile links to contribute to the dev docs may catch most of the cases if we want something on the simple side:
jklymak commentedon Nov 30, 2023
Probably a redirect is more appropriate, unless all the links in devel are relative, which I doubt they are.
I think we need to be careful to do this anyways, so I don't think thats too strong an objection.
I think that is OK, and a strong argument for a redirect versus rewrite. If you want to go to
develfor some reason, it's not too likely you are then likely to get confused if you are not onstable/examplesanymore. Whereas clearlystable/develis confusing.QuLogic commentedon Dec 1, 2023
Here is the followup PR from the meeting discussion: matplotlib/matplotlib.org#36