You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We are building online HTML documentation + additional downloadable PDF and zipped HTML. The project uses many plot directives, and in Sphinx's conf.py we specify plot_formats = ['svg', 'png', 'pdf'] such that SVG images can be used for HTML builds and PDF images for the PDF build (PNG images are saved just in case, they shouldn't harm).
When I do sphinx-build -b singlehtml locally, only the necessary SVG images are copied to the output directory (into its _images subdirectory), as expected, producing a clean result. So I expect that the ZIP archive for “Zipped HTML” built by Read the Docs should have only the index.html file and the _images and _static subdirectories with only the necessary files.
Actual Result
The ZIP archive generated on Read the Docs (htmlzip in .readthedocs.yml; -b readthedocssinglehtmllocalmedia in the log file) contains all the generated image files, including absolutely unneedeed PNG and PDF and even .py files related to them, scattered everywhere, including the root (where only index.html should be) and copies of the subdirectories of our doc source tree.
I'm using Sphinx v4.5.0 locally, whereas our last RTD log says:
Running Sphinx v1.8.6
which seems to be some old branch (I don't know why RTD uses it by default).
However, when I was doing further investigations and tried to prepare a minimal working example, I've found that the “correct” local behavior apparently depends on the order in which different targets (html, singlehtml, latex) are built, and a significant problem is actually in the Matplotlib plot directive. Then I was distracted and didn't finish, but I'll try to localize the problem and report it to Matplotlib (and update this issue here).
Nonetheless, maybe you have some workaround? For example, is it possible to modify the output directory structure before it is packed into the ZIP archive? Using some declarative include/exclude list or maybe some imperative hook?
OK, looks interesting. I'm curious whether post_build mentioned on top of that page is executed before packing the Sphinx output into ZIP or after? Because in the former case it indeed can be easily customized to remove unnecessary files. (On the other hand, even if it runs after, but the location of the resulting ZIP file is known and writable, the archive itself can be cleaned-up, I hope...)
OK. Is then the ZIP file accessible to the post_build commands, such that we can do zip -d ... on it?
It probably is, but this isn't something we will guarantee for the file to always be there, I think the file is generated in the ../../artifacts/ directory (from the root of your project)
Yes, I was afraid that this might be fragile. On the other hand, it's not a big problem if zip -d fails, so this might be worth trying as a temporary solution.
Let me investigate the details with Matplotlib and see what can be done...
Hello 👋🏼 We standardized the output paths for all the formats that Read the Docs supports in #9888. This allow people to output, for example, a HTMLZip under _readthedocs/htmlzip/<project-slug>.zip to be recognized by Read the Docs and expose it as a downloadable content (docs.readthedocs.io/en/stable/downloadable-documentation.html) 💯
This is the exact directory that Read the Docs will read immediately before uploading this .zip file and host it. So, you can use build.jobs.post_build to modify the _readthedocs/htmlzip/<project-slug>.zip file as you want after Read the Docs built it but before uploading it.
At build.jobs.post_build you will be able to remove all the unnecessary files you don't want to be included in the final .zip file. Let me know if that works for your use case.
I'm closing the issue, but feel free to comment here or re-open if you consider it's not solved yet.
Activity
MikhailRyazanov commentedon Apr 2, 2022
Forgot to mention: we also have
in
conf.py, so there are really no links to any of these files in additional formats, neither to their “sources”.stsewd commentedon Aug 18, 2022
Hi, sorry for the late reply, can you make sure you are using the same sphinx version on RTD as you do locally? See https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html
MikhailRyazanov commentedon Aug 21, 2022
I'm using Sphinx v4.5.0 locally, whereas our last RTD log says:
which seems to be some old branch (I don't know why RTD uses it by default).
However, when I was doing further investigations and tried to prepare a minimal working example, I've found that the “correct” local behavior apparently depends on the order in which different targets (html, singlehtml, latex) are built, and a significant problem is actually in the Matplotlib plot directive. Then I was distracted and didn't finish, but I'll try to localize the problem and report it to Matplotlib (and update this issue here).
Nonetheless, maybe you have some workaround? For example, is it possible to modify the output directory structure before it is packed into the ZIP archive? Using some declarative include/exclude list or maybe some imperative hook?
stsewd commentedon Aug 22, 2022
@MikhailRyazanov
This is to keep backwards compatibility with old projects that depended on RTD pinning sphinx to a specific version, new projects get the latest version by default, see https://docs.readthedocs.io/en/stable/build-default-versions.html#default-versions-of-dependencies.
We don't support that, but we may add support for this with a process similar to https://docs.readthedocs.io/en/stable/build-customization.html#override-the-build-process I think
MikhailRyazanov commentedon Aug 22, 2022
Is there any option to switch on “old” project to “new” (and back, if needed)?
OK, looks interesting. I'm curious whether
post_buildmentioned on top of that page is executed before packing the Sphinx output into ZIP or after? Because in the former case it indeed can be easily customized to remove unnecessary files. (On the other hand, even if it runs after, but the location of the resulting ZIP file is known and writable, the archive itself can be cleaned-up, I hope...)stsewd commentedon Aug 22, 2022
It's possible, this isn't exposed to users, but it can be done via a support request.
It's executed after.
MikhailRyazanov commentedon Aug 23, 2022
Thank!
OK. Is then the ZIP file accessible to the
post_buildcommands, such that we can dozip -d ...on it?stsewd commentedon Aug 24, 2022
It probably is, but this isn't something we will guarantee for the file to always be there, I think the file is generated in the
../../artifacts/directory (from the root of your project)MikhailRyazanov commentedon Aug 24, 2022
Yes, I was afraid that this might be fragile. On the other hand, it's not a big problem if
zip -dfails, so this might be worth trying as a temporary solution.Let me investigate the details with Matplotlib and see what can be done...
humitos commentedon Aug 25, 2022
@MikhailRyazanov
It may not be exactly your case, but when working with
post_buildyou may hit this other issue #9179humitos commentedon Jan 31, 2023
Hello 👋🏼 We standardized the output paths for all the formats that Read the Docs supports in #9888. This allow people to output, for example, a HTMLZip under
_readthedocs/htmlzip/<project-slug>.zipto be recognized by Read the Docs and expose it as a downloadable content (docs.readthedocs.io/en/stable/downloadable-documentation.html) 💯This is the exact directory that Read the Docs will read immediately before uploading this
.zipfile and host it. So, you can usebuild.jobs.post_buildto modify the_readthedocs/htmlzip/<project-slug>.zipfile as you want after Read the Docs built it but before uploading it.At
build.jobs.post_buildyou will be able to remove all the unnecessary files you don't want to be included in the final.zipfile. Let me know if that works for your use case.I'm closing the issue, but feel free to comment here or re-open if you consider it's not solved yet.