Build llms.txt files in Sphinx documentation #12192
Conversation
Unit Test ResultsSee test report for an extended history of previous test failures. This is useful for diagnosing flaky tests. 9 files ±0 9 suites ±0 3h 15m 40s ⏱️ + 2m 56s Results for commit a6e4365. ± Comparison against base commit 068be28. |
|
Nice, we have merged support for it in Xarray. Thanks for writing the plugin! |
|
@dcherian actually it looks like you're using the other plugin. There's a comparison table in the one I wrote. The most notable thing is that the other one doesn't actually build the docs, it just passes the source straight through. My plugin runs a full parallel sphinx build so that all extensions and directives are processed. This is super important for things like intersphinx, docrefs and autoapi. |
3 participants
Adds documentation support for llms.txt via the
sphinx-llmextension that I wrote.In addition to building HTML pages, a markdown version of the documentation is also built for consumption by LLMs. This reduces the amount of context window space LLMs need to use by removing HTML/CSS/JavaScript markup in favour of more streamlined markdown.
.mdto any documentation URL to view this)llms.txtfile file is generated which acts as a sitemap for LLMs to use to discover pagesllms-full.txtfile is generated with the entire docs contained in a single markdown file to allow loading all of the Dask docs into context with a single HTTP request (which for Dask would use 1.1M input tokens 🙀)Build preview highlights
dask.array.angle)