Skip to content

Commit 5d9b7fd

Browse files
yiaanyalambmartin-g
authored
docs: add version picker and manual release snapshots (#24792)
## Which issue does this PR close? - Closes #17071. - Depends on #25881 - Depends on #25880 ## Rationale for this change The documentation site currently follows `main`, so users cannot easily read the documentation that was published with a released DataFusion version. This change adds a version picker while keeping the site root on the current development documentation. ## What changes are included in this PR? - Adds version switcher. ## Are these changes tested? Tested manually ## Are there any user-facing changes? Yes. The site will expose a version switcher and release URLs under `/versions/<version>/`, while the root URL continues to serve the latest `main` documentation. <img width="1452" height="618" alt="Screenshot 2026-10-08 at 4 44 35 PM" src="https://github.com/user-attachments/assets/c0d355c8-f9b8-4af5-9cc7-633970fb96bc" /> <img width="1424" height="762" alt="Screenshot 2026-10-08 at 4 44 53 PM" src="https://github.com/user-attachments/assets/2ba0ccca-9a9c-4bcf-8b38-ea3765eae636" /> --------- Co-authored-by: Andrew Lamb <[email protected]> Co-authored-by: Martin Grigorov <[email protected]>
1 parent dc1878b commit 5d9b7fd

5 files changed

Lines changed: 179 additions & 7 deletions

File tree

‎dev/release/README.md‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -313,6 +313,74 @@ git tag 55.2.0
313313
git push apache 55.2.0
314314
```
315315

316+
#### Publish the versioned documentation
317+
318+
**1. Build the documentation from the release tag** using the configuration
319+
overlay (`docs/scripts/release/conf.py`) from the main checkout. For example,
320+
for `55.0.0`, from the repository root:
321+
322+
```shell
323+
export DATAFUSION_DOCS_OVERLAY="$PWD/docs/scripts/release"
324+
git fetch apache tag 55.0.0
325+
git worktree add --detach /tmp/datafusion-55.0.0 55.0.0
326+
cd /tmp/datafusion-55.0.0/docs
327+
DATAFUSION_DOCS_SOURCE="$PWD/source" DATAFUSION_DOCS_VERSION=55.0.0 \
328+
SPHINXOPTS="-W -c $DATAFUSION_DOCS_OVERLAY" \
329+
uv run --package datafusion-docs --with sphinx-sitemap ./build.sh
330+
cd -
331+
```
332+
333+
Notes:
334+
335+
1. You need to use an absolute path to the overlay in your current checkout.
336+
2. The `--with sphinx-sitemap` option supplies the sitemap extension for older tags.
337+
338+
To preview the result, build the docs from your main checkout
339+
(`uv run --package datafusion-docs ./build.sh` in `docs/`), copy the release
340+
build in under `versions/<version>/`, and serve the tree over HTTP (the version
341+
picker does not work when the pages are opened directly as files):
342+
343+
```shell
344+
mkdir -p docs/build/html/versions/55.0.0
345+
rsync -a /tmp/datafusion-55.0.0/docs/build/html/ docs/build/html/versions/55.0.0/
346+
python3 -m http.server --directory docs/build/html 8000
347+
```
348+
349+
Then open http://localhost:8000/ and http://localhost:8000/versions/55.0.0/
350+
and check the version picker in the navigation bar.
351+
352+
**2. Add the built site to the `asf-site` branch.** Open a PR against
353+
`asf-site` that adds the release under `versions/<version>/` (for example
354+
[#25881](https://github.com/apache/datafusion/pull/25881)):
355+
356+
```shell
357+
git fetch apache asf-site
358+
git worktree add -b docs/publish-55.0.0 /tmp/datafusion-asf-site apache/asf-site
359+
rsync -a --exclude '/.buildinfo' \
360+
/tmp/datafusion-55.0.0/docs/build/html/ \
361+
/tmp/datafusion-asf-site/versions/55.0.0/
362+
cd /tmp/datafusion-asf-site
363+
git add versions/55.0.0
364+
git commit -m "Publish documentation for 55.0.0"
365+
git push origin docs/publish-55.0.0
366+
```
367+
368+
The release is live at `https://datafusion.apache.org/versions/55.0.0/` once
369+
the PR is merged.
370+
371+
**3. Add the release to the version picker.** Open a PR against `main` that
372+
adds the release to `docs/source/_static/versions.json`. Put the new release
373+
directly after the `Development (main)` entry and move `"preferred": true` to it, so that
374+
it is treated as the current stable release. Only add releases that are already
375+
published, since the picker links to them directly.
376+
377+
Finally, remove the temporary worktrees:
378+
379+
```shell
380+
git worktree remove /tmp/datafusion-55.0.0
381+
git worktree remove /tmp/datafusion-asf-site
382+
```
383+
316384
### 10. Publish on Crates.io
317385

318386
Only approved releases of the tarball should be published to

‎docs/README.md‎

Lines changed: 15 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,9 @@
2020
# DataFusion Documentation
2121

2222
This folder contains the source content of the [User Guide](./source/user-guide)
23-
and [Contributor Guide](./source/contributor-guide). These are both published to
24-
https://datafusion.apache.org/ as part of the release process.
23+
and [Contributor Guide](./source/contributor-guide). The site root shows the
24+
development documentation built from `main`. Released versions of the complete
25+
site are available under `/versions/<version>/`.
2526

2627
## Dependencies
2728

@@ -48,9 +49,7 @@ Run the provided script to build the HTML pages.
4849
```
4950

5051
The HTML will be generated into a `build` directory. Open `build/html/index.html`
51-
in your preferred browser, e.g.
52-
53-
Preview the site on Linux by running this command.
52+
in your preferred browser, for example on Linux via
5453

5554
```bash
5655
# On macOS
@@ -59,6 +58,16 @@ open build/html/index.html
5958
firefox build/html/index.html
6059
```
6160

61+
Note that some features of the site, such as the version picker, do not work
62+
when the pages are opened directly as files. To preview such features, serve
63+
the site over HTTP with sphinx-autobuild:
64+
65+
```bash
66+
uv run --with sphinx-autobuild sphinx-autobuild source build/html --port 8000
67+
```
68+
69+
And then open http://localhost:8000/ in your browser
70+
6271
## Making Changes
6372

6473
To make changes to the docs, simply make a Pull Request with your
@@ -69,7 +78,7 @@ automatically updated.
6978

7079
This documentation is hosted at https://datafusion.apache.org/
7180

72-
When the PR is merged to the `main` branch of the DataFusion
81+
When a PR is merged to the `main` branch of the DataFusion
7382
repository, a [github workflow](https://github.com/apache/datafusion/blob/main/.github/workflows/docs.yaml) which:
7483

7584
1. Builds the html content

‎docs/scripts/release/conf.py‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
"""Sphinx configuration overlay for building docs from a release tag.
19+
20+
Loads ``conf.py`` from a``docs/source`` directory located at
21+
``DATAFUSION_DOCS_SOURCE`` and overrides the following settings:
22+
- the version string (``DATAFUSION_DOCS_VERSION``),
23+
- the site base URL used for the sitemap
24+
- the version switcher.
25+
26+
Why?
27+
28+
Documentation from older release tags do not have the version picker code and
29+
expect their base URL to be ``/`` rather than ```versions/<version>/``, so we can
30+
not publish docs built for the archived version without modification.
31+
32+
Rather than modifying the documentation in the old release branch,
33+
the release process builds docs with the following command from a main
34+
checkout:
35+
36+
sphinx-build -c docs/scripts/release
37+
"""
38+
39+
import os
40+
from pathlib import Path
41+
import sys
42+
43+
source = Path(os.environ["DATAFUSION_DOCS_SOURCE"]).resolve()
44+
sys.path.insert(0, str(source.parent))
45+
exec(compile((source / "conf.py").read_bytes(), str(source / "conf.py"), "exec"))
46+
47+
# Resolve paths relative to the tag, not this configuration overlay.
48+
templates_path = [str(source / path) for path in templates_path]
49+
html_static_path = [str(source / path) for path in html_static_path]
50+
html_extra_path = [str(source / path) for path in html_extra_path]
51+
html_logo = str(source / html_logo)
52+
html_favicon = str(source / html_favicon)
53+
54+
version = release = os.environ["DATAFUSION_DOCS_VERSION"]
55+
html_baseurl = f"https://datafusion.apache.org/versions/{version}/"
56+
sitemap_url_scheme = "{link}"
57+
if "sphinx_sitemap" not in extensions:
58+
extensions.append("sphinx_sitemap")
59+
html_context = {**html_context, "github_repo": "datafusion", "github_version": version}
60+
html_theme_options = {
61+
**html_theme_options,
62+
"navbar_end": ["version-switcher", "theme-switcher"],
63+
"check_switcher": False,
64+
"switcher": {
65+
"json_url": "/_static/versions.json",
66+
"version_match": version,
67+
},
68+
}
69+
# The old tag's absolute redirect would leave the release documentation.
70+
redirects = {**redirects, "library-user-guide/upgrading": "upgrading/index.html"}

‎docs/source/_static/versions.json‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
[
2+
{
3+
"name": "Development (main)",
4+
"url": "/",
5+
"version": "main"
6+
},
7+
{
8+
"name": "55.1.0",
9+
"url": "/versions/55.1.0/",
10+
"version": "55.1.0",
11+
"preferred": true
12+
},
13+
{
14+
"name": "55.0.0",
15+
"url": "/versions/55.0.0/",
16+
"version": "55.0.0"
17+
}
18+
]

‎docs/source/conf.py‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@
3737
project = "Apache DataFusion"
3838
copyright = "2019-2025, Apache Software Foundation"
3939
author = "Apache Software Foundation"
40+
version = release = "main"
4041

4142

4243
# -- General configuration ---------------------------------------------------
@@ -95,7 +96,13 @@
9596
},
9697
"use_edit_page_button": True,
9798
"navbar_center": [],
98-
"navbar_end": ["theme-switcher"],
99+
"navbar_end": ["version-switcher", "theme-switcher"],
100+
# Release docs are published separately; the manifest may not be live yet.
101+
"check_switcher": False,
102+
"switcher": {
103+
"json_url": "/_static/versions.json",
104+
"version_match": version,
105+
},
99106
}
100107

101108
html_context = {

0 commit comments

Comments
 (0)