Skip to content

Commit 3351dd5

Browse files
fix(docs): preserve metadata and aliases in mirrored output
Use stable Open Graph URLs and rewrite absolute feed, print and asset attributes in generated copies. Mark the version-selection landing explicitly so corrected og:url metadata cannot replace it on repeat runs. Track major copies in the same file/hash manifest as root copies. Remove only owned, unchanged output on rollover and preserve Hugo alias stubs. Use the existing version chooser paths when running Hugo server. Add regressions for metadata, development navigation, major rollover, alias preservation and invalid major destinations. Assisted-By: LLM Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent bba1202 commit 3351dd5

8 files changed

Lines changed: 146 additions & 45 deletions

File tree

‎CLAUDE.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
44

55
## Project Overview
66

7-
Hugo documentation website for Cozystack (Kubernetes-based cloud platform) using the Google Docsy theme. **Production is served by GitHub Pages**. Netlify is only used for deploy previews on PRs. Do not rely on Netlify-specific redirects or headers for production behavior. The stable `/docs/` and `/docs/v<major>/` paths are real copies of the latest version, produced after `hugo` by `hack/mirror_latest.sh` (Python 3 standard library). Current numbered docs and both copies use canonical URLs under `/docs/`. Older numbered docs remain noindex and are excluded from sitemaps. Preview builds are noindex. The version-selection landing is copied to `/docs/versions/`; root mirroring preserves numbered versions and unrelated legacy aliases. The `.docs-mirror.json` file tracks root copies for cleanup on repeat builds.
7+
Hugo documentation website for Cozystack (Kubernetes-based cloud platform) using the Google Docsy theme. **Production is served by GitHub Pages**. Netlify is only used for deploy previews on PRs. Do not rely on Netlify-specific redirects or headers for production behavior. The stable `/docs/` and `/docs/v<major>/` paths are real copies of the latest version, produced after `hugo` by `hack/mirror_latest.sh` (Python 3 standard library). Current numbered docs and both copies use canonical URLs under `/docs/`. Older numbered docs remain noindex and are excluded from sitemaps. Preview builds are noindex. The version-selection landing is copied to `/docs/versions/`; mirroring preserves numbered versions and unrelated legacy aliases. The `.docs-mirror.json` file tracks root and major copies for cleanup on repeat builds. Under `hugo server`, version-selection links use the existing `/docs/` landing.
88

99
## Build & Development Commands
1010

‎CONTRIBUTING.md‎

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -259,7 +259,9 @@ If you need to understand or change the versioning machinery itself:
259259
docs to `/docs/` and `/docs/v<major>/`. It also copies translated docs
260260
when they exist. The original landing is published at `/docs/versions/`, with a
261261
link in the version dropdown. Pages and Netlify builds run it; `make serve` does
262-
not. Local `make serve` keeps the version-selection landing at `/docs/`.
262+
not. Local `make serve` keeps the version-selection landing and its navigation
263+
links at `/docs/`. The landing template carries `data-docs-version-landing` so
264+
repeat mirror runs can identify it independently of Open Graph metadata.
263265
- `layouts/partials/docs-seo.html` supplies canonical and robots rules to the
264266
document head and sitemap. Current numbered docs and their copies canonicalize
265267
to `/docs/`. Older versions remain noindex and stay out of sitemaps.
@@ -268,11 +270,13 @@ If you need to understand or change the versioning machinery itself:
268270
tag. `navbar-lang-selector.html` keeps the version-selection page at its new
269271
path when changing language. Compare both with upstream when updating Docsy.
270272

271-
The mirrors add two copies of the current docs to the published artifact. Root
272-
mirroring preserves numbered versions, `next/` and unrelated legacy aliases. The
273-
`.docs-mirror.json` build file tracks copied files and their hashes so repeat
274-
builds remove stale copies without deleting aliases restored by Hugo. Keep it
275-
when rebuilding in the same output directory.
273+
The mirrors add two copies of the current docs to the published artifact.
274+
Mirroring preserves numbered versions, `next/` and unrelated legacy aliases. The
275+
`.docs-mirror.json` build file tracks root and major copies and their hashes so
276+
minor and major rollover remove stale copies without deleting aliases restored
277+
by Hugo. Only tracked, unchanged files are removed. Keep the manifest when
278+
rebuilding in the same output directory. Mirrored Open Graph URLs match the
279+
stable canonical; absolute feed and print links follow the served copy.
276280
After a release, run Hugo and the mirror step together. Publishing just Hugo's
277281
output would leave the stable canonical and sitemap URLs without their pages.
278282

‎hack/mirror_latest.py‎

Lines changed: 35 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,6 @@
66
import json
77
from pathlib import Path
88
import re
9-
import shutil
109
import sys
1110
from urllib.parse import urlsplit
1211

@@ -15,14 +14,21 @@ class PageURL(HTMLParser):
1514
def __init__(self, html):
1615
super().__init__()
1716
self.url = None
17+
self.version_landing = False
1818
self.feed(html)
1919

2020
def handle_starttag(self, tag, attrs):
2121
attrs = dict(attrs)
22-
# Open Graph retains the original page URL, including on previews where
23-
# no canonical is emitted. This also supplies the baseURL subpath.
22+
# Numbered source pages retain og:url even when previews omit canonical.
2423
if tag == "meta" and attrs.get("property") == "og:url":
2524
self.url = attrs.get("content")
25+
if "data-docs-version-landing" in attrs:
26+
self.version_landing = True
27+
28+
29+
def rewrite_og_url(text, source_url, target_url):
30+
return re.sub(r'''(<meta\b(?=[^>]*\bproperty=["']?og:url(?:["'\s>]))[^>]*\bcontent=["']?)''' + re.escape(source_url),
31+
lambda m: m[1] + target_url, text)
2632

2733

2834
def rewrite(text, source_url, alias):
@@ -34,27 +40,29 @@ def rewrite(text, source_url, alias):
3440
# Keep the original formatting (Hugo's minified attributes may be unquoted).
3541
text = re.sub(r'''(["'(=])''' + re.escape(source_path),
3642
lambda m: m[1] + destination_path, text)
37-
text = re.sub(r'''(<a\b[^>]*\bhref=["']?)''' + re.escape(source_url),
43+
# Hugo alias stubs do not use the regular head template.
44+
text = re.sub(r'''(<link\b(?=[^>]*\brel=["']?canonical(?:["'\s>]))[^>]*\bhref=["']?)''' + re.escape(source_url),
45+
lambda m: m[1] + canonical_url, text)
46+
text = rewrite_og_url(text, source_url, canonical_url)
47+
text = re.sub(r'''(\b(?:href|src)=["']?)''' + re.escape(source_url),
3848
lambda m: m[1] + destination_url, text)
3949
text = re.sub(r'''(http-equiv="?refresh"? content="?[0-9]+;\s*url=)''' + re.escape(source_url),
4050
lambda m: m[1] + destination_url, text)
41-
# Hugo alias stubs do not use the regular head template.
42-
text = re.sub(r'''(<link\b[^>]*\brel="?canonical"?\s+href=["']?)''' + re.escape(source_url),
43-
lambda m: m[1] + canonical_url, text)
4451
return text
4552

4653

4754
def reserved(name):
4855
return re.fullmatch(r"v[0-9]+(?:\.[0-9]+)?|next|versions|latest", name)
4956

5057

51-
def root_target(public, name):
52-
"""Keep cleanup and root copies outside version trees, including symlinks."""
58+
def root_target(public, name, *, allow_major=False):
59+
"""Keep managed copies outside numbered archives, including symlinks."""
5360
path = Path(name)
5461
parts = path.parts
5562
offset = 1 if parts and parts[0] == "docs" else 2
5663
if (path.is_absolute() or ".." in parts or len(parts) <= offset
57-
or parts[offset - 1] != "docs" or reserved(parts[offset])):
64+
or parts[offset - 1] != "docs"
65+
or (reserved(parts[offset]) and not (allow_major and re.fullmatch(r"v[1-9][0-9]*", parts[offset])))):
5866
raise ValueError(f"invalid root mirror path: {name}")
5967
target = public / path
6068
if target.resolve() != target or target.is_dir():
@@ -78,7 +86,7 @@ def mirror(public):
7886
if not isinstance(previous, dict):
7987
raise ValueError(f"invalid root mirror manifest: {manifest}")
8088
for name, digest in previous.items():
81-
root_target(public, name)
89+
root_target(public, name, allow_major=True)
8290
if not isinstance(digest, str) or not re.fullmatch(r"[a-f0-9]{64}", digest):
8391
raise ValueError(f"invalid root mirror digest: {name}")
8492

@@ -89,9 +97,12 @@ def mirror(public):
8997
versions = landing.parent / "versions/index.html"
9098
if versions.resolve() != versions or versions.is_dir():
9199
raise ValueError(f"invalid versions landing path: {versions}")
92-
landing_url = PageURL(landing.read_text()).url
93-
if landing_url and urlsplit(landing_url).path.endswith("/docs/"):
94-
landings.append((versions, landing.read_bytes()))
100+
html = landing.read_text()
101+
page_url = PageURL(html)
102+
if page_url.version_landing:
103+
if not page_url.url:
104+
raise ValueError(f"missing og:url in {landing}")
105+
landings.append((versions, rewrite_og_url(html, page_url.url, page_url.url + "versions/").encode()))
95106
elif not versions.is_file():
96107
raise ValueError(f"missing version-selection landing: {landing}")
97108
for source in sources:
@@ -100,25 +111,27 @@ def mirror(public):
100111
source_url = PageURL((source / "index.html").read_text()).url
101112
if not source_url or not urlsplit(source_url).path.endswith(f"/docs/{latest}/"):
102113
raise ValueError(f"invalid og:url in {source / 'index.html'}")
114+
destination = source.parent / major
115+
if major != latest and (destination.resolve() != destination or destination.is_file()):
116+
raise ValueError(f"invalid major mirror destination: {destination}")
103117
for file in source.rglob("*"):
104118
if file.is_symlink() or reserved(file.relative_to(source).parts[0]):
105119
raise ValueError(f"reserved or symlink source path: {file}")
106120
if file.is_file():
107-
target = source.parent / file.relative_to(source)
108-
root_target(public, target.relative_to(public))
109-
plans.append((file, source_url, target))
110-
destination = source.parent / major
111-
if major != latest and (destination.resolve() != destination or destination.is_file()):
112-
raise ValueError(f"invalid major mirror destination: {destination}")
121+
aliases = [""] if major == latest else ["", major + "/"]
122+
for alias in aliases:
123+
target = source.parent / alias / file.relative_to(source)
124+
root_target(public, target.relative_to(public), allow_major=bool(alias))
125+
plans.append((file, source_url, target, alias))
113126

114127
for versions, content in landings:
115128
versions.parent.mkdir(parents=True, exist_ok=True)
116129
versions.write_bytes(content)
117130
current = {}
118-
for file, source_url, target in plans:
131+
for file, source_url, target, alias in plans:
119132
content = file.read_bytes()
120133
if file.suffix in (".html", ".xml"):
121-
content = rewrite(content.decode(), source_url, "").encode()
134+
content = rewrite(content.decode(), source_url, alias).encode()
122135
target.parent.mkdir(parents=True, exist_ok=True)
123136
target.write_bytes(content)
124137
current[str(target.relative_to(public))] = sha256(content).hexdigest()
@@ -128,16 +141,6 @@ def mirror(public):
128141
if name not in current and target.is_file() and sha256(target.read_bytes()).hexdigest() == digest:
129142
target.unlink()
130143
manifest.write_text(json.dumps(current, indent=2) + "\n")
131-
if major != latest:
132-
for source in sources:
133-
destination = source.parent / major
134-
if destination.exists():
135-
shutil.rmtree(destination)
136-
shutil.copytree(source, destination)
137-
source_url = PageURL((source / "index.html").read_text()).url
138-
for file in destination.rglob("*"):
139-
if file.suffix in (".html", ".xml"):
140-
file.write_text(rewrite(file.read_text(), source_url, major + "/"))
141144
print(f"mirrored {latest} to /docs/ and /docs/{major}/; version selection at /docs/versions/")
142145

143146

‎hack/test_docs_seo.py‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@ def __init__(self, text):
2222
self.refresh = []
2323
self.links = []
2424
self.languages = {}
25+
self.og_urls = []
26+
self.formats = {}
2527
self.feed(text)
2628

2729
def handle_starttag(self, tag, attrs):
@@ -36,6 +38,10 @@ def handle_starttag(self, tag, attrs):
3638
self.links.append(attrs["href"])
3739
if tag == "link" and attrs.get("hreflang"):
3840
self.languages[attrs["hreflang"]] = attrs.get("href")
41+
if tag == "meta" and attrs.get("property") == "og:url":
42+
self.og_urls.append(attrs.get("content"))
43+
if tag == "link" and attrs.get("rel") == "alternate" and attrs.get("type"):
44+
self.formats[attrs["type"]] = attrs.get("href")
3945

4046

4147
class BuiltSiteTests(unittest.TestCase):
@@ -60,6 +66,10 @@ def test_current_docs_canonical(self):
6066
self.assertEqual(urlsplit(page.canonical[0]).path, "/docs/" + str(relative).removesuffix("index.html"))
6167
self.assertEqual(page.canonical, original.canonical)
6268
self.assertEqual(page.robots, ["index, follow"])
69+
if version != self.latest:
70+
self.assertEqual(page.og_urls, page.canonical)
71+
for url in page.formats.values():
72+
self.assertNotIn(f"/docs/{self.latest}/", urlsplit(url).path)
6373
self.assertGreater(count, 0)
6474

6575
def test_archive_robots(self):

0 commit comments

Comments
 (0)