Repository navigation
fix(docs): serve current documentation at stable /docs/ URLs - #772
Conversation
✅ Deploy Preview for cozystack ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configuration
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
5dd7d6c to
6334898
Compare
myasnikovdaniil
changed the title
Publish current docs at /docs/ with canonical URLs that survive minor releases. Keep numbered docs and the major alias available, and publish the version chooser at /docs/versions/. Share canonical and robots rules between page heads and sitemaps. Keep archived versions readable and noindex. Mirror over matching legacy aliases while preserving other aliases and removing stale copied files on repeat builds. Cover release rollover, translated landings, legacy collisions, query strings and fragments, previews, print output and static HTTP responses. Assisted-By: LLM Signed-off-by: Myasnikov Daniil <[email protected]>
6334898 to
bba1202
Compare
myasnikovdaniil
changed the title
IvanHunters
left a comment
IvanHunters
left a comment
There was a problem hiding this comment.
Verdict: LGTM — no blocking issues.
Verified routing on the live Netlify preview (/docs/, /docs/versions/, /docs/v1/, /docs/v1.6/ all serve the right page), ran rewrite() and the mirror end-to-end on hand-built trees, and diffed the SEO templates against the old logic and Docsy v0.12.0. Solid work: the routing is correct in a real build, cleanup is guarded against deleting archives, and the SEO policy is preserved (old versions noindex, latest and translated trees self- or root-canonical, print/preview noindex, sitemap lists only canonicals).
All findings below are non-blocking.
Note: the only red check is FOSSA License Compliance, which is unrelated to this PR (no dependency manifests changed).
| # Keep the original formatting (Hugo's minified attributes may be unquoted). | ||
| text = re.sub(r'''(["'(=])''' + re.escape(source_path), | ||
| lambda m: m[1] + destination_path, text) | ||
| text = re.sub(r'''(<a\b[^>]*\bhref=["']?)''' + re.escape(source_url), |
There was a problem hiding this comment.
[MINOR] rewrite() misses absolute non- URLs on mirrored copies
On the mirrored /docs/ and /docs/v<major>/ copies, og:url and <link rel="alternate"> (RSS, print) still point at /docs/v1.6/..., while the canonical is correctly rewritten to /docs/. Rule 1 (line 35) only fires after " ' ( =, and this <a href> rule (line 37) only matches anchors, so absolute URLs in <meta property=og:url> and <link rel=alternate href> are left on the numbered path. Confirmed on the live Netlify preview: /docs/ carries og:url=.../docs/v1.6/ and rel=alternate ... /docs/v1.6/index.xml. Affects social-card URLs and feed autodiscovery only; indexing is unaffected since the canonical is correct.
| @@ -0,0 +1,107 @@ | |||
| #!/usr/bin/env python3 | |||
There was a problem hiding this comment.
[MINOR] New SEO/routing tests run in no CI job
grep -rn test_docs_seo .github Makefile is empty. Only i18n-lint.yml runs on pull requests; the deploy workflow (hugo.yaml) runs on push to the default branch and never invokes these tests. test_docs_seo.py is the only check against a real built tree and it is never run anywhere, so the canonical/robots/sitemap invariants this PR establishes can regress silently later. A pull_request job running the fixtures (with the pinned Hugo) plus the built-tree check would close the gap.
| if name not in current and target.is_file() and sha256(target.read_bytes()).hexdigest() == digest: | ||
| target.unlink() | ||
| manifest.write_text(json.dumps(current, indent=2) + "\n") | ||
| if major != latest: |
There was a problem hiding this comment.
[MINOR] /docs/v/ is untracked and rebuilt with a blunt rmtree
The /docs/v<major>/ copy is not recorded in .docs-mirror.json; it is wiped with shutil.rmtree and recreated with copytree on every run. On a reused output directory after a major rollover (v1 to v2) a stale, indexable /docs/v1/ copy is left behind with canonicals pointing at the new /docs/, and the rmtree also removes any Hugo alias stub that legitimately lands under /docs/v<major>/. No production impact because CI always builds into a clean directory; this only bites a reused local output dir.
|
|
||
|
|
||
| def mirror(public): | ||
| config = Path("hugo.yaml").read_text() |
There was a problem hiding this comment.
[MINOR] latest_version_id is parsed from root hugo.yaml, not Hugo's merged config
The regex requires exactly two-space indentation and a double-quoted value, and the read is relative to cwd (must be the repo root). A config/production/hugo.yaml already exists and Hugo merges it; if latest_version_id ever moves or is overridden there, this mirror (reading the root file) and docs-seo.html (reading Hugo's value) silently diverge: one version is copied while canonicals and the sitemap point at another. Latent today since only hugo.yaml sets it. Reading it via hugo config would remove the coupling.
| {{- end }} | ||
| {{- end }} | ||
| <li><hr class="dropdown-divider"></li> | ||
| <li><a href="{{ .Site.Home.RelPermalink }}docs/versions/" class="dropdown-item">{{ T "docs_landing_available_versions" }}</a></li> |
There was a problem hiding this comment.
[MINOR] /docs/versions/ link 404s under make serve
The mirror step does not run in make serve (it is hugo serve only), so /docs/versions/ does not exist in dev and this new switcher link 404s, while /docs/ shows the version chooser rather than the latest docs. The dev/prod divergence is documented in CLAUDE.md, but a one-line comment near this link would save a future reader the surprise.
| if destination.exists(): | ||
| shutil.rmtree(destination) | ||
| shutil.copytree(source, destination) | ||
| source_url = PageURL((source / "index.html").read_text()).url |
There was a problem hiding this comment.
[NIT] Non-atomic / locale-dependent writes in the v copy path
A mid-run failure leaves a half-mirrored public/ and no manifest: landings and root copies are written before manifest.write_text, and this v path uses locale-dependent read_text/write_text (lines 137-140) where the root-copy path uses explicit bytes with .decode()/.encode(). A non-UTF-8 input or a non-UTF-8 locale could raise here. Low impact: Hugo output is UTF-8 and CI fails the step and deploys nothing, but consider failing before any write, or writing to a temp dir and swapping.
Use stable Open Graph URLs and rewrite absolute feed, print and asset attributes in generated copies. Mark the version-selection landing explicitly so corrected og:url metadata cannot replace it on repeat runs. Track major copies in the same file/hash manifest as root copies. Remove only owned, unchanged output on rollover and preserve Hugo alias stubs. Use the existing version chooser paths when running Hugo server. Add regressions for metadata, development navigation, major rollover, alias preservation and invalid major destinations. Assisted-By: LLM Signed-off-by: Myasnikov Daniil <[email protected]>
IvanHunters
left a comment
IvanHunters
left a comment
There was a problem hiding this comment.
Reviewed this architecturally and ran the fixture suite against the pinned Hugo 0.164.0 (extracted that exact binary so the templates are evaluated on the version CI uses), plus a few targeted probes on real build output. 16/16 fixtures pass.
The approach holds up. Stable /docs/ canonical URLs are the right call, the copy-after-build step is well guarded (symlink/traversal/reserved-path checks, manifest-tracked cleanup that only removes unchanged tracked files), and folding the old Netlify 301 and the layouts/404.html JS redirect into one real-files mirror that behaves identically on Pages and Netlify is a real simplification rather than just another layer.
Two things I specifically checked that turned out fine, noting them so they don't get re-litigated later:
- Structured data is not a rewrite gap. Docsy's
schema.htmlonly emits the site-levelOrganizationblock, andog:image/twitter:imageresolve to the site social image, so on the current theme output no versioned/docs/<latest>/URL escapes the rewrite allowlist. The one caveat is that this is an implicit allowlist: if a future Docsy bump starts emitting page-level absolute URLs in a new carrier, the mirror would silently miss it, and the fixtures stubschema.html/twitter_cards.htmlempty so they could not catch it. - The new
layouts/sitemap.xmltakeshreflangfrom.Language.Localewhile the head uses.Language.LanguageCode. On the prod config (nolocaleset,languageCode: zh-Hansfor thezh-cnkey) both resolve tozh-Hans, so they stay consistent. Worth remembering if a language ever setslocaleexplicitly, since the two would then diverge.
One non-blocking note:
The mirrored RSS feeds keep the versioned URL inside their body. The rewrite in hack/mirror_latest.py covers href/src/canonical/og:url and relative paths, but an absolute URL that sits as element text (<link>, <guid>, item links) matches none of the rules, so only <atom:link rel="self"> gets corrected. Concretely, public/docs/install/index.xml ends up with <link>https://cozystack.io/docs/v1.6/install/</link> and <guid>.../docs/v1.6/...</guid> instead of the stable path. Two small effects: the stable feed advertises the archive page as its own link, and since the guid is the versioned permalink it changes on every minor release, so anyone subscribed to the stable feed gets every item re-flagged as new on each rollover. Harmless to canonical/SEO (the archive exists and serves the same bytes), and the fixtures do not assert feed body content so it went unnoticed. If you want the copies fully self-consistent, rewriting the absolute source_url prefix globally for .xml files (not only in attributes) and stabilizing the guid would close it.
Approving. The feed note is a follow-up, not a merge blocker.
This restores six legacy URLs as static HTML pages. Follow-up to [PR 772](#772), part of [the website epic](#773). | Old URL | Content | | --- | --- | | `/docs/reference/applications/` | `/docs/applications/` | | `/docs/reference/applications/mysql/` | `/docs/applications/mariadb/` | | `/docs/install/talos/kexec/` | `/docs/install/talos/boot-to-talos/` | | `/roadmap/` | `/docs/roadmap/` | | `/docs/bundles/distro-hosted/` | `/docs/v0/operations/configuration/bundles/` | | `/docs/operations/storage/linstor/` | `/docs/storage/` | The current-doc mappings also serve their four `/docs/v1/` forms. All copies return HTTP 200 without JavaScript, query strings and existing fragments stay on the requested URL. Current content uses the target canonical. `distro-hosted` stays on archived, noindex content. LINSTOR uses the storage overview because its exact historical page could not be confirmed. Mappings are checked before output is replaced, including missing targets, symlinks and collisions with real pages. Compatibility copies use the existing ownership manifest and are cleaned on rollover. Checked with Hugo extended 0.164.0: production and preview builds, full-site canonical/robots/sitemap checks, 24 routing fixtures, 129 translation tests, version-pin tests and i18n lint. Anchor corrections remain in [PR 745](#745).
This serves current docs directly at
/docs/, so links and canonical URLs stay the same when a new minor version is released. Numbered versions stay available,/docs/v1/serves the current 1.x docs, and version selection moves to/docs/versions/.Hugo builds the numbered pages, then the same copy step runs in GitHub Pages and Netlify builds. The file/hash manifest tracks root and major copies, removes stale generated files on rollover and preserves Hugo aliases.
Open Graph URLs match the stable canonical, feed and print links follow the served copy. Archives remain noindex and are excluded from sitemaps. Preview and print output are noindex too.
The copy step must run after Hugo.
make servekeeps numbered pages and links to the existing version chooser at/docs/.Checked with Hugo extended 0.164.0: full production and preview builds, production canonical/robots/sitemap checks, 16 fixture tests (minor and major rollover, Hugo server navigation, metadata, root collisions, translated landings, query strings, fragments and static HTTP), plus the existing 129 translation tests, version-pin tests and i18n lint.
Part of the website epic, A1 and the related part of A3. Renamed legacy paths, README and
llms.txtlinks, language navigation and Algolia crawler settings remain follow-ups.