Repository navigation
docs: preserve English heading anchors in translated pages - #4580
Conversation
Translating a heading also changes the id mkdocs derives from it, so every `#...` link written against the English page stops resolving once the page is translated. A build of main reports 273 such anchors across docs/ja, docs/ko and docs/zh, and none in the English pages. attr_list is already enabled, so a heading can carry its id explicitly. After a translation completes, give each translated heading the id of the English heading at the same position, with repeated headings numbered the same way the toc extension numbers them. Headings inside fenced code are ignored and the H1 is left alone, since mkdocs reads the page title from it. When a page is skipped as up to date, apply the same step to its existing translations, so the next `make translate` repairs the current pages without sending them back through the model. The regenerated pages are not part of this change; `make translate` produces them.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5bc16a5def
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Slugifying a cleaned-up copy of the raw heading text only matches mkdocs for the heading shapes the clean-up happens to cover. A reference-style link, an HTML entity or inline HTML in an English heading renders to different text, so the id written into the translation would not be the one the site uses. Build the parser the way mkdocs does, from the extensions in mkdocs.yml, and read the ids off the toc tokens instead. The translated side is still matched by position.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e75958264c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
Security review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
Locating translated headings and editing their attribute lists with
regexes written here meant two more grammars that had to agree with the
Markdown parser by hand. Reuse the parser's own hash-heading and attr_list
grammars instead, and state the contract plainly: one column-zero ATX
heading per English heading, carrying no attribute list other than the
`{#id}` written here. A page outside that shape is reported by name and
left untouched rather than edited on a guess.
Before a rewritten page is returned, parse it again with the configured
extensions and require it to render exactly the English ids, so any page
that is written is one the site will link correctly.
Summary
Translating a heading changes the id mkdocs derives from it. The links inside the translated pages still point at the English ids, so once
translate_docs.pyhas run, every#...link on those pages stops resolving. A build of currentmainreports 273 of them acrossdocs/ja,docs/koanddocs/zh, and none in the English pages. I could not find an issue for it.attr_listis already on, so a heading can carry its id explicitly. This change makes the translation script do that.After a page is translated, each heading gets
{#<id>}, where<id>is the id the site renders for the English heading at the same position. The ids come from the parser itself: the script builds aMarkdowninstance from themarkdown_extensionsinmkdocs.yml(plus thetoc,tablesandfenced_codebuiltins mkdocs adds) and reads them off the toc tokens, so reference links, entities, inline HTML and repeated headings (example, thenexample_1) all come out the way mkdocs produces them. The H1 is left alone, since mkdocs takes the page title from it.When a page is skipped as up to date, the same step runs over its existing translations, so the next
make translaterepairs the current pages without sending anything back through the model.The contract is the shape the docs already use: one ATX heading in the translation per English heading, carrying no attribute list other than the
{#id}written here. Translated headings are located with the parser's own hash-heading grammar and any existing attribute list is recognised with theattr_listextension's own grammar. A page outside that shape is reported by name and left as it is. Before a rewritten page is returned it is parsed again with the same extensions and has to render exactly the English ids, so a page that gets written is a page the site will link correctly. All 114 generated pages onmainare inside the contract today.The regenerated pages are not in this PR.
AGENTS.mdtreats them as generated output, so producing them stays withmake translate.Why
These links are easy to miss.
make build-docsdoes not run in strict mode, and the warnings sit in a log that already carries several thousand lines of autorefs and griffe output. A reader on the Japanese, Korean or Chinese site who clicks "see the dynamic instructions section" lands nowhere.I also checked whether any of the 273 pointed at an H1. Once each warning is resolved to its target page, and same-named H2s on other pages are ruled out, none do. Skipping the H1 costs nothing.
Validation
uv run pytest tests/docs/test_translate_docs.py: 4 new tests, failing onmainwithAttributeError, passing hereruff checkandruff format --checkon both files;pyright tests/docsreports 0 errorsgit diffof the refreshed pages: 1365 changed lines, every one of them an H2 to H6 heading; no H1 and no non-heading line touched; line endings preserved byte for byteuv run mkdocs buildbefore and after: anchor warnings 273 to 0, total warnings 9783 to 9510 (the difference is exactly 273), no new warning kindssite/ja/agents/index.htmlhas<h2 id="dynamic-instructions">,site/ko/running_agents/index.htmlhas<h3 id="choose-a-memory-strategy">, and the page<title>elements are unchangedNot run: a translation through the model. The hook in
translate_filesits after the existing placeholder checks and before the search-exclusion front matter goes on, so it sees the same text that gets written to disk.Notes for reviewers
The part I am least sure about is refreshing existing translations on the "skipped, up to date" path. A skip can now write files, which it never did before. The alternative is a separate flag, but then the 273 stay broken until someone remembers to run it, and the refresh is cheap and idempotent. If you would rather keep the skip path read-only, moving this behind a flag is a small change.
Two things I noticed and did not touch.
docs/ja/sessions.md,docs/ko/sessions.mdanddocs/zh/sessions.mdno longer have an English source;docs/sessions.mdis gone frommain, so those three are stale generated pages the script will never revisit. And on Windows,"ref/" in relative_pathintranslate_single_source_filedoes not match the backslash form thatos.path.relpathreturns, so theref/pages get picked up for translation there. Neither is related to this change, and the second only matters on Windows, which is presumably not where the translations are produced.