Skip to content

docs: preserve English heading anchors in translated pages - #4580

Merged
seratch merged 3 commits into
openai:mainfrom
saime428:docs-translation-heading-anchors
Aug 22, 2026
Merged

seratch merged 3 commits into
openai:mainfrom
saime428:docs-translation-heading-anchors

Conversation

@saime428

@saime428 saime428 commented Aug 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Translating a heading changes the id mkdocs derives from it. The links inside the translated pages still point at the English ids, so once translate_docs.py has run, every #... link on those pages stops resolving. A build of current main reports 273 of them across docs/ja, docs/ko and docs/zh, and none in the English pages. I could not find an issue for it.

attr_list is already on, so a heading can carry its id explicitly. This change makes the translation script do that.

After a page is translated, each heading gets {#<id>}, where <id> is the id the site renders for the English heading at the same position. The ids come from the parser itself: the script builds a Markdown instance from the markdown_extensions in mkdocs.yml (plus the toc, tables and fenced_code builtins mkdocs adds) and reads them off the toc tokens, so reference links, entities, inline HTML and repeated headings (example, then example_1) all come out the way mkdocs produces them. The H1 is left alone, since mkdocs takes the page title from it.

When a page is skipped as up to date, the same step runs over its existing translations, so the next make translate repairs the current pages without sending anything back through the model.

The contract is the shape the docs already use: one ATX heading in the translation per English heading, carrying no attribute list other than the {#id} written here. Translated headings are located with the parser's own hash-heading grammar and any existing attribute list is recognised with the attr_list extension's own grammar. A page outside that shape is reported by name and left as it is. Before a rewritten page is returned it is parsed again with the same extensions and has to render exactly the English ids, so a page that gets written is a page the site will link correctly. All 114 generated pages on main are inside the contract today.

The regenerated pages are not in this PR. AGENTS.md treats them as generated output, so producing them stays with make translate.

Why

These links are easy to miss. make build-docs does not run in strict mode, and the warnings sit in a log that already carries several thousand lines of autorefs and griffe output. A reader on the Japanese, Korean or Chinese site who clicks "see the dynamic instructions section" lands nowhere.

I also checked whether any of the 273 pointed at an H1. Once each warning is resolved to its target page, and same-named H2s on other pages are ruled out, none do. Skipping the H1 costs nothing.

Validation

  • uv run pytest tests/docs/test_translate_docs.py: 4 new tests, failing on main with AttributeError, passing here
  • ruff check and ruff format --check on both files; pyright tests/docs reports 0 errors
  • Ran the script on the up-to-date path over all 37 English sources with a placeholder API key (that path makes no model calls): 37 skipped, 102 translated pages refreshed
  • git diff of the refreshed pages: 1365 changed lines, every one of them an H2 to H6 heading; no H1 and no non-heading line touched; line endings preserved byte for byte
  • uv run mkdocs build before and after: anchor warnings 273 to 0, total warnings 9783 to 9510 (the difference is exactly 273), no new warning kinds
  • Rendered output: site/ja/agents/index.html has <h2 id="dynamic-instructions">, site/ko/running_agents/index.html has <h3 id="choose-a-memory-strategy">, and the page <title> elements are unchanged
  • The refreshed pages were then discarded; only the script and the test are committed

Not run: a translation through the model. The hook in translate_file sits after the existing placeholder checks and before the search-exclusion front matter goes on, so it sees the same text that gets written to disk.

Notes for reviewers

The part I am least sure about is refreshing existing translations on the "skipped, up to date" path. A skip can now write files, which it never did before. The alternative is a separate flag, but then the 273 stay broken until someone remembers to run it, and the refresh is cheap and idempotent. If you would rather keep the skip path read-only, moving this behind a flag is a small change.

Two things I noticed and did not touch. docs/ja/sessions.md, docs/ko/sessions.md and docs/zh/sessions.md no longer have an English source; docs/sessions.md is gone from main, so those three are stale generated pages the script will never revisit. And on Windows, "ref/" in relative_path in translate_single_source_file does not match the backslash form that os.path.relpath returns, so the ref/ pages get picked up for translation there. Neither is related to this change, and the second only matters on Windows, which is presumably not where the translations are produced.

Translating a heading also changes the id mkdocs derives from it, so every
`#...` link written against the English page stops resolving once the page
is translated. A build of main reports 273 such anchors across docs/ja,
docs/ko and docs/zh, and none in the English pages.

attr_list is already enabled, so a heading can carry its id explicitly.
After a translation completes, give each translated heading the id of the
English heading at the same position, with repeated headings numbered the
same way the toc extension numbers them. Headings inside fenced code are
ignored and the H1 is left alone, since mkdocs reads the page title from it.
When a page is skipped as up to date, apply the same step to its existing
translations, so the next `make translate` repairs the current pages
without sending them back through the model.

The regenerated pages are not part of this change; `make translate`
produces them.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5bc16a5def

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread docs/scripts/translate_docs.py Resolved
Slugifying a cleaned-up copy of the raw heading text only matches mkdocs for
the heading shapes the clean-up happens to cover. A reference-style link, an
HTML entity or inline HTML in an English heading renders to different text,
so the id written into the translation would not be the one the site uses.

Build the parser the way mkdocs does, from the extensions in mkdocs.yml, and
read the ids off the toc tokens instead. The translated side is still
matched by position.
@saime428

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e75958264c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread docs/scripts/translate_docs.py Resolved
Comment thread docs/scripts/translate_docs.py Resolved
@chatgpt-codex-connector

Copy link
Copy Markdown

Security review completed. No security issues were found in this pull request.

Reviewed commit: e75958264c

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

Locating translated headings and editing their attribute lists with
regexes written here meant two more grammars that had to agree with the
Markdown parser by hand. Reuse the parser's own hash-heading and attr_list
grammars instead, and state the contract plainly: one column-zero ATX
heading per English heading, carrying no attribute list other than the
`{#id}` written here. A page outside that shape is reported by name and
left untouched rather than edited on a guess.

Before a rewritten page is returned, parse it again with the configured
extensions and require it to render exactly the English ids, so any page
that is written is one the site will link correctly.
@seratch seratch added the documentation Improvements or additions to documentation label Aug 22, 2026
@seratch seratch added this to the 0.22.x milestone Aug 22, 2026
@seratch
seratch enabled auto-merge (squash) August 22, 2026 04:47
@seratch
seratch merged commit 60c2c41 into openai:main Aug 22, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants