docs-i18n: chunk raw doc translation#62969
Conversation
openclaw-barnacle
Bot
added
scripts
|
Added one follow-up fix after the larger 15-page sample run exposed a remaining blocker in What changed:
Validation:
|
fdc0b0c to
cd8e860
Compare
|
Follow-up: added a narrow runtime retry for terminated Pi sessions in What changed:
Validation:
Latest commit on this branch: |
|
@greptile-apps review |
|
@codex review |
Greptile SummaryThis PR replaces the single-shot whole-document raw translation path with a chunked approach: the body is split into paragraph-level blocks, grouped by byte budget, translated chunk-by-chunk with per-chunk structural validation (code fence counts, component tag counts, protocol token leakage), and recursively bisected on failure. A leaf-block masked fallback ( Confidence Score: 5/5Safe to merge — all remaining findings are P2 style/edge-case items that do not affect the default production path. Both findings are P2: dead code ( No files require special attention for merge readiness.
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 79afb532f0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
79afb53 to
35f1b5e
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 35f1b5e02c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 098e31ab21
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4bc5ab4238
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 35cd4f8868
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d91c790e1f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5be46c96fd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1fef456187
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9ea756da48
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fcdcb82c2e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if !strings.HasPrefix(lower, bodyStartLower) || !strings.HasSuffix(lower, bodyEndLower) { | ||
| return "", false | ||
| } | ||
| body := text[len(bodyTagStart) : len(text)-len(bodyTagEnd)] | ||
| bodyLower := lower[len(bodyTagStart) : len(lower)-len(bodyTagEnd)] |
There was a problem hiding this comment.
This unwrapping logic treats any text that starts with <body> and ends with </body> as a valid wrapper, even when the final </body> token is part of translated content rather than a real closing tag. In a truncated model response like <body>... literal </body> (no actual wrapper close), sanitizeDocChunkProtocolWrappers will strip the outer tags and accept a truncated chunk, silently dropping the trailing literal token instead of forcing a retry/error. That can corrupt generated docs for pages that mention </body> literally.
Useful? React with 👍 / 👎.
fcdcb82 to
6a16d66
Compare
|
Merged via squash.
Thanks @hxy91819! |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6a16d66486
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| current.WriteString(line) | ||
| if fenceDelimiter != "" { | ||
| if lineDelimiter != "" && lineDelimiter[0] == fenceDelimiter[0] && len(lineDelimiter) >= len(fenceDelimiter) && isClosingFenceLine(line, fenceDelimiter) { |
There was a problem hiding this comment.
splitDocBlockSections treats a fence as closed only when isClosingFenceLine matches the original opener length exactly, so valid Markdown closers with more fence characters (for example opening with ``` and closing with ````) are not recognized. In that case fenceDelimiter stays open through the rest of the block, and retry planning falls back to non-fence-aware splits that can cut fenced content into invalid chunks and fail translation/validation for otherwise valid docs.
Useful? React with 👍 / 👎.
Merged via squash. Prepared head SHA: 6a16d66 Co-authored-by: hxy91819 <[email protected]> Co-authored-by: hxy91819 <[email protected]> Reviewed-by: @hxy91819
Merged via squash. Prepared head SHA: 6a16d66 Co-authored-by: hxy91819 <[email protected]> Co-authored-by: hxy91819 <[email protected]> Reviewed-by: @hxy91819
Merged via squash. Prepared head SHA: 6a16d66 Co-authored-by: hxy91819 <[email protected]> Co-authored-by: hxy91819 <[email protected]> Reviewed-by: @hxy91819
Merged via squash. Prepared head SHA: 6a16d66 Co-authored-by: hxy91819 <[email protected]> Co-authored-by: hxy91819 <[email protected]> Reviewed-by: @hxy91819
Merged via squash. Prepared head SHA: 6a16d66 Co-authored-by: hxy91819 <[email protected]> Co-authored-by: hxy91819 <[email protected]> Reviewed-by: @hxy91819
Merged via squash. Prepared head SHA: 6a16d66 Co-authored-by: hxy91819 <[email protected]> Co-authored-by: hxy91819 <[email protected]> Reviewed-by: @hxy91819
2 participants
Summary
</body>tag instead of accepting truncated EOF bodiesopenai/gpt-5.4pathWhy
Compatibility
openaiprovider andgpt-5.4model, it still passes--provider openai --model gpt-5.4PI_CODING_AGENT_DIRValidation
cd scripts/docs-i18n && go test ./...docs/help/troubleshooting.mddocs/help/faq.mddocs/gateway/configuration-reference.mddocs/install/fly.mddocs/install/node.mddocs/help/faq.md,docs/gateway/configuration-reference.md,docs/install/fly.md,docs/install/installer.md,docs/plugins/sdk-provider-plugins.mddocs/index.md,docs/channels/discord.md,docs/gateway/index.md,docs/gateway/openai-http-api.md,docs/plugins/bundles.md,docs/plugins/sdk-overview.md,docs/plugins/building-plugins.md,docs/plugins/sdk-channel-plugins.md,docs/reference/wizard.mddocs/install/azure.md,docs/install/gcp.md,docs/install/migrating.md,docs/install/raspberry-pi.md,docs/channels/telegram.md,docs/gateway/troubleshooting.md,docs/gateway/authentication.md,docs/plugins/sdk-runtime.md,docs/plugins/sdk-entrypoints.md,docs/reference/token-use.mddocs/concepts/model-providers.md,docs/concepts/oauth.md,docs/gateway/configuration-examples.md,docs/gateway/doctor.md,docs/help/environment.md,docs/help/testing.md,docs/providers/anthropic.md,docs/providers/openai.md,docs/reference/api-usage-costs.md,docs/reference/test.md,docs/tools/media-overview.md,docs/tools/music-generation.md,docs/tools/video-generation.mdprocessed=28 skipped=0<frontmatter>,</frontmatter>,<body>,</body>,[[[FM_*]]])Notes