fix(tool-payload): enforce UTF-8 byte limits for serialized payloads by qingminglong · Pull Request #102450 · openclaw/openclaw

qingminglong · 2026-07-09T06:11:52Z

What Problem This Solves

Fixes an issue where plain-text tool-call repair could accept serialized JSON or XML-style payloads whose UTF-8 size exceeded maxPayloadBytes when their text contained multibyte characters. Both parser paths previously relied on UTF-16 code-unit counts at the acceptance boundary.

Why This Change Was Made

The parser now applies one UTF-8 byte-length helper to both formats. JSON validates the complete serialized object after its structural scan. XML parameters report each disjoint serialized segment's byte length, allowing cumulative enforcement without repeatedly encoding the entire growing prefix.

This preserves the existing parser grammar and public option while making the documented byte limit accurate for every accepted serialization.

User Impact

Tool-call repair consistently rejects oversized serialized tool arguments for ASCII and multibyte text, preventing over-limit payloads from being promoted into executable tool calls.

Evidence

Exact head: a7db009db708fb3536c4138335e3e90653b36ad4
Blacksmith Testbox run: https://github.com/openclaw/openclaw/actions/runs/28998838177
- pnpm test src/plugin-sdk/tool-payload.test.ts
- 1 file, 24 tests passed.
Real parser invocation on the exact head:

$env:GIT_HEAD = (git rev-parse HEAD)
node --import tsx --input-type=module --eval "import { parseStandalonePlainTextToolCallBlocks } from './src/plugin-sdk/tool-payload.ts'; const parameter = ['<parameter=content>', '\u00e9'.repeat(20), '</parameter>'].join('\n'); const raw = ['<function=write>', parameter, '</function>'].join('\n'); const parserResult = parseStandalonePlainTextToolCallBlocks(raw, { allowedToolNames: ['write'], maxPayloadBytes: 64 }); const proof = { head: process.env.GIT_HEAD, command: 'parseStandalonePlainTextToolCallBlocks multibyte XML parameter', parameterUtf16Length: parameter.length, parameterUtf8Bytes: new TextEncoder().encode(parameter).byteLength, maxPayloadBytes: 64, parserResult }; console.log(JSON.stringify(proof, null, 2)); if (parserResult !== null) { process.exitCode = 1; }"

Output:

{
  "head": "a7db009db708fb3536c4138335e3e90653b36ad4",
  "command": "parseStandalonePlainTextToolCallBlocks multibyte XML parameter",
  "parameterUtf16Length": 53,
  "parameterUtf8Bytes": 73,
  "maxPayloadBytes": 64,
  "parserResult": null
}

Targeted oxfmt and git diff --check passed.
Fresh uncommitted autoreview: clean, no accepted/actionable findings (0.98).

clawsweeper · 2026-07-09T06:13:10Z

Codex review: needs real behavior proof before merge. Reviewed July 9, 2026, 2:18 AM ET / 06:18 UTC.

Summary
The PR changes XML-style plain-text tool-call repair to measure parameter payload limits in UTF-8 bytes and adds a regression test for multibyte content.

PR surface: Source +10, Tests +14. Total +24 across 2 files.

Reproducibility: yes. Source inspection on current main shows XML parameter limits compare UTF-16 offsets, and the PR's regression case is a concrete multibyte payload that stays under the code-unit cap while exceeding UTF-8 bytes.

Review metrics: none identified.

Stored data model
Persistent data-model change detected: serialized state: src/plugin-sdk/tool-payload.test.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🧂 unranked krab
Patch quality: 🐚 platinum hermit
Result: blocked until real behavior proof from a real setup is added.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

[P1] Add redacted terminal output or a terminal screenshot from a real parser invocation showing the multibyte XML payload returns null after the fix.

Proof guidance:

[P1] Needs real behavior proof before merge: The PR body lists targeted Vitest/diff/autoreview results, but that is test evidence rather than after-fix real behavior proof; add redacted terminal output or a terminal screenshot from invoking the parser on the multibyte XML case. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Risk before merge

[P1] External-PR proof is still test-only: the body reports targeted Vitest/diff/autoreview results, but does not include copied terminal output, logs, screenshot, or another real after-fix behavior artifact.
[P1] Live CI had relevant exact-head shards still in progress during inspection, so merge should wait for normal CI completion even though no code finding was found.

Maintainer options:

Decide the mitigation before merge
Land the narrow byte-count parser fix after real behavior proof and exact-head CI complete; keep any broader stream-scrub byte policy as a separate follow-up if maintainers want no visible oversized XML text.
Pause or close
Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

[P1] Human review is needed only to enforce the external-PR proof gate and wait for CI; there is no narrow code repair for ClawSweeper to perform on this branch.

Security
Cleared: The diff only tightens in-repo parser validation and adds a focused test; it does not change dependencies, CI, scripts, secrets, permissions, or package resolution.

Review details

Best possible solution:

Land the narrow byte-count parser fix after real behavior proof and exact-head CI complete; keep any broader stream-scrub byte policy as a separate follow-up if maintainers want no visible oversized XML text.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection on current main shows XML parameter limits compare UTF-16 offsets, and the PR's regression case is a concrete multibyte payload that stays under the code-unit cap while exceeding UTF-8 bytes.

Is this the best way to solve the issue?

Yes for the executable-promotion boundary. Checking UTF-8 byte length at both XML parameter limit points is the narrowest maintainable parser fix; changing stream buffering policy would be broader and is not required to stop over-limit promotion.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 240ca9dbd7f5.

Label changes

Label changes:

add P2: This is a normal-priority tool-call repair bug with limited blast radius and a focused parser/test change.
add rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🐚 platinum hermit.
add status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body lists targeted Vitest/diff/autoreview results, but that is test evidence rather than after-fix real behavior proof; add redacted terminal output or a terminal screenshot from invoking the parser on the multibyte XML case. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Label justifications:

P2: This is a normal-priority tool-call repair bug with limited blast radius and a focused parser/test change.
rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🐚 platinum hermit.
status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The PR body lists targeted Vitest/diff/autoreview results, but that is test evidence rather than after-fix real behavior proof; add redacted terminal output or a terminal screenshot from invoking the parser on the multibyte XML case. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence reviewed

PR surface:

Source +10, Tests +14. Total +24 across 2 files.

View PR surface stats

Area	Files	Added	Removed	Net
Source	1	12	2	+10
Tests	1	14	0	+14
Docs	0	0	0	0
Config	0	0	0	0
Generated	0	0	0	0
Other	0	0	0	0
Total	2	26	2	+24

What I checked:

Repository policy read: Root AGENTS.md and the scoped Plugin SDK guide were read fully; their review guidance required checking the parser boundary, adjacent callers/tests, history, and Codex protocol source before verdict. (AGENTS.md:12, 240ca9dbd7f5)
Live PR state: Live PR metadata shows one contributor commit, no maintainer reviews, only the size: XS label, maintainer edits enabled, and mergeable state behind current main. (250677bf1e8a)
Current-main bug source: Current main rejects XML parameters by comparing string offsets against maxPayloadBytes, so multibyte text can stay under the UTF-16 length cap while exceeding the intended UTF-8 byte cap. (packages/tool-call-repair/src/payload.ts:255, 240ca9dbd7f5)
Cumulative XML cap source: The cumulative XML payload guard on current main also compares offsets rather than UTF-8 bytes, matching the PR's second changed check. (packages/tool-call-repair/src/payload.ts:347, 240ca9dbd7f5)
JSON sibling behavior: The sibling JSON payload path delegates to findJsonObjectEnd with maxPayloadBytes, confirming the existing parser contract is byte-oriented for repaired JSON tool-call payloads. (packages/tool-call-repair/src/payload.ts:142, 240ca9dbd7f5)
PR diff shape: The PR adds a UTF-8 byte length helper and routes both per-parameter and cumulative XML checks through it without changing config, public API, or dependency surfaces. (packages/tool-call-repair/src/payload.ts:262, 250677bf1e8a)

Likely related people:

@vincentkoc: Blame and git log tie the current XML parameter parser, tool-call-repair package files, and adjacent tests to commit 7d0f9de; CONTRIBUTING.md also lists Vincent Koc in Agents and Security areas relevant to tool-call repair hardening. (role: introduced behavior and likely follow-up owner; confidence: high; commits: 7d0f9de907c7; files: packages/tool-call-repair/src/payload.ts, packages/tool-call-repair/src/stream-normalizer.ts, src/plugin-sdk/tool-payload.test.ts)

What the crustacean ranks mean

🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works

ClawSweeper keeps one durable marker-backed review comment per issue or PR.
Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
Maintainers can also comment @clawsweeper review to request a fresh review only.
Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

qingminglong · 2026-07-09T06:42:55Z

@clawsweeper re-review

Added exact-head real parser invocation proof to the PR body. It invokes parseStandalonePlainTextToolCallBlocks on the multibyte XML payload and shows parserResult: null at a7db009db708fb3536c4138335e3e90653b36ad4.

clawsweeper · 2026-07-09T06:42:58Z

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: the existing ClawSweeper review comment will be edited in place when the review finishes.

steipete · 2026-07-09T06:47:09Z

Land-ready maintainer pass complete at exact head a7db009db708fb3536c4138335e3e90653b36ad4.

Improvements beyond the original patch:

Extended the UTF-8 byte-limit fix to JSON, which had the same UTF-16 code-unit gap.
Unified both formats on one bounded byte-length helper.
Changed cumulative XML accounting to add disjoint segment lengths, avoiding repeated prefix encoding.
Corrected the exported option docs and added production-path multibyte JSON coverage.

Proof:

Blacksmith Testbox run https://github.com/openclaw/openclaw/actions/runs/28998838177: pnpm test src/plugin-sdk/tool-payload.test.ts — 24 tests passed.
Exact-head CI run https://github.com/openclaw/openclaw/actions/runs/28999140002 — success.
Targeted oxfmt and git diff --check — passed.
Fresh full-branch autoreview — clean, no accepted/actionable findings (0.98).
Native review artifacts validated; OPENCLAW_TESTBOX=1 scripts/pr prepare-run 102450 passed at the same SHA.

Known proof gaps: none for this parser-only change.

steipete · 2026-07-09T06:47:28Z

Merged via squash.

Prepared head SHA: a7db009db708fb3536c4138335e3e90653b36ad4
Landed commit: b20b02a4763323d7aafa42f4b5334f72684a61f2

…penclaw#102450) * fix(tool-payload): reject oversized XML payload bytes * fix(tools): enforce serialized payload byte limits --------- Co-authored-by: Peter Steinberger <[email protected]>

…penclaw#102450) * fix(tool-payload): reject oversized XML payload bytes * fix(tools): enforce serialized payload byte limits --------- Co-authored-by: Peter Steinberger <[email protected]> (cherry picked from commit b20b02a)

fix(tool-payload): reject oversized XML payload bytes

250677b

openclaw-barnacle Bot added the size: XS label Jul 9, 2026

fix(tools): enforce serialized payload byte limits

a7db009

openclaw-barnacle Bot added size: S and removed size: XS labels Jul 9, 2026

steipete changed the title ~~fix(tool-payload): reject oversized XML parameters when text is multibyte~~ fix(tool-payload): enforce UTF-8 byte limits for serialized payloads Jul 9, 2026

steipete merged commit b20b02a into openclaw:main Jul 9, 2026
120 of 122 checks passed

clawsweeper Bot mentioned this pull request Jul 9, 2026

fix(tool-call-repair): parse JSON-format plain-text tool calls alongside bracket/XML/Harmony #102394

Open

steipete mentioned this pull request Jul 9, 2026

fix: enforce tool call payload byte limits #96399

Closed

clawsweeper Bot mentioned this pull request Jul 10, 2026

fix: antml:/mm: namespaced tool-call XML leaks into chat replies (#97750) #97845

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(tool-payload): enforce UTF-8 byte limits for serialized payloads#102450

fix(tool-payload): enforce UTF-8 byte limits for serialized payloads#102450
steipete merged 2 commits into
openclaw:mainfrom
qingminglong:codex/fix-tool-payload-xml-byte-limit-bytes

qingminglong commented Jul 9, 2026 •

edited

Loading

Uh oh!

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

qingminglong commented Jul 9, 2026

Uh oh!

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

qingminglong commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Problem This Solves

Why This Change Was Made

User Impact

Evidence

Uh oh!

clawsweeper Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

qingminglong commented Jul 9, 2026

Uh oh!

clawsweeper Bot commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Uh oh!

steipete commented Jul 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

qingminglong commented Jul 9, 2026 •

edited

Loading

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading

clawsweeper Bot commented Jul 9, 2026 •

edited

Loading