fix(extensions): measure the team response size gate in UTF-8 bytes by Lesereingrape · Pull Request #6274 · bytedance/deer-flow

Lesereingrape · 2026-10-04T10:10:58Z

Why

The trigger here is the team
collaboration plugin that landed on main a few hours ago (#6243). Its resume action
sizes a clarification answer with len(json.dumps(response).encode())
(examples/deerflow-extension-agent-teams/deerflow_extension_agent_teams/service.py:321),
and json.dumps escapes non-ASCII by default: a 漢 becomes the six ASCII bytes
\u6f22, and an astral character such as 🙂 becomes twelve. The gate then compares that
escaped length against 8,000, so it enforces roughly a third to a half of the bound the
plugin documents and validates in its own UI.

The pain is user-facing and specific to the language the answer is written in. The README
promises that "tasks and clarification answers accept up to 4,000 characters / 8,000 UTF-8
bytes" and that "the UI validates both bounds before submission and preserves oversized
text for editing" (README.md:107-112), and static/index.mjs does exactly that —
inputText rejects only Array.from(value).length > 4000 or
new TextEncoder().encode(value).length > 8000 (:29-33, applied to the answer at
:802-804). A Chinese answer between 1,334 and 2,666 characters is inside both of those
bounds, is accepted by the form, is refused by the server, and leaves the job in
waiting_input. The measured divergence:

answer	chars	real UTF-8 bytes	counted by the gate	per the bound	gate today	gate with this change
`漢` × 1,333	1,333	3,999	8,000	accepted	accepted	accepted
`漢` × 1,334	1,334	4,002	8,006	accepted	refused	accepted
`漢` × 2,000	2,000	6,000	12,002	accepted	refused	accepted
`漢` × 2,666	2,666	7,998	15,998	accepted	refused	accepted
`漢` × 2,667	2,667	8,001	16,004	refused	refused	refused
`🙂` × 1,000	1,000	4,000	12,002	accepted	refused	accepted

The bound is also not something the user can work towards, because the number in the
refusal ("at most 8 KiB") describes escaped JSON rather than the text in the box.

What changed

A clarification answer (and any JSON response to a confirmation request) is now measured
in UTF-8 bytes, which is the unit the error message, the README and the UI all name. Callers
sending non-ASCII answers regain the documented ceiling: up to 2,666 CJK characters or
4,000 characters of any mix that fits in 8,000 UTF-8 bytes, instead of 1,333 CJK
characters. Answers that genuinely exceed 8,000 UTF-8 bytes are still refused, unchanged.
No storage shape, no API shape and no other bound changes.

ensure_ascii=False was already this module's convention for every site that measures
serialized bytes — the 24 KiB shared-context budget walker (service.py:47), the revision
hash (:203), the routing prompt (:445) and the receipt payload (:538) all pass it,
and the 6,000-byte final-answer cap measures answer.encode() directly (:518). Line 321
was the one exception, so this also makes the module internally consistent. Tasks — which
the README documents with the identical 4,000 / 8,000 pair — were already checked against
real UTF-8 bytes by the module's own text() helper; only answers diverged.

Surface area

Frontend UI
Backend API — the team plugin's resume backend action under
examples/deerflow-extension-agent-teams
Agents / LangGraph
Sandbox
Skills
Dependencies
Default behavior change — answers that this gate silently refused before (non-ASCII,
inside the documented bound) are now accepted; nothing that was accepted becomes refused,
except in a 2-byte boundary band described under "Bug fix verification"
Docs / tests / CI only

Screenshots / Recording

Not applicable — no frontend file is touched.

Bug fix verification

Test path: backend/tests/test_agent_teams_plugin.py.

It goes red on main and green on this branch. Measured at 5c3329af by checking out the
upstream service.py under the new tests, then restoring:

2 failed, 3 passed   # upstream service.py + the new cases
5 passed             # this branch

The two red cases are …[cjk-2000-chars] and …[emoji-1000-chars], both failing with
ValueError: An explicit response of at most 8 KiB is required at service.py:322. The
three green cases in the same run are the control group: 4,000 ASCII characters (accepted
both before and after), and the two oversized answers — 3,000 漢 (9,000 UTF-8 bytes) and
a JSON object with a 9,000-byte string — which are refused both before and after, so the
change cannot be credited with simply disabling the gate.

resume reaches the gate before any other bound. On the host-interrupt path the job is
parked in waiting_input with clarification = None (service.py:482-484), so
control() never applies the module's text() helper and line 321 is the only size check
the answer faces; the new interrupted_job helper parks a job that way deliberately, so
the tests measure the gate rather than an unrelated check. On the team-clarification path
control() does apply text(payload, "response"), which compares len(value.encode())
against 8,000 and would accept the same answer — but the escaped gate refuses it one line
earlier, which is exactly the divergence the UI cannot warn about.

One honest boundary residual: json.dumps still adds the two quote bytes of the JSON
envelope, so an answer of exactly 8,000 UTF-8 bytes is refused by the fixed gate
(8,002 counted). The largest CJK answer that passes is therefore 2,666 characters
(7,998 bytes), which is still the number the bound implies, since 2,667 characters is
8,001 bytes and genuinely over. Tightening the comparison to the envelope, or rewording
the message to say "8 KiB of serialized JSON", is a one-line follow-up if a maintainer
prefers the bound stated the other way.

Validation

Run on this branch at 5c3329af, Python 3.12.11:

$ pytest tests/test_agent_teams_plugin.py -k clarification_answer -q
5 passed

$ pytest tests/test_agent_teams_plugin.py -q
47 passed        # 42 on main + the 5 new cases

$ ruff check ../examples/deerflow-extension-agent-teams/deerflow_extension_agent_teams/service.py tests/test_agent_teams_plugin.py
All checks passed!

$ ruff format --check ../examples/deerflow-extension-agent-teams/deerflow_extension_agent_teams/service.py tests/test_agent_teams_plugin.py
2 files already formatted

make lint && make test from backend/ was not run: this Windows machine has no make,
so the ruff commands above are the same tools invoked through the venv directly, limited to
the two changed files. The diff touches only examples/deerflow-extension-agent-teams/**
and its backend test file; no file under backend/app is modified.

Diff is 2 files, +35 −1: one argument at the call site, one helper and two parametrized
tests.

AI assistance

Tool(s) used: Qoder (AI coding agent)

How you used it: The agent traced the UI/server divergence, wrote the failing tests,
applied the ensure_ascii=False change and ran every command above; the RED numbers were
measured by restoring the upstream service.py under the new tests and re-running them.

I've read and understand every line of this change and take responsibility for it —
it's not unreviewed AI output.

Left unticked deliberately: an agent wrote and measured this diff, so the box is for the
account holder to tick after their own read.

Agent-assisted contribution, disclosed per AGENTS.md.

json.dumps escapes non-ASCII by default, so a CJK clarification answer counted ~6 escaped bytes per character and an emoji 12, while the plugin documents 4,000 characters / 8,000 UTF-8 bytes and its own UI validates exactly those two bounds. A 漢 answer of 2,000 characters (6,000 UTF-8 bytes, inside the bound) was refused by the gate. Serialize with ensure_ascii=False so the gate measures the UTF-8 bytes it names, matching the budget walkers in the same module.

willem-bd

Correct, well-motivated fix with a genuine red-on-main/green-here test split, and it matches the module's existing ensure_ascii=False convention for byte measurement. Two residual, non-blocking points on the gate line: the 2-byte JSON envelope still cuts a narrow band out of the documented 8,000-byte ceiling (suggestion), and unpaired-surrogate input now fails with a codec message instead of the friendly size error (nit).

willem-bd · 2026-10-04T11:32:12Z

    async def resume(self, payload, context):
        fields(payload, "team_id job_id response request_id")
-        if payload["response"] is None or len(json.dumps(payload["response"]).encode()) > 8000:
+        if payload["response"] is None or len(json.dumps(payload["response"], ensure_ascii=False).encode()) > 8000:


The change is right, but a narrow band of the original UI/server divergence survives: the gate still counts the two JSON envelope quotes, so a plain-string answer of 7,999-8,000 UTF-8 bytes serializes to 8,001-8,002 counted bytes and is refused, while the UI (static/index.mjs, inputText: new TextEncoder().encode(value).length > limit * 2) accepts up to exactly 8,000. An answer at the documented ceiling still passes UI validation and lands the job back in waiting_input. Since the documented answer case is always a string, consider measuring it directly and reserving the serialized envelope for non-strings, e.g. size = len(payload["response"].encode()) if isinstance(payload["response"], str) else len(json.dumps(payload["response"], ensure_ascii=False).encode()), plus a 7,999/8,000/8,001 boundary test to pin whichever semantics you choose.

willem-bd · 2026-10-04T11:32:12Z

        fields(payload, "team_id job_id response request_id")
-        if payload["response"] is None or len(json.dumps(payload["response"]).encode()) > 8000:
+        if payload["response"] is None or len(json.dumps(payload["response"], ensure_ascii=False).encode()) > 8000:
            raise ValueError("An explicit response of at most 8 KiB is required")


Related edge: with ensure_ascii=False, a response containing an unpaired surrogate (JSON "\ud800" — decodable by Python's json.loads) now raises UnicodeEncodeError from .encode() before the size comparison, so the caller gets a codec message instead of this friendly "8 KiB" error. It is still a ValueError subclass, so API error handling is unchanged; wrapping the measurement or pre-validating would keep the message uniform for malformed input.

Measure string answers without JSON quotes or escapes, retain the structured JSON byte limit, and reject malformed Unicode with the normal response validation error. Add boundary and state-preservation regressions and document the response budgets.

github-actions Bot added risk:medium Medium risk: regular code changes size/S PR changes 20-100 lines labels Oct 4, 2026

willem-bd reviewed Oct 4, 2026

View reviewed changes

github-actions Bot added area:docs Documentation and Markdown only size/M PR changes 100-300 lines and removed size/S PR changes 20-100 lines labels Oct 5, 2026

WillemJiang added this to the 2.2.0 milestone Oct 5, 2026

WillemJiang approved these changes Oct 5, 2026

View reviewed changes

WillemJiang merged commit d67a9e4 into bytedance:main Oct 5, 2026
14 checks passed

Navigation Menu

Repository navigation

fix(extensions): measure the team response size gate in UTF-8 bytes - #6274

fix(extensions): measure the team response size gate in UTF-8 bytes#6274
WillemJiang merged 2 commits into
bytedance:mainfrom
Lesereingrape:fix/agent-teams-response-size-gate-utf8

Lesereingrape commented Oct 4, 2026

Uh oh!

willem-bd left a comment

Uh oh!

willem-bd Oct 4, 2026

Uh oh!

willem-bd Oct 4, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Lesereingrape commented Oct 4, 2026

Why

What changed

Surface area

Screenshots / Recording

Bug fix verification

Validation

AI assistance

Uh oh!

willem-bd left a comment

Choose a reason for hiding this comment

Uh oh!

willem-bd Oct 4, 2026

Choose a reason for hiding this comment

Uh oh!

willem-bd Oct 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants