Skip to content

Cap the rendered tool result at the response size limit - #1298

Merged
helpcodeai merged 3 commits into
mainfrom
fix/tool-output-size-cap
Oct 27, 2026
Merged

helpcodeai merged 3 commits into
mainfrom
fix/tool-output-size-cap

Conversation

@helpcodeai

Copy link
Copy Markdown
Contributor

Summary

#1279 caps the upstream body that the REST (also JSON-RPC and OData), GraphQL and SOAP engines read (TOOL_MAX_RESPONSE_BYTES, 10 MB by default). The client receives more than that. The executor renders the result as indented JSON, which can be much bigger than the body it came from, and database and MCP-bridge connectors read no HTTP body at all. On Cloud, 11 results between 10.4 and 16.2 MB still reached clients in 24 hours. This PR applies the same limit to the text the MCP client actually receives.

Changes

  • renderResult (dynamic-mcp-tools.ts) checks the rendered text, workflow hint included, against TOOL_MAX_RESPONSE_BYTES. This covers every connector type and results read back from the response cache.
    • Under the limit: no change. Same pretty-printed text, byte for byte.
    • Over the limit, and the value is an object or array: the text is rendered again as compact JSON, and the hint is kept.
    • Still over: the call returns the TOOL_RESPONSE_TOO_LARGE error (isError: true, { error, code }). It uses the same message as Cap the upstream response size of a tool call #1279, with the same advice to narrow the request and the same mention of TOOL_MAX_RESPONSE_BYTES, but says "tool result" instead of "API response". No text, structuredContent, image or file from that call is sent. One tool_result_too_large warning is logged through the Nest Logger.
  • fitsResponseLimit() (common/response-size-limit.ts) measures UTF-8 bytes. One UTF-16 code unit takes 1 to 3 bytes, so the string length alone decides most cases. Buffer.byteLength runs only for texts between one third of the limit and the limit. A value that is not an object or array (a plain string) is not serialized twice.
  • An oversized call is now recorded in tool_invocations as ERROR, with the error text and no output. This matches the upstream-cap path. Building the 16 KB excerpt would also serialize the whole payload one more time. The call is not cached (every cache read would fail the same way) and not fed to the knowledge graph. To make this possible, the render step now runs before the audit row is written, not after.
  • ResponseTooLargeError accepts source: 'result' for that wording. The upstream message is unchanged.
  • Docs: .env.example, docs/deployment.md and docs/tool-definition.md describe the second check.

The upstream maxContentLength cap is unchanged.

Cloud and self-hosted: same behaviour on both, with the same 10 MB default and the same TOOL_MAX_RESPONSE_BYTES override. No ee/ code is touched.

Not covered here (worth a follow-up):

  • When a tool advertises an outputSchema, the controller also sends structuredContent, the same value again. A result just under the limit can therefore reach about twice the limit on the wire.
  • Base64 images and files are not counted; files already have their own per-file limit of at most 10 MiB.
  • boundPayload in the audit log still serializes each successful output once in full to measure it, as it did before.

Type

  • Bug fix
  • New feature
  • Documentation
  • Refactoring
  • Performance improvement
  • Other (describe below)

Testing

  • Added/updated unit tests
  • Tested manually (describe steps below)
  • No tests needed (explain why)

New specs:

  • dynamic-mcp-tools.spec.ts › "the size limit on the rendered result": 11 tests. They cover a result under the limit (unchanged), the compact fallback, the hint kept and counted, the error result, the audit row (ERROR, no output), no cache write, bytes counted instead of characters, the DATABASE, MCP, GRAPHQL and SOAP engines, the cache-hit path, and the 10 MB default.
  • response-size-limit.spec.ts: fitsResponseLimit (multi-byte and surrogate pairs, length short-circuit) and the "tool result" message.

Without the source change, 10 of the 11 new executor tests fail. The one that passes is "leaves a result under the limit untouched".

Commands, all run in packages/backend after npm ci at the root and DATABASE_URL=postgresql://u:p@localhost:5433/x npx prisma generate:

Command Result
npx jest src/mcp-server src/common/response-size-limit.spec.ts src/connectors/engines/response-size-limit.engines.spec.ts src/audit --silent 40 suites passed, 682 tests passed
npx jest --silent (full backend suite) 777 suites passed, 4 skipped; 23147 tests passed, 1593 skipped, 0 failed
npx tsc --noEmit -p tsconfig.json exit 0
npx eslint src/common/response-size-limit.ts src/common/response-size-limit.spec.ts src/mcp-server/dynamic-mcp-tools.ts src/mcp-server/dynamic-mcp-tools.spec.ts exit 0, no output

I did not run the repo-wide npm run lint, because it rewrites files.

Checklist

  • My code follows the existing code style
  • I have tested my changes locally
  • I have updated documentation (if applicable)
  • All existing tests pass (npm test)
  • This PR has a descriptive title

Related Issues

Relates to #1279

#1279 limits the upstream body the REST, GraphQL and SOAP engines read
(TOOL_MAX_RESPONSE_BYTES, 10 MB by default). What the client receives is
larger: the result is rendered as indented JSON, and database and
MCP-bridge connectors read no HTTP body at all. On Cloud, 11 results of
10.4 to 16.2 MB still went out to clients in one day.

The executor now holds the rendered text to the same limit for every
connector type, including results read back from the response cache.
A text over the limit is rendered again as compact JSON, with the
workflow hint kept; if that is still too big, the call returns the
TOOL_RESPONSE_TOO_LARGE error with the same guidance to narrow the
request, and no text, structuredContent, image or file. The string
length settles most sizes, so Buffer.byteLength only runs on texts
between a third of the limit and the limit, and a value that is not an
object or array is not serialized twice.

Such a call is logged as an error without its output (the excerpt would
serialize the payload once more), and is neither cached nor fed to the
knowledge graph. Results under the limit are unchanged. Same behaviour on
Cloud and self-hosted.
@helpcodeai
helpcodeai enabled auto-merge (squash) October 27, 2026 10:34
@helpcodeai helpcodeai mentioned this pull request Oct 27, 2026
3 tasks done
@helpcodeai
helpcodeai merged commit eced9be into main Oct 27, 2026
14 checks passed
@helpcodeai
helpcodeai deleted the fix/tool-output-size-cap branch October 27, 2026 10:42
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 27, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant