Repository navigation
Cap the rendered tool result at the response size limit - #1298
Merged
Merged
Conversation
#1279 limits the upstream body the REST, GraphQL and SOAP engines read (TOOL_MAX_RESPONSE_BYTES, 10 MB by default). What the client receives is larger: the result is rendered as indented JSON, and database and MCP-bridge connectors read no HTTP body at all. On Cloud, 11 results of 10.4 to 16.2 MB still went out to clients in one day. The executor now holds the rendered text to the same limit for every connector type, including results read back from the response cache. A text over the limit is rendered again as compact JSON, with the workflow hint kept; if that is still too big, the call returns the TOOL_RESPONSE_TOO_LARGE error with the same guidance to narrow the request, and no text, structuredContent, image or file. The string length settles most sizes, so Buffer.byteLength only runs on texts between a third of the limit and the limit, and a value that is not an object or array is not serialized twice. Such a call is logged as an error without its output (the excerpt would serialize the payload once more), and is neither cached nor fed to the knowledge graph. Results under the limit are unchanged. Same behaviour on Cloud and self-hosted.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
#1279 caps the upstream body that the REST (also JSON-RPC and OData), GraphQL and SOAP engines read (
TOOL_MAX_RESPONSE_BYTES, 10 MB by default). The client receives more than that. The executor renders the result as indented JSON, which can be much bigger than the body it came from, and database and MCP-bridge connectors read no HTTP body at all. On Cloud, 11 results between 10.4 and 16.2 MB still reached clients in 24 hours. This PR applies the same limit to the text the MCP client actually receives.Changes
renderResult(dynamic-mcp-tools.ts) checks the rendered text, workflow hint included, againstTOOL_MAX_RESPONSE_BYTES. This covers every connector type and results read back from the response cache.TOOL_RESPONSE_TOO_LARGEerror (isError: true,{ error, code }). It uses the same message as Cap the upstream response size of a tool call #1279, with the same advice to narrow the request and the same mention ofTOOL_MAX_RESPONSE_BYTES, but says "tool result" instead of "API response". No text,structuredContent, image or file from that call is sent. Onetool_result_too_largewarning is logged through the NestLogger.fitsResponseLimit()(common/response-size-limit.ts) measures UTF-8 bytes. One UTF-16 code unit takes 1 to 3 bytes, so the string length alone decides most cases.Buffer.byteLengthruns only for texts between one third of the limit and the limit. A value that is not an object or array (a plain string) is not serialized twice.tool_invocationsasERROR, with the error text and nooutput. This matches the upstream-cap path. Building the 16 KB excerpt would also serialize the whole payload one more time. The call is not cached (every cache read would fail the same way) and not fed to the knowledge graph. To make this possible, the render step now runs before the audit row is written, not after.ResponseTooLargeErroracceptssource: 'result'for that wording. The upstream message is unchanged..env.example,docs/deployment.mdanddocs/tool-definition.mddescribe the second check.The upstream
maxContentLengthcap is unchanged.Cloud and self-hosted: same behaviour on both, with the same 10 MB default and the same
TOOL_MAX_RESPONSE_BYTESoverride. Noee/code is touched.Not covered here (worth a follow-up):
outputSchema, the controller also sendsstructuredContent, the same value again. A result just under the limit can therefore reach about twice the limit on the wire.boundPayloadin the audit log still serializes each successful output once in full to measure it, as it did before.Type
Testing
New specs:
dynamic-mcp-tools.spec.ts› "the size limit on the rendered result": 11 tests. They cover a result under the limit (unchanged), the compact fallback, the hint kept and counted, the error result, the audit row (ERROR, no output), no cache write, bytes counted instead of characters, the DATABASE, MCP, GRAPHQL and SOAP engines, the cache-hit path, and the 10 MB default.response-size-limit.spec.ts:fitsResponseLimit(multi-byte and surrogate pairs, length short-circuit) and the "tool result" message.Without the source change, 10 of the 11 new executor tests fail. The one that passes is "leaves a result under the limit untouched".
Commands, all run in
packages/backendafternpm ciat the root andDATABASE_URL=postgresql://u:p@localhost:5433/x npx prisma generate:npx jest src/mcp-server src/common/response-size-limit.spec.ts src/connectors/engines/response-size-limit.engines.spec.ts src/audit --silentnpx jest --silent(full backend suite)npx tsc --noEmit -p tsconfig.jsonnpx eslint src/common/response-size-limit.ts src/common/response-size-limit.spec.ts src/mcp-server/dynamic-mcp-tools.ts src/mcp-server/dynamic-mcp-tools.spec.tsI did not run the repo-wide
npm run lint, because it rewrites files.Checklist
npm test)Related Issues
Relates to #1279