|
| 1 | +# 002 — Tool-result encoding decode |
| 2 | + |
| 3 | +Source: `src/adapters/cursor/protobuf-request.ts`, `gen/agent_pb.ts`, |
| 4 | +`native-exec-mcp.ts`. Verified by direct read plus an independent |
| 5 | +`gpt-5.6-sol` audit. |
| 6 | + |
| 7 | +## F2 — images are destroyed, and the wire did not ask for that (High) |
| 8 | + |
| 9 | +`contentToText` (`protobuf-request.ts:328`) maps every non-text part of a tool |
| 10 | +result to a literal placeholder: |
| 11 | + |
| 12 | +```ts |
| 13 | +.map(part => part.type === "text" ? part.text : `[image input unsupported by Cursor adapter phase 3: ${part.detail ?? "auto"}]`) |
| 14 | +``` |
| 15 | + |
| 16 | +`toolResultPart` (`:384`) then always builds exactly one `text` content item. So |
| 17 | +a screenshot returned by Computer Use, a browser QA tool, or any image-returning |
| 18 | +MCP tool reaches Cursor as the string |
| 19 | +`[image input unsupported by Cursor adapter phase 3: auto]` and nothing else. |
| 20 | + |
| 21 | +**The Cursor protobuf supports images.** `McpToolResultContentItem.content` is a |
| 22 | +oneof with exactly two defined cases (`gen/agent_pb.ts:8476`): |
| 23 | + |
| 24 | +```ts |
| 25 | +content: |
| 26 | + | { value: McpTextContent; case: "text" } // field 1 |
| 27 | + | { value: McpImageContent; case: "image" } // field 2 |
| 28 | + | { case: undefined; value?: undefined }; |
| 29 | +``` |
| 30 | + |
| 31 | +`McpImageContent` (`:8449`) carries `data: Uint8Array` (bytes, base64 in JSON) |
| 32 | +and `mimeType: string`. |
| 33 | + |
| 34 | +**The adapter already knows how to send one.** `native-exec-mcp.ts:115` decodes |
| 35 | +base64 from an MCP block and emits a real `McpImageContent`. That path covers |
| 36 | +tools invoked through `CursorMcpManager`; it does not cover Codex |
| 37 | +`OcxToolResultMessage` values flowing through `protobuf-request.ts`. The |
| 38 | +placeholder is therefore not a wire limitation but an unfinished migration — the |
| 39 | +"phase 3" in its own text. |
| 40 | + |
| 41 | +A model driving Computer Use therefore receives a blind result and cannot see |
| 42 | +what its own action did. That is a real capability loss, proven by source |
| 43 | +reading. |
| 44 | + |
| 45 | +**It is not proven to be the cause of the reported retries and session resets.** |
| 46 | +No live trace ties those symptoms to this branch, and the external evidence in |
| 47 | +`004` points at least partly elsewhere (the `node_repl` runtime). An adversarial |
| 48 | +audit flagged the original causal claim as unsupported; it is withdrawn. The |
| 49 | +defect stands on its own merits and does not need to explain every symptom to be |
| 50 | +worth fixing. |
| 51 | + |
| 52 | +## The three result paths in `conversationTurns` |
| 53 | + |
| 54 | +| Path | Behavior | Lines | |
| 55 | +|------|----------|-------| |
| 56 | +| External model | `AssistantMessage` with `[Tool Result]`/`[Tool Error]` + placeholder | 481-490 | |
| 57 | +| Native model, matching pending call | `toolResultPart(result)` attached to the MCP call — still text-only | 493-496 | |
| 58 | +| Native model, no matching call | `toolResultToText` as an `AssistantMessage` | 498-503 | |
| 59 | + |
| 60 | +All three lose the image. A fix must cover the native path (real |
| 61 | +`McpImageContent`) and degrade honestly on the external path. |
| 62 | + |
| 63 | +## Other image surfaces (context, not in scope) |
| 64 | + |
| 65 | +User-message images are also placeholdered, at `request-builder.ts:201` and |
| 66 | +`protobuf-request.ts:314`. The schema would support them: `UserMessage` has |
| 67 | +`selectedContext` -> `selectedImages` (`agent_pb.ts:1823`, `:11178`), and |
| 68 | +`SelectedImage` accepts `blobId`, inline `data`, or `blobIdWithData` |
| 69 | +(`:10389`). No adapter code populates these. Out of scope for this unit; noted |
| 70 | +so a later unit does not have to rediscover it. |
| 71 | + |
| 72 | +## Size limits (hypothesis partly disproved) |
| 73 | + |
| 74 | +There is no single whole-request cap and no cap inside `contentToText`. What |
| 75 | +exists: |
| 76 | + |
| 77 | +- tool catalog: 330 tools / 120,000 protobuf bytes (`request-builder.ts:29`) — |
| 78 | + definitions only, not results; |
| 79 | +- external root replay: 192 roots / 512 KiB with UTF-8 truncation marked |
| 80 | + `…[truncated for Cursor external replay budget]` (`protobuf-request.ts:60`, `:122`); |
| 81 | +- blob store: 16 MiB per blob, 4,096 entries, 64 MiB total, 15-minute TTL |
| 82 | + (`native-exec.ts:81`); |
| 83 | +- `requestScope` pins blobs for the in-flight request so eviction cannot |
| 84 | + invalidate advertised ids; over-capacity raises `CursorBlobAdmissionError` |
| 85 | + rather than truncating (`native-exec.ts:363`, `:374`). |
| 86 | + |
| 87 | +Implication for `020`: adding real image bytes to results makes the blob and |
| 88 | +replay budgets load-bearing, so the fix must bound image payloads deliberately |
| 89 | +instead of trusting these limits to absorb them. |
| 90 | + |
0 commit comments