fix: send screenshot attachments to ChatGPT Codex#157
Conversation
Signed-off-by: Sun-sunshine06 <[email protected]>
Signed-off-by: Sun-sunshine06 <[email protected]>
There was a problem hiding this comment.
Findings
- [Major] Unbounded image payload forwarding can exceed request limits and cause avoidable memory pressure — images are read and base64-encoded up to 10MB each, then every encoded image is appended to the same Codex request without a combined cap (
apps/desktop/src/main/prompt-context.ts:43,apps/desktop/src/main/prompt-context.ts:109,apps/desktop/src/main/prompt-context.ts:155,apps/desktop/src/main/codex-generate.ts:157).
Suggested fix:const MAX_TOTAL_IMAGE_BYTES = 4_000_000; let totalImageBytes = 0; for (const attachment of input.attachments) { if (!attachment.imageDataUrl) continue; const b64 = attachment.imageDataUrl.slice(attachment.imageDataUrl.indexOf(',') + 1); const bytes = Math.floor((b64.length * 3) / 4); if (totalImageBytes + bytes > MAX_TOTAL_IMAGE_BYTES) { throw new CodesignError( 'Attached images are too large in total for ChatGPT Codex. Reduce image count/size.', ERROR_CODES.ATTACHMENT_TOO_LARGE, ); } totalImageBytes += bytes; userContent.push({ type: 'input_image', image_url: attachment.imageDataUrl, detail: 'auto' }); }
Summary
Review mode: initial
- 1 issue found (Major), introduced by the current diff.
docs/VISION.mdanddocs/PRINCIPLES.mdwere not available in this checkout (Not found in repo/docs), so principle/vision checks were limited to in-repo code andCLAUDE.md.
Testing
- Not run (automation). Add a Vitest case that enforces a combined image-byte limit and verifies a typed error is surfaced.
open-codesign Bot
| const userContent: ResponsesInputItem['content'] = [ | ||
| { type: 'input_text', text: buildUserPrompt(input.prompt, sections) }, | ||
| ]; | ||
| for (const attachment of input.attachments) { |
There was a problem hiding this comment.
Forwarding every imageDataUrl here has no aggregate-size guard. With current limits, a user can attach multiple large images and produce a very large JSON request body (base64 expansion included), which risks request rejection and main-process memory spikes. Add a total image-byte cap before pushing input_image parts and throw a typed error when exceeded.
Signed-off-by: Sun-sunshine06 <[email protected]>
There was a problem hiding this comment.
Findings
- None.
Summary
- Review mode: follow-up after new commits
- No new issues found in the latest diff. The previously flagged aggregate image-size risk is now guarded by a total-byte cap with a typed error path (
apps/desktop/src/main/codex-generate.ts). docs/VISION.mdanddocs/PRINCIPLES.md: Not found in repo/docs.
Testing
- Not run (automation). Residual gap: no explicit test for malformed/non-base64
imageDataUrlinputs.
open-codesign Bot
Signed-off-by: Sun-sunshine06 <[email protected]>
There was a problem hiding this comment.
Findings
- None.
Summary
- Review mode: follow-up after new commits
- No new high-confidence issues found in added/modified lines of the latest PR diff.
docs/VISION.mdanddocs/PRINCIPLES.md: Not found in repo/docs.- Residual risk: coverage is layer-local; not found in changed tests is an end-to-end assertion that a desktop image attachment is propagated through
core.generateinto provideruserImageson theopenai-codex-responsesroute.
Testing
- Not run (automation).
open-codesign Bot
Signed-off-by: Sun-sunshine06 <[email protected]>
There was a problem hiding this comment.
Findings
- [Major] Silent drop of malformed image attachments on Codex wire —
attachmentToImageInputreturnsnullwhenimageDataUrldoes not match the expected prefix, andimageInputsForWirefilters those out without surfacing an error. This violates the project hard constraint that fallbacks must not be silent and can cause user-attached images to be omitted from model input with no UI signal. Evidence:packages/core/src/index.ts:176,packages/core/src/index.ts:190.
Suggested fix:function attachmentToImageInput(attachment: AttachmentContext): { data: string; mimeType: string } | null { if (!attachment.imageDataUrl || !attachment.mediaType) return null; const prefix = `data:${attachment.mediaType};base64,`; if (!attachment.imageDataUrl.startsWith(prefix)) { throw new CodesignError( `Malformed image attachment payload for ${attachment.name}`, ERROR_CODES.ATTACHMENT_READ_FAILED, ); } return { data: attachment.imageDataUrl.slice(prefix.length), mimeType: attachment.mediaType, }; }
Summary
- Review mode: follow-up after new commits
- 1 issue found (silent fallback on malformed image payload handling).
docs/VISION.mdanddocs/PRINCIPLES.mdwere referenced in the PR template but are not present in this checkout (Not found in repo/docs), so constraint alignment was validated againstCLAUDE.mdplus diffed code paths only.
Testing
- Not run (automation)
| ): { data: string; mimeType: string } | null { | ||
| if (!attachment.imageDataUrl || !attachment.mediaType) return null; | ||
| const prefix = `data:${attachment.mediaType};base64,`; | ||
| if (!attachment.imageDataUrl.startsWith(prefix)) return null; |
There was a problem hiding this comment.
[Major] This branch silently drops malformed image payloads (return null then filtered out), which can omit user attachments without any surfaced error. Hard constraint says no silent fallbacks.
Suggested fix:
if (!attachment.imageDataUrl.startsWith(prefix)) {
throw new CodesignError(
`Malformed image attachment payload for ${attachment.name}`,
ERROR_CODES.ATTACHMENT_READ_FAILED,
);
}
Summary
Fixes a
chatgpt-codexbug where uploaded screenshots were only reduced to filename hints, so models likegpt-5.4could ignore the attached visual reference entirely. This patch keeps text attachments unchanged, encodes supported image attachments as data URLs in prompt preparation, and forwards them as Responsesinput_imageparts on the ChatGPT Codex generate path.Type of change
Linked issue
Checklist
docs/VISION.md,docs/PRINCIPLES.md, andCLAUDE.mdbefore startinggit commit -s)pnpm lint && pnpm typecheck && pnpm testpasses locallypnpm changeset) if user-visibleValidation run for this PR:
corepack pnpm --filter @open-codesign/desktop test -- --run src/main/prompt-context.test.ts src/main/codex-generate.test.tscorepack pnpm --filter @open-codesign/desktop typecheckbiome check .Note on full test suite:
pnpm teststill has existing Windows baseline failures unrelated to this patch (provider token-store permission assertion, core builtin-skill loader expectation, exporter PDF timeout, and opencode path separator assertions).Screenshots / recordings (UI changes)