Repository navigation
feat(core): cap images per request with a per-model limit.images - #62688
TheStranjer wants to merge 1 commit into
Conversation
Endpoints such as vLLM behind LiteLLM reject requests with more images than they serve. A model's limit.images now makes OpenCode replace the oldest images with a note to stay within it. GitHub Copilot models take it from max_prompt_images. Compaction estimates apply the same cap.
There was a problem hiding this comment.
The change looks sound. boundImages counts and replaces images in the same order: user images, assistant images and images inside tool results. Images that are only a link count toward the image limit but not the byte budget, which is right for a count cap. Compaction's estimates and its recent-messages step use the same cap, so they match what is actually sent. Leaving the setting unset changes nothing.
How I checked: I ran the changed test files and the compaction tests (all pass), plus a quick throwaway test with mixed user and tool-result images. With limits of 2, 0, larger than the image count, and unset, it kept exactly the newest N images. I could not test against a backend that limits images per request, so this was not checked end to end.
Public API: this adds limit.images (a non-negative integer, optional) to model config, at provider.<id>.models.<id>.limit. It also adds limit.images? to Model.Info and to the generated client types ModelInfo and ConfigEntry. It is backwards compatible, since existing configs and callers are unaffected. Copilot models now fill it in from max_prompt_images.
No blocking concerns. Maintainers should note that #61563 fixes the same issue with one global media.image.max_images setting, so only one of the two approaches should land.
Issue for this PR
Closes #59794
Type of change
What does this PR do?
Some backends reject any request with more images than they allow, e.g. vLLM with
--limit-mm-per-prompt '{"image":4}', often behind LiteLLM. OpenCode sends every image in the session history again on each request, so once a session goes over the cap, every later request fails, even plain text prompts.boundImagesonly limits image bytes, not count.This adds an optional per-model
limit.images, next to the token limits. If a request would carry more images than that,boundImagesreplaces the oldest ones with the placeholder the byte budget already uses and keeps the newest N. Leaving it unset changes nothing.limit.imagesthe same way as the otherlimitfields.max_prompt_imagesvalue in Copilot's model list.Related PRs:
limit.imagesand 30.media.image.max_images. I made it per model because the cap belongs to the backend serving each model. One config often mixes a 4-image vLLM model with hosted models that accept far more, and a global 4 would strip images from all of them. The token limits already live inlimit, and the per-model field gives catalog sources such as Copilot somewhere to put the value they report.Trade-off: the byte budget cuts in steps so the prompt cache survives. The count cap can't do that, because the backend rejects even one image over the limit. Once a session is past the cap, each new image moves the cut by one, and the prompt cache misses from the first replaced image onward.
No feature flag: the setting does nothing unless configured. The only models that get a value automatically are Copilot's, from the cap Copilot itself reports.
How did you verify your code works?
test/session-model-request.test.ts: with six images andlimit.images: 4, the oldest two are replaced and the newest four are sent unchanged. With a limit of 6, the messages come back unchanged.test/github-copilot/models.test.ts:limit.imagesis read frommax_prompt_images, and stays unset for models that report no vision limits.test/config/provider.test.ts:limit.imagessurvives layered config merges.bun testin core (session request, compaction, runner, Copilot, config provider), schema and protocol: all pass.bun run checkfrom the root: 33/33 tasks pass.bun run generateinpackages/client.Visual
Screenshots / recordings
N/A, not a UI change.
Checklist
🤖 Generated with Claude Code
Added by the opencode bot (AI-generated).
Related: #47721, #48095
Public API: adds optional per-model
limit.imagesconfig key andModel.Info.limit.images(backwards compatible)