Skip to content

feat(core): cap images per request with a per-model limit.images - #62688

Open
TheStranjer wants to merge 1 commit into
anomalyco:devfrom
TheStranjer:model-image-limit
Open

TheStranjer wants to merge 1 commit into
anomalyco:devfrom
TheStranjer:model-image-limit

Conversation

@TheStranjer

@TheStranjer TheStranjer commented Nov 8, 2026 •

Copy link
Copy Markdown

Issue for this PR

Closes #59794

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Some backends reject any request with more images than they allow, e.g. vLLM with --limit-mm-per-prompt '{"image":4}', often behind LiteLLM. OpenCode sends every image in the session history again on each request, so once a session goes over the cap, every later request fails, even plain text prompts. boundImages only limits image bytes, not count.

This adds an optional per-model limit.images, next to the token limits. If a request would carry more images than that, boundImages replaces the oldest ones with the placeholder the byte budget already uses and keeps the newest N. Leaving it unset changes nothing.

"models": {
  "qwen3-vl": {
    "capabilities": { "input": ["text", "image"] },
    "limit": { "context": 131072, "output": 8192, "images": 4 },
  },
}
  • Config layers merge limit.images the same way as the other limit fields.
  • GitHub Copilot models get it automatically from the max_prompt_images value in Copilot's model list.
  • Compaction's prompt estimates apply the same cap, so they match what is actually sent.
  • Docs: a note on the models page and a row in the attachments limits table. Client types are regenerated.

Related PRs:

Trade-off: the byte budget cuts in steps so the prompt cache survives. The count cap can't do that, because the backend rejects even one image over the limit. Once a session is past the cap, each new image moves the cut by one, and the prompt cache misses from the first replaced image onward.

No feature flag: the setting does nothing unless configured. The only models that get a value automatically are Copilot's, from the cap Copilot itself reports.

How did you verify your code works?

  • test/session-model-request.test.ts: with six images and limit.images: 4, the oldest two are replaced and the newest four are sent unchanged. With a limit of 6, the messages come back unchanged.
  • test/github-copilot/models.test.ts: limit.images is read from max_prompt_images, and stays unset for models that report no vision limits.
  • test/config/provider.test.ts: limit.images survives layered config merges.
  • bun test in core (session request, compaction, runner, Copilot, config provider), schema and protocol: all pass.
  • bun run check from the root: 33/33 tasks pass. bun run generate in packages/client.

Visual

<svg xmlns="http://www.w3.org/2000/svg" width="240" height="72" viewBox="0 0 240 72" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round">
<style>
.new{opacity:0;animation:new 4s ease-in-out infinite}
.gone{animation:gone 4s ease-in-out infinite}
.ph{opacity:0;animation:ph 4s ease-in-out infinite}
.br{animation:br 4s ease-in-out infinite}
@keyframes new{0%,15%{opacity:0;transform:translateY(-8px)}35%,100%{opacity:1;transform:none}}
@keyframes gone{0%,40%{opacity:1}60%,100%{opacity:0}}
@keyframes ph{0%,45%{opacity:0}65%,100%{opacity:1}}
@keyframes br{0%,40%{transform:none}60%,100%{transform:translateX(32px)}}
@media (prefers-reduced-motion:reduce){.new,.gone,.ph,.br{animation:none}}
</style>
<defs>
<g id="img"><rect width="24" height="24" rx="3"/><path d="M4 19l6-7 4 4 3-3 3 6"/><circle cx="17" cy="7" r="2"/></g>
<g id="ph" opacity=".5"><rect width="24" height="24" rx="3" stroke-dasharray="3 3"/><path d="M6 10h12M6 15h8"/></g>
</defs>
<use href="#ph" x="12" y="20"/>
<use href="#ph" x="44" y="20"/>
<use href="#img" x="76" y="20" class="gone" opacity=".85"/>
<use href="#ph" x="76" y="20" class="ph"/>
<use href="#img" x="108" y="20" opacity=".85"/>
<use href="#img" x="140" y="20" opacity=".85"/>
<use href="#img" x="172" y="20" opacity=".85"/>
<g class="new"><use href="#img" x="204" y="20" opacity=".85"/></g>
<g class="br" opacity=".45"><path d="M76 52v4h120v-4"/><text x="136" y="68" fill="currentColor" stroke="none" font-family="ui-monospace,monospace" font-size="9" text-anchor="middle">limit.images: 4</text></g>
</svg>

Screenshots / recordings

N/A, not a UI change.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

🤖 Generated with Claude Code


Added by the opencode bot (AI-generated).

Related: #47721, #48095

Public API: adds optional per-model limit.images config key and Model.Info.limit.images (backwards compatible)

Endpoints such as vLLM behind LiteLLM reject requests with more images than
they serve. A model's limit.images now makes OpenCode replace the oldest images
with a note to stay within it. GitHub Copilot models take it from
max_prompt_images. Compaction estimates apply the same cap.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change looks sound. boundImages counts and replaces images in the same order: user images, assistant images and images inside tool results. Images that are only a link count toward the image limit but not the byte budget, which is right for a count cap. Compaction's estimates and its recent-messages step use the same cap, so they match what is actually sent. Leaving the setting unset changes nothing.

How I checked: I ran the changed test files and the compaction tests (all pass), plus a quick throwaway test with mixed user and tool-result images. With limits of 2, 0, larger than the image count, and unset, it kept exactly the newest N images. I could not test against a backend that limits images per request, so this was not checked end to end.

Public API: this adds limit.images (a non-negative integer, optional) to model config, at provider.<id>.models.<id>.limit. It also adds limit.images? to Model.Info and to the generated client types ModelInfo and ConfigEntry. It is backwards compatible, since existing configs and callers are unaffected. Copilot models now fill it in from max_prompt_images.

No blocking concerns. Maintainers should note that #61563 fixes the same issue with one global media.image.max_images setting, so only one of the two approaches should land.

Related: #47721, #48095

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

session: all history images are re-sent on every request, permanently breaking sessions on image-capped backends

1 participant