Skip to content

feat(core): make image generation a Code Mode tool - #60086

Closed
rekram1-node wants to merge 1 commit into
devfrom
image-tool-codemode
Closed

rekram1-node wants to merge 1 commit into
devfrom
image-tool-codemode

Conversation

@rekram1-node

Copy link
Copy Markdown
Collaborator

image_generate was a direct tool, offered next to execute on every request where an image model exists. I moved it into Code Mode, unpinned, so it sits in the catalog until an agent needs it, and the prompt stays smaller on every turn that doesn't generate images.

It gets its own image namespace rather than opencode, since opencode is about managing OpenCode itself (sessions, worktrees, models). That also keeps it clear of the internal plugin's opencode_image_generate.

 tools offered to the model
   execute
-  image_generate            (direct, every request)
 code mode catalog
+  image (Generate and edit images with the image models of your connected providers.)
+    generate                (tools.image.generate, not pinned)

The tool's effective name stays image_generate, so permission rules ("image_generate": "deny"), background jobs and restart recovery are unchanged. It's still registered only while some connected provider has an image model, and it re-checks when the catalog changes. With Code Mode denied it falls back to a direct image_generate tool like the other Code Mode tools.

Verification

  • packages/core test/tool-image-generate.test.ts (15 pass, twice): the tool shows up in the Code Mode catalog as image.generate, isn't pinned and isn't a direct tool; it appears and disappears with image models; a new test calls tools.image.generate through execute. The other tests run it with Code Mode denied, where it's direct. test/lib/tool.ts's executeTool takes optional permissions for that.
  • session-execution, codemode and tool-schedule tests pass (45).
  • bun run typecheck:tsc in packages/core and bun run check pass.

@opencodejr opencodejr Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The move into Code Mode works as described. tools.image.generate shows up in the catalog unpinned, appears and disappears with image models, and its effective name stays image_generate, so permission rules, background jobs and restart recovery are unchanged. In packages/core the image-generate, codemode and session-execution tests pass (74) and typecheck:tsc is clean.

Public API: unchanged. The model-facing tool moves from image_generate to tools.image.generate, but the permission action, job type and config keys stay the same.

One concern, inline: called through Code Mode, the agent only gets the structured output ({ path, model }). The "started in the background, you'll be notified, don't poll" text and the model-fallback note are lost, and path has no extension.

name,
options: { codemode: false },
name: "generate",
options: { namespace, codemode: true },

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that this runs through Code Mode, the agent only gets output back: Code Mode returns executed.output whenever it is defined and ignores content. I instrumented the new test and the execute result is just { "path": ".../image_<id>", "model": "test/painter" }.

That drops three things the direct tool used to tell the agent:

  • that the job is still running in the background and a notice will follow (without it, the agent may try to read the file right away);
  • that path has no extension yet;
  • the resolved.note saying the configured model was unavailable and another was used.

The catalog description partly covers the first point, but the fallback note has no other route to the agent. One fix would be to add these to Output, e.g. a status: "started" field and an optional note.

@rekram1-node

Copy link
Copy Markdown
Collaborator Author

Closing this. The image tool is now opt-in (experimental.image_generate) and stays a direct tool. In benchmarks, Code Mode hid the background-job behavior from agents, and they polled for the file much more often.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant