feat(app): add generative UI to the GUI - #54747

Agents can now show interactive UI in the timeline. They write a small GenUI program in a genui_render call; the app draws it with OpenCode's own components while the call streams, and every call is checked so the agent knows whether its UI rendered. It is on by default, and Settings > Experimental > Generated UI turns it off.

_{The demo telemetry is synthetic.}

How it works

sequenceDiagram
  participant A as App
  participant C as Core
  participant P as GenUI plugin
  participant M as Model
  A->>C: prompt with metadata.genui = true
  C->>P: prompt hook notes the user is in the app
  C->>P: context hook keeps genui_render and adds a one-time reminder
  M->>C: genui_render argument chunks
  C-->>A: session.tool.input.delta, batched every 100 ms
  A->>A: parse new statements, render, reconcile by statement ID
  C->>P: complete call, checked against the catalog
  P-->>M: rendered, or each problem by statement
  A->>A: buttons run set and reset in place
  A->>C: reply actions and forms send the user's next message

• Core streams tool input. publish-llm-event.ts publishes session.tool.input.delta through the same batching as text and reasoning, and flushes held tool input before the next block starts. The event, the client reducer, the TUI and the CLI already handled it. Schema and the rest of Core are unchanged.
• Offered only in the app, and only when on. The app marks its prompts with metadata.genui, from the Generated UI setting. The plugin removes genui_render from requests whose latest prompt did not say true, such as a terminal, a subagent or the setting turned off, and adds a short reminder when that changes, the way Plan mode reminds. No RPC, polling or permission writes.
• Checked on every call. The tool returns each problem by statement and line. Blocks that fail are left out; everything else still renders.
• Streams without flicker. Each finished statement parses once. While the call streams, a block that does not render yet, such as a chart whose last data point is half written, keeps its previous version.

The language

root = Surface([counts, status, actions])
$approved = 0
$total = 4
counts = Metrics([done, left])
done = Metric("Approved", $approved + " of " + $total)
left = Metric("Remaining", $total - $approved)
status = $approved == $total ? ready : review
ready = Notice("All changes approved", "Apply them when you are ready.", "success")
review = Progress("Review", $approved / $total * 100)
actions = Actions([approve, undo, apply])
approve = Button("Approve next", set($approved, $approved < $total ? $approved + 1 : $total), "primary")
undo = Button("Undo", set($approved, $approved > 0 ? $approved - 1 : 0), "ghost")
apply = Button("Apply approved", reply("Apply the " + $approved + " approved changes"), "secondary")

• One statement per line, components with positional arguments, and text, numbers, booleans, null, lists and objects.
• $name state lives in the app. Expressions (+ - * / %, comparisons, && || !, ? :) read it; set, reset, reply and open run when the user presses a button. There are no loops, functions or host access.
• The component catalog is one Effect schema per component. Its props order is the argument order, its types check programs, and the genui skill prints typed signatures from it, e.g. Metric(label: string, value: string | number, detail?: string, delta?: number, trend?: number[], inverse?: boolean).
• The plugin ships the genui skill whenever it is enabled. Besides the syntax, components and examples, it keeps generated UI rare: text is the default, a surface must clearly beat text, a list names what never gets one, and an answer gets at most one. It also tells the agent to gather data before rendering, to end the turn after a surface that asks for a choice, and what a reply looks like. Its description says to load it before the first genui_render call. The skill description, the tool description and the in-app reminder also say that text is the default, so the agent knows before it loads the skill.

What the agent gets back for a broken program:

count (line 3): Metric needs value: Metric(label: string, value: string | number, detail?: string, ...)
chart (line 4): BarChartt is not a component. Did you mean BarChart?
root (line 1): Surface.children takes Block | Columns | Section, not Row; Row goes in Group
spare (line 6): spare is defined but nothing under root uses it, so it does not render

Why our own language

The syntax follows OpenUI Lang, but we dropped @openuidev/lang-core:

Changes

• core: tool input deltas, as above, with tests for batching and ordering.
• @opencode/plugin-genui:
  • lang/: lexer, incremental parser, evaluator and checks, generic over a catalog.
  • catalog.ts: components and examples.
  • guide.ts: the genui skill.
  • tools.ts: genui_render.
  • session.ts: app-only offering and the reminder.
  • ./genui, ./lang and ./catalog are browser-safe entry points.
• @opencode/plugin-browser: a built-in browser skill: explicit tab IDs, snapshots and refs, showing results to the user, recipes for debugging, performance and memory, recovery after failures, and every operation from Browser.Operations. It is gated like the browser tools: hidden until a desktop browser attaches to the session.
• session-ui:
  • genui/ renders surfaces, charts, state and actions.
  • Only genui_render reads arguments while they stream; every other tool still waits for complete input.
  • DataProvider gains onPrompt for replies.
• app:
  • Settings > Experimental > Generated UI, on by default. The Experimental page shows again now that it has a setting.
  • Prompts and surface replies carry metadata.genui from that setting.
  • Wires onPrompt.

Experimental settings

Components

• Everything draws with @opencode/ui and --v2-* tokens. One series uses the text color; several use --genui-series-1..6 from v2 hue steps; state colors mean outcomes only.

Follow-ups

• Open a surface in the side panel as a tab. The earlier side-panel prototype is removed.
• Chart number formatting still uses en-US; it should follow the app locale.