feat: render inline web chat widgets via capability-gated show_widget tool#101840
Conversation
openclaw-barnacle
Bot
added
docs
|
Codex review: needs changes before merge. Reviewed July 9, 2026, 5:18 AM ET / 09:18 UTC. Summary PR surface: Source +446, Tests +474, Docs +66. Total +986 across 59 files. Reproducibility: not applicable. this is a feature PR rather than a broken existing behavior report. Source inspection confirms current main lacks the requested capability and the PR head adds it. Review metrics: 3 noteworthy metrics.
Stored data model Root-cause cluster Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge readiness Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch. Rank-up moves:
Mantis proof suggestion Risk before merge
Maintainer options:
Copy recommended automerge instructionNext step before merge
Maintainer decision needed
Security Review findings
Review detailsBest possible solution: Land only after the mechanical blockers are repaired, exact-head CI is green, real web-chat proof remains attached, and maintainers explicitly accept the plugin capability API plus scripted widget sandbox boundary. Do we have a high-confidence way to reproduce the issue? Not applicable: this is a feature PR rather than a broken existing behavior report. Source inspection confirms current main lacks the requested capability and the PR head adds it. Is this the best way to solve the issue? Mostly yes, but not merge-ready: a generic client-capability seam plus Canvas-owned rendering is a cleaner fit than channel gating or raw DOM rendering. The current branch still needs the retention, type, and extension-boundary fixes plus maintainer acceptance of the API/security direction. Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 0de5d37f92fc. Label changesLabel changes:
Label justifications:
Evidence reviewedPR surface: Source +446, Tests +474, Docs +66. Total +986 across 59 files. View PR surface stats
Acceptance criteria:
What I checked:
Likely related people:
What the crustacean ranks mean
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics. How this review workflow works
Review history (3 earlier review cycles)
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: af2a54f8ef
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| .toSorted( | ||
| (left, right) => | ||
| left.createdAt.localeCompare(right.createdAt) || left.id.localeCompare(right.id), | ||
| ) | ||
| .slice(0, deleteCount); |
There was a problem hiding this comment.
When a session creates widgets fast enough that the new manifest gets the same millisecond createdAt as existing widgets after the 32-document cap is exceeded, this sort can select the just-created random id as one of the “oldest” entries and the fs.rm below deletes it before createCanvasDocument returns. That leaves show_widget returning a preview URL whose index.html was already removed, so the inline widget renders as missing/404; exclude the current id from pruning or prune before writing the new document.
Useful? React with 👍 / 👎.
clawsweeper
Bot
added
rating: 🦐 gold shrimp
af2a54f to
2ab460c
Compare
Live end-to-end proof (Blacksmith Testbox, real gpt-5.5)Ran the full user path on a Testbox lease: built this branch, started a real gateway with a live OpenAI key, drove the Control UI with headless Chromium (connect → prompt → tool call → render), and screenshot-verified the result. The model was asked to render a "Deploy status" dashboard; it called
Verified live, not just in unit seams:
The E2E loop surfaced and fixed four real gaps that unit tests missed (now covered by regression tests in this PR):
Known limitation (documented on the docs page, follow-up filed): CLI-backed model backends do not yet transport client capabilities, so gated tools stay unavailable there (fail closed) — #102577. |
clawsweeper
Bot
added
proof: sufficient
… tool Adds client-capability-gated tool availability: gateway clients declare capabilities at connect (new inline-widgets cap), chat.send stamps them into the run context, and every tool assembly path (embedded runner, queued followups, Codex app-server harness, plugin-only construction plans) drops tools whose requiredClientCaps the originating client did not declare. The Canvas plugin ships the first such tool, show_widget: agents pass SVG or an HTML fragment plus a title; the plugin hosts it as a bounded, retention-scoped Canvas document and returns the existing canvas preview handle, which web chat renders as a sandboxed iframe fitted to the widget's reported content height. Widget frames never get allow-same-origin (per-preview sandbox ceiling, including the sidebar path) and the Canvas host serves widget documents with a CSP sandbox header so direct navigation runs in an opaque origin. Verified live end-to-end on a Testbox with gpt-5.5 (screenshots on the PR). CLI-backed model backends do not carry client caps yet and stay fail-closed (#102577). Closes #101790
2ab460c to
b793955
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b7939558ab
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return jsonResult({ | ||
| kind: "canvas", | ||
| presentation: { target: "assistant_message", title, sandbox: "scripts" }, | ||
| view: { id: document.id, url: document.entryUrl }, | ||
| text: `Widget hosted at ${document.entryUrl}`, |
There was a problem hiding this comment.
When show_widget runs in the live Control UI, this result is delivered as a tool-stream/tool-result JSON text payload; the live renderer extracts card.preview from that text, but renderToolPreview(..., "chat_tool") deliberately returns nothing, and only chat.history later lifts the canvas preview onto an assistant message. That means the new widget does not render inline during the run as documented, and users only see raw details/sidebar until a history reload; emit or project a live assistant canvas block for this result path.
Useful? React with 👍 / 👎.
… tool (openclaw#101840) Adds client-capability-gated tool availability: gateway clients declare capabilities at connect (new inline-widgets cap), chat.send stamps them into the run context, and every tool assembly path (embedded runner, queued followups, Codex app-server harness, plugin-only construction plans) drops tools whose requiredClientCaps the originating client did not declare. The Canvas plugin ships the first such tool, show_widget: agents pass SVG or an HTML fragment plus a title; the plugin hosts it as a bounded, retention-scoped Canvas document and returns the existing canvas preview handle, which web chat renders as a sandboxed iframe fitted to the widget's reported content height. Widget frames never get allow-same-origin (per-preview sandbox ceiling, including the sidebar path) and the Canvas host serves widget documents with a CSP sandbox header so direct navigation runs in an opaque origin. Verified live end-to-end on a Testbox with gpt-5.5 (screenshots on the PR). CLI-backed model backends do not carry client caps yet and stay fail-closed (openclaw#102577). Closes openclaw#101790
1 participant
Closes #101790
What Problem This Solves
Agents in the web chat can only answer in text and markdown. For content that is inherently visual — flow diagrams, small dashboards, comparisons, interactive explainers — a rendered inline widget communicates far better, and OpenClaw already ships the rendering substrate (the Canvas plugin's sandboxed embed pipeline). What was missing is a first-class tool for "render this markup inline in the chat transcript," and the underlying capability such a tool needs: a way to offer a tool only where the user's client can actually render its output. Today every gateway client (web UI, iOS, Android, macOS, CLI) collapses to the same internal message channel at tool-assembly time, so a web-only tool would have appeared — and dead-ended — on phones and terminals too.
Why This Change Was Made
Two parts, deliberately layered:
Client-capability-gated tool availability (generic core seam). Gateway clients already declare capability flags at connect (
GATEWAY_CLIENT_CAPS, previously onlytool-events). This adds aninline-widgetscapability that the browser Control UI declares, stamps the connected client's caps into the per-run message context atchat.senddispatch (mirroringGatewayClientScopes), and threads them through the run/queue/compaction paths into tool assembly. Tools — core or plugin — can declarerequiredClientCaps;createOpenClawToolsdrops any tool whose required caps the originating client did not declare. Channel runs (Telegram, WhatsApp, …) have no gateway client and therefore no caps, so gated tools are simply absent there, andtools.allowcannot resurrect them (capability is a hard availability fact, not policy). Queue collect-mode batching keys on caps so runs from different surfaces don't merge. Additive protocol only: one new capability string, no version bump, no new config keys.show_widgettool (Canvas plugin). The agent passes atitlepluswidget_code(<svg …>or an HTML fragment, ≤256 KiB). The plugin wraps it in a minimal HTML shell and persists it through the existing Canvas document store — bounded to 32 documents per session scope, oldest evicted — and returns the canvas preview handle the web chat already knows how to render, so the widget appears inline from the tool call itself and survives history reload with zero new UI rendering machinery.Security model (hardened through two structured-review cycles):
allow-same-origin: the preview carries a sandbox ceiling that caps the operator's globalembedSandboxmode, enforced on both the inline preview and the sidebar open path.Content-Security-Policy: sandbox allow-scriptsresponse header, so navigating to the hosted URL directly also runs the widget in an opaque origin rather than the Control UI origin.default-src 'none'; inline style/script;data:images) blocks external fetches and resource loads; the Control UI page CSP (default-src 'self', noframe-src) additionally restricts where embedded frames can navigate. Scripts remain allowed by design — widgets are interactive — with the residual self-navigation limitation documented on the docs page.User Impact
Agents talking to the web Control UI gain a
show_widgettool and can answer with rendered, optionally interactive, self-contained widgets inline in the chat. Users on Telegram, WhatsApp, the CLI, or the native apps see no change: the tool does not exist for those runs, so agents cannot call it into a void. Plugin authors get a documented, generic way to ship surface-conditional tools (requiredClientCaps), and native apps can adopt the widget surface later by declaring the capability. No configuration changes are required; the existinggateway.controlUi.embedSandboxtiers keep applying, with widgets capped belowtrusted.Evidence
src/agents/openclaw-tools.client-caps.test.ts(absent caps exclude, superset includes, channel runs exclude,tools.allowcannot resurrect),src/auto-reply/reply/queue/drain.client-caps.test.ts(collect-mode batching keys on caps),src/gateway/server-methods/chat.directive-tags.test.ts(ctx stamping; 270 tests),src/plugins/tool-descriptor-cache.test.ts(caps survive descriptor caching).extensions/canvas/src/widget-tool.test.ts(validation, SVG/HTML wrapping with CSP meta, opaque ids, per-scope eviction, manifest sandbox marker),extensions/canvas/src/host/server.test.ts(sandbox-marked documents get the CSP sandbox header and skip live reload; unmarked documents unchanged).ui/src/api/gateway.node.test.ts(Control UI declaresinline-widgets),ui/src/lib/chat/tool-display.node.test.ts(sandbox ceiling math),ui/src/pages/chat/components/chat-tool-cards.node.test.ts(canvas handle extraction incl. sandbox; sidebar conversion preserves the ceiling),ui/src/lib/chat/message-normalizer.test.ts.pnpm buildgreen with no[INEFFECTIVE_DYNAMIC_IMPORT]; docs map/MDX/link checks green; oxfmt/oxlint clean on touched files;git diff --checkclean.trustedmode; hosted widget documents executing as the app origin when opened directly), final review reported no actionable findings.