Skip to content

[FEATURE]: TUI plugin API to render fenced code blocks in session content #54126

Closed as not planned

Description

@koladev32

Feature hasn't been suggested before.

  • I have verified this feature I'm about to request hasn't been suggested before.

Describe the enhancement you want to request

Let TUI plugins register a renderer for a fenced code block language, so assistant replies can show rich content (diagrams, charts, math, 3D) in the conversation instead of raw source.

Proposal

api.markdown.codeBlock.register(lang, (block, ctx) => Renderable | undefined): () => void

block = { lang, text, complete, key, sessionID, messageID }
// complete: closing fence received (false while streaming)
// key: stable per block (message part + position) so plugins can keep state across streaming updates
// returning undefined falls back to the normal code block

It builds on what's already there: OpenTUI's Markdown.renderNode and createMarkdownCodeBlockRenderer. Changes are small and follow existing patterns:

  • @opencode-ai/plugin/tui: add markdown to TuiPluginApi
  • tui/plugin/runtime.tsx: a code-block registry, same shape as the plugin routes registry
  • opencode/plugin/tui/runtime.ts: wrap registrations with scope.track() so they're removed when the plugin is disabled or unloaded
  • routes/session/index.tsx (TextPart): build renderNode from the registry

Why implement this

  • V2 TUI: add native Mermaid and LaTeX rendering #40508 asks for "the minimum plugin API needed for internal plugins to recognize and render custom session content" (Mermaid, LaTeX). This would be that extension point, public, so built-in and third-party plugins use the same mechanism.
  • Coding agents are used more and more to explain code and systems. Diagrams are far better than ASCII art, and today the TUI can only show the source.
  • It keeps renderers out of core: d2, Mermaid, LaTeX, charts and so on can live in plugins with their own dependencies.

Prototype

I built a quick prototype on a fork to validate the idea: https://github.com/koladev32/opencode/tree/inline-diagrams (work in progress, rendering code still in core; it would move to a plugin on top of this API).

  • d2 blocks render as themed diagrams while the reply streams, with step-by-step animation
  • scene3d blocks (YAML parts, never code) render as 3D exploded views (three.js + WebGPU, headless) that you can rotate
  • Display uses Kitty graphics Unicode placeholders (works in Ghostty, and through tmux)

Here is a link to a demo I've done: https://x.com/koladev32/status/2108224214628991334

Design considerations

Security (renderers run on model output, i.e. untrusted input):

  • core catches renderer errors and falls back to the code block (OpenTUI's Markdown doesn't guard custom renderers)
  • copy still copies the original source; the existing "Disable code concealment" toggle disables renderers and shows the raw source
  • optionally reserve shell languages (sh, bash, …) so a plugin can't display something different from the command a user might copy and run
  • conflicts: last registration wins (like routes), with a warning

Performance:

  • no registered renderers means no renderNode at all, so it's identical to today
  • O(1) lookup by language; the code-block-only flag is kept so other Markdown elements skip the hook
  • Markdown only re-invokes a renderer when that block's text changed; complete and key let plugins avoid work while streaming

Terminal support:

  • The API is rendering-agnostic: a plugin returns an OpenTUI Renderable, so it decides how to draw based on the terminal's capabilities (api.renderer.capabilities).
  • Graphics degrade through OpenTUI's ImageRenderable: Kitty graphics → Sixel → Unicode half-block characters (any truecolor terminal), so images still show in terminals without a graphics protocol, at lower fidelity.
  • Plugins can also render text only (e.g. box-drawing characters) or return undefined to keep the normal code block; that is also the behavior on any renderer error.
  • My prototype uses Kitty Unicode placeholders where available (flicker-free animation, works through tmux) and falls back to ImageRenderable elsewhere.

Happy to open a PR if the direction looks right.

Activity

  1. opencode-agent commented on Oct 9, 2026

    @opencode-agent
    Contributor

    Thanks for the detailed proposal and the prototype. The d2 and 3D demos look great.

    The file paths in the proposal (@opencode-ai/plugin/tui, tui/plugin/runtime.tsx, opencode/plugin/tui/runtime.ts) come from OpenCode v1 (the opencode-ai package, 0.x/1.x). v1 now only gets fixes for critical bugs, so new plugin APIs won't be added there.

    The good news is that v2 (@opencode/cli, 2.x) already has this extension point. The TUI plugin context includes:

    context.markdown.registerCodeBlockRenderer(language: string, render: MarkdownCodeBlockRenderer): () => void
    • render is OpenTUI's MarkdownCodeBlockRenderer. Registrations belong to the plugin and are removed when it is disabled or unloaded.
    • If no renderer is registered, renderNode isn't set at all, so Markdown renders as it does without plugins.
    • v2 rejects a second registration for the same language (it throws) instead of letting the last one win.
    • The built-in Mermaid support for V2 TUI: add native Mermaid and LaTeX rendering #40508 uses this same API. See packages/merman/src/plugin.ts on the v2 branch for an example:
    import { Plugin } from "@opencode/plugin/tui"
    
    export default Plugin.define({
      id: "my.d2",
      setup(context) {
        context.markdown.registerCodeBlockRenderer("d2", (token, ctx) => /* Renderable | undefined */ undefined)
      },
    })

    Your d2 and scene3d renderers could probably be ported to a v2 plugin on top of this. If you then need something the v2 API lacks, such as a stable per-block key or reserved shell languages, please open a new issue against v2 with those specifics.

    Related:

    If you think I've got this wrong and it does matter for v2 or is critical, please reply here.

  2. koladev32 commented on Oct 9, 2026

    @koladev32
    Author

    Thank you for the reply. It helps. Closing this issue.

  3. opencode-agent commented on Oct 9, 2026

    @opencode-agent
    Contributor

    Thanks for confirming, @koladev32, and for the detailed proposal. If you port your d2 or scene3d renderers to a v2 plugin and find something registerCodeBlockRenderer can't do, please open a new v2 issue with the details.

  4. opencode-agent commented on Oct 10, 2026

    @opencode-agent
    Contributor

    Closing as not planned, as discussed. This proposal targeted v1, which only gets critical fixes, and v2 already has the requested extension point: context.markdown.registerCodeBlockRenderer. Thanks again, @koladev32. If a v2 plugin for d2 or scene3d needs something the API doesn't offer yet, please open a new v2 issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions