Summary

Replacement for #80229, carrying forward @100yenadmin's work from #75578, #75581, #75588, and the relevant pieces of #79334 on an OpenClaw-owned branch so maintainers can keep fixing it up. History is rewritten into three review commits while preserving contributor authorship/credit on the feature commits.

Commit layout:

• feat(plugin-sdk): consolidate session workflow APIs
• feat(media-understanding): add structured extraction runtime
• docs(plugin-sdk): document consolidated workflow seams

This consolidates several host workflow seams behind grouped Plugin SDK namespaces:

• api.session.controls.* for Control UI descriptors and typed session actions.
• api.session.workflow.* for next-turn injections, host-mediated attachments, Cron-backed scheduled turns, and tag cleanup.
• api.session.state.* for plugin-owned session metadata projected into session rows.
• api.agent.events.* for sanitized plugin-attributed agent events.
• api.runContext.* for JSON-compatible run-scoped scratch state.
• api.lifecycle.* for plugin-owned cleanup hooks.

The old flat methods remain available, but are now marked deprecated inline and point to the grouped API. The injected grouped namespaces are required on OpenClawPluginApi, not optional.

What this enables

1. Register a typed session action

A bundled plugin can expose a typed action that Gateway clients or Control UI can invoke later. The action owns its payload schema and required operator scope.

export default function register(api: OpenClawPluginApi) {
  api.session.controls.registerSessionAction({
    id: "approve-deploy",
    description: "Approve the current deployment workflow",
    requiredScopes: ["operator.approvals"],
    schema: {
      type: "object",
      additionalProperties: false,
      required: ["environment"],
      properties: {
        environment: { type: "string" },
      },
    },
    handler: async ({ payload, sessionKey, client }) => {
      api.agent.events.emitAgentEvent({
        runId: "deploy-run",
        sessionKey,
        stream: "approval",
        data: {
          state: "approved",
          environment: payload?.environment,
          scopes: client?.scopes ?? [],
        },
      });

      return {
        result: { accepted: true, environment: payload?.environment },
        reply: { text: `Approved ${payload?.environment}` },
        continueAgent: true,
      };
    },
  });
}

A Gateway caller dispatches it through the new RPC:

{
  "type": "req",
  "id": "approve-1",
  "method": "plugins.sessionAction",
  "params": {
    "pluginId": "deploy-plugin",
    "actionId": "approve-deploy",
    "sessionKey": "agent:main:main",
    "payload": { "environment": "prod" }
  }
}

The Gateway now authorizes dynamic plugins.sessionAction calls against the registered action scope before dispatch. In the example above, operator.approvals is required; operator.write alone is rejected.

2. Publish Control UI descriptors next to the action

Plugins can describe small UI contributions without coupling Control UI to plugin internals.

api.session.controls.registerControlUiDescriptor({
  id: "deploy-approval-panel",
  title: "Deploy approval",
  surface: "session",
  actions: [
    {
      id: "approve-prod",
      label: "Approve prod",
      sessionAction: {
        pluginId: api.id,
        actionId: "approve-deploy",
        payload: { environment: "prod" },
      },
    },
  ],
});

Clients can discover descriptors through plugins.uiDescriptors and then call plugins.sessionAction with the descriptor action payload.

3. Send files back through the active session route

A bundled plugin can attach validated local files to the active session's direct outbound channel. This is useful for generated reports, screenshots, exports, or diagnostics.

const sent = await api.session.workflow.sendSessionAttachment({
  sessionKey: "agent:main:main",
  files: [
    {
      path: reportPath,
      fileName: "deployment-report.txt",
      mime: "text/plain",
    },
  ],
  caption: "Deployment report",
  captionFormat: "plain",
  channelHints: {
    telegram: { disableNotification: true, parseMode: "HTML" },
    slack: { threadTs: "1700000000.000100" },
  },
});

if (!sent.ok) {
  api.logger.warn(`Could not send attachment: ${sent.reason}`);
}

The host validates file paths and MIME/content, resolves the session delivery context, and sends via the channel-owned outbound path. Telegram-specific hints stay in the Telegram delivery layer; the core API stays generic.

4. Schedule and later clean up a follow-up agent turn

Plugins can schedule a future turn in a session without inventing a separate task system. Cron remains the scheduler and task ledger owner.

const handle = await api.session.workflow.scheduleSessionTurn({
  sessionKey: "agent:main:main",
  message: "Check whether the deployment finished and summarize the result.",
  delayMs: 10 * 60 * 1000,
  tag: "deploy-prod",
  name: "post-deploy-check",
  deliveryMode: "announce",
});

api.logger.info(`scheduled follow-up: ${handle?.id ?? "none"}`);

If the workflow is cancelled, the plugin can remove all matching follow-ups by tag:

await api.session.workflow.unscheduleSessionTurnsByTag({
  sessionKey: "agent:main:main",
  tag: "deploy-prod",
});

Generated Cron names include plugin ownership, tag, session key, and stable job name metadata, so cleanup can target only that plugin's scheduled session turns.

5. Keep per-run plugin state and emit workflow events

Plugins can store JSON-compatible state for the active run and emit sanitized events for UI/workflow subscribers.

api.runContext.setRunContext({
  runId,
  namespace: "deploy",
  value: { environment: "prod", step: "waiting-for-approval" },
});

const state = api.runContext.getRunContext({ runId, namespace: "deploy" });

api.agent.events.emitAgentEvent({
  runId,
  sessionKey: "agent:main:main",
  stream: "deploy-plugin.workflow",
  data: { state, event: "approval-requested" },
});

Workspace plugins are kept scoped to their own event stream names. Reserved streams stay bundled-only.

6. Run bounded image-first structured extraction

Trusted plugins can use provider-owned media-understanding runtimes for image-first extraction with optional supplemental text context.

const receipt = await api.runtime.mediaUnderstanding.extractStructuredWithModel({
  provider: "codex",
  model: "gpt-5.5",
  input: [
    {
      type: "image",
      buffer: receiptPng,
      mime: "image/png",
      fileName: "receipt.png",
    },
    {
      type: "text",
      text: "The image is a vendor receipt. Extract totals only.",
    },
  ],
  instructions: "Extract merchant, date, currency, subtotal, tax, and total.",
  schemaName: "ReceiptFields",
  jsonSchema: {
    type: "object",
    additionalProperties: false,
    required: ["merchant", "total"],
    properties: {
      merchant: { type: "string" },
      total: { type: "number" },
      currency: { type: "string" },
    },
  },
});

api.logger.info(JSON.stringify(receipt.parsed));

The Codex provider checks model modality support, denies native tool/file approvals, requires at least one image input, parses JSON-mode results, and validates parsed JSON against the provided schema.

Maintainer fixups in this replacement

• Rebased on current origin/main and refreshed the Plugin SDK API baseline hash.
• Removed the plugin-side data result alias; session actions now use the same result name as the Gateway wire response.
• Tightened plugins.sessionAction operator auth to the exact registered action scopes before dispatch.
• Required grouped SDK facades on the injected API type instead of making them optional.
• Clarified structured extraction as image-first in docs and inline types.
• Deduped the changelog entry into the active release section.
• Kept the old flat APIs as deprecated compatibility aliases with inline migration docs.

Real behavior proof

• Behavior or issue addressed: [plugin sdk] consolidate host workflow seams #80267 keeps the rewritten three-commit review stack, dogfoods the namespaced SDK APIs in internal fixtures/snippets, validates plugin session-action result envelopes with the existing TypeBox/Ajv protocol schema instead of hand-rolled key checks, factors the duplicated Codex app-server media-understanding turn lifecycle into one bounded runner, splits host workflow helpers into attachment-delivery and scheduled-turn modules, shrinks repeated session action/attachment/scheduled-turn contract fixture scaffolding, preserves explicit Telegram HTML before chunking, preserves Telegram HTML mode for pre-rendered outbound text chunks, fixes the Telegram explicit-formatting regression, fixes Swift protocol generator idempotency, prevents plugin session action handlers from mutating the live gateway client scope array, and keeps runtime agent event emission callable after plugin registration closes. Raw Telegram direct sends still default to Markdown; explicit formatting.parseMode: "HTML" uses Telegram HTML splitting instead of Markdown escaping before delivery; chunker-rendered Telegram text carries formatting.parseMode: "HTML" into sendText; protocol:gen:swift no longer emits trailing spaces; ctx.client.scopes is now a defensive copy; captured api.emitAgentEvent and api.agent.events.emitAgentEvent remain late-callable.

• Real environment tested: local OpenClaw checkout at 4f6453224d0990be8c9bbd3ace4c1d7fde75ad0d, pushed to openclaw/openclaw PR [plugin sdk] consolidate host workflow seams #80267.

• Exact steps or command run after this patch: pnpm test src/plugins/contracts/session-actions.contract.test.ts extensions/codex/media-understanding-provider.test.ts; pnpm test src/plugins/contracts/session-actions.contract.test.ts src/plugins/contracts/session-attachments.contract.test.ts src/plugins/contracts/scheduled-turns.contract.test.ts src/plugins/captured-registration.test.ts src/plugins/loader.test.ts; pnpm test src/plugins/contracts/session-attachments.contract.test.ts src/plugins/contracts/scheduled-turns.contract.test.ts; env -u OPENCLAW_TESTBOX -u OPENCLAW_TESTBOX_REMOTE_RUN -u OPENCLAW_TESTBOX_ID pnpm check:changed; git diff --check; pnpm test src/plugins/contracts/host-hooks.contract.test.ts src/plugins/contracts/run-context-lifecycle.contract.test.ts src/plugins/contracts/scheduled-turns.contract.test.ts src/plugins/captured-registration.test.ts src/plugins/loader.test.ts; pnpm test extensions/telegram/src/telegram-outbound.test.ts extensions/telegram/src/channel.gateway.test.ts extensions/telegram/src/outbound-adapter.test.ts src/infra/outbound/message-plan.test.ts src/infra/outbound/deliver.test.ts; pnpm protocol:check; git log --oneline origin/main..HEAD; gh api repos/openclaw/openclaw/pulls/80267 --jq '{commits, head: .head.sha, additions, deletions, changed_files}'.

• Evidence after fix:

  $ pnpm test src/plugins/contracts/session-actions.contract.test.ts extensions/codex/media-understanding-provider.test.ts
  Test Files  2 passed (2)
  Tests  19 passed (19)
  
  $ pnpm test src/plugins/contracts/session-actions.contract.test.ts src/plugins/contracts/session-attachments.contract.test.ts src/plugins/contracts/scheduled-turns.contract.test.ts src/plugins/captured-registration.test.ts src/plugins/loader.test.ts
  Test Files  5 passed (5)
  Tests  177 passed (177)
  
  $ pnpm test src/plugins/contracts/session-attachments.contract.test.ts src/plugins/contracts/scheduled-turns.contract.test.ts
  Test Files  2 passed (2)
  Tests  40 passed (40)
  
  $ env -u OPENCLAW_TESTBOX -u OPENCLAW_TESTBOX_REMOTE_RUN -u OPENCLAW_TESTBOX_ID pnpm check:changed
  check:changed completed successfully
  
  $ git diff --check
  # no output
  
  $ pnpm test src/plugins/contracts/host-hooks.contract.test.ts src/plugins/contracts/run-context-lifecycle.contract.test.ts src/plugins/contracts/scheduled-turns.contract.test.ts src/plugins/captured-registration.test.ts src/plugins/loader.test.ts
  Test Files  5 passed (5)
  Tests  215 passed (215)
  
  $ pnpm test extensions/telegram/src/telegram-outbound.test.ts extensions/telegram/src/channel.gateway.test.ts extensions/telegram/src/outbound-adapter.test.ts src/infra/outbound/message-plan.test.ts src/infra/outbound/deliver.test.ts
  Test Files  5 passed (5)
  Tests  101 passed (101)
  
  $ pnpm protocol:check
  wrote dist/protocol.schema.json
  wrote apps/shared/OpenClawKit/Sources/OpenClawProtocol/GatewayModels.swift
  
  $ git log --oneline origin/main..HEAD
  4f6453224d docs(plugin-sdk): document consolidated workflow seams
  e4d2a63521 feat(media-understanding): add structured extraction runtime
  ed2216955c feat(plugin-sdk): consolidate session workflow APIs
  
  $ gh api repos/openclaw/openclaw/pulls/80267 --jq '{commits, head: .head.sha, additions, deletions, changed_files}'
  {"additions":7231,"changed_files":91,"commits":3,"deletions":167,"head":"4f6453224d0990be8c9bbd3ace4c1d7fde75ad0d"}
  

• Observed result after fix: plugin session-action result envelopes now reuse the protocol schema validator and keep JSON-compatible payload checks, Codex image description and structured extraction share one bounded app-server runner, internal host-hook and scheduler fixtures use namespaced SDK APIs while deprecated flat API coverage remains elsewhere, explicit Telegram HTML is split as HTML before delivery instead of being Markdown-escaped, Telegram chunker-rendered text keeps HTML mode through outbound delivery while raw direct sends keep Markdown defaults, late captured root and namespaced agent event emission bypass the closed registration guard, session action handlers receive a mutable copy without changing client.connect.scopes, protocol generation is idempotent, repeated contract-test setup is smaller while preserving coverage, host workflow helpers are split by attachment-vs-scheduler ownership, the review stack remains three commits, and the PR head matches the pushed branch.

• What was not tested: no additional live Telegram/Codex model call was run after these focused review fixes; previous broad and focused local verification is listed below.

Verification

• pnpm test src/plugins/contracts/session-actions.contract.test.ts extensions/codex/media-understanding-provider.test.ts after TypeBox-backed session-action result validation and shared Codex media-understanding runner refactor
• pnpm test src/plugins/contracts/session-actions.contract.test.ts src/plugins/contracts/session-attachments.contract.test.ts src/plugins/contracts/scheduled-turns.contract.test.ts src/plugins/captured-registration.test.ts src/plugins/loader.test.ts after the host workflow split and registry loaded-plugin helper refactor
• pnpm test src/plugins/contracts/session-attachments.contract.test.ts src/plugins/contracts/scheduled-turns.contract.test.ts after the contract fixture LOC-reduction refactor
• env -u OPENCLAW_TESTBOX -u OPENCLAW_TESTBOX_REMOTE_RUN -u OPENCLAW_TESTBOX_ID pnpm check:changed after the host workflow split
• pnpm build after the host workflow module-boundary split
• env -u OPENCLAW_TESTBOX -u OPENCLAW_TESTBOX_REMOTE_RUN -u OPENCLAW_TESTBOX_ID pnpm check:changed after the previous LOC-reduction refactor
• pnpm install after rebasing onto current origin/main dependency changes
• pnpm test src/gateway/method-scopes.test.ts src/plugins/contracts/session-actions.contract.test.ts src/plugins/contracts/session-attachments.contract.test.ts src/plugins/contracts/scheduled-turns.contract.test.ts src/media-understanding/runtime.test.ts extensions/codex/media-understanding-provider.test.ts extensions/telegram/src/outbound-adapter.test.ts src/agents/tools/gateway.test.ts
• pnpm test extensions/telegram/src/channel.gateway.test.ts extensions/telegram/src/outbound-adapter.test.ts after the Telegram formatting review fix
• pnpm protocol:check after the Swift generator idempotency review fix
• pnpm test src/plugins/contracts/session-actions.contract.test.ts after the session action scope-copy review fix
• pnpm test src/plugins/loader.test.ts src/plugins/contracts/session-actions.contract.test.ts after the late event emission policy review fix
• pnpm test src/infra/outbound/message-plan.test.ts src/infra/outbound/deliver.test.ts extensions/telegram/src/telegram-outbound.test.ts extensions/telegram/src/channel.gateway.test.ts extensions/telegram/src/outbound-adapter.test.ts after the Telegram chunked-text HTML-mode review fix
• pnpm test extensions/telegram/src/telegram-outbound.test.ts extensions/telegram/src/channel.gateway.test.ts extensions/telegram/src/outbound-adapter.test.ts src/infra/outbound/message-plan.test.ts src/infra/outbound/deliver.test.ts after the explicit Telegram HTML chunking review fix
• pnpm test src/plugins/contracts/host-hooks.contract.test.ts src/plugins/contracts/run-context-lifecycle.contract.test.ts src/plugins/contracts/scheduled-turns.contract.test.ts src/plugins/captured-registration.test.ts src/plugins/loader.test.ts after namespaced internal fixture migration
• pnpm check:docs
• pnpm plugin-sdk:api:check
• git diff --check
• env -u OPENCLAW_TESTBOX -u OPENCLAW_TESTBOX_REMOTE_RUN -u OPENCLAW_TESTBOX_ID pnpm check:changed

SwiftLint in check:changed still reports the pre-existing non-serious type-body-length warnings in apps/macos/Sources/OpenClaw/TalkModeRuntime.swift and apps/macos/Sources/OpenClaw/NodeMode/MacNodeRuntime.swift; the command exits green.