Summary

Downstream plugins need a generic OpenClaw SDK/runtime hook for bounded, OAuth-backed structured extraction.

The immediate motivation came from a downstream GBrain/OpenClaw integration that originally added a GBrain-specific route inside the Codex extension. That route worked as a bridge, but it is the wrong long-term boundary: OpenClaw should expose a generic host capability, and product plugins should own their own schemas/routes.

OpenClaw main already has a good Codex media-understanding provider for image description via the app-server/OAuth path. The remaining gap is a host-agnostic structured extraction surface that plugins can call for text/image JSON extraction without taking a dependency on user API keys or embedding product-specific route names in core.

Desired SDK shape

One of these would work:

• a new StructuredExtractionProvider contract, or
• an extension to the existing media-understanding provider contract for structured text+image extraction.

The exact API can be bikeshed, but the capability should let a plugin request:

• text-only extraction
• image extraction with bytes or data URLs
• optional JSON schema / JSON-mode response expectations
• bounded timeout
• model/profile selection through the host runtime
• no tools, no file edits, no approval grants
• ephemeral execution
• controlled success/error envelopes

Non-goals

This should not add GBrain-specific routes or schema names to OpenClaw core.

For example, a downstream plugin may expose /plugins/gbrain/extract and return gbrain.media-extraction.v1, but OpenClaw should only provide the generic OAuth-backed extraction capability that the plugin consumes.

Why this matters

Many plugins need enrichment/extraction, but should not require users to configure separate model API keys when OpenClaw already has a logged-in Codex/OpenAI runtime.

A generic hook keeps the platform clean:

flowchart LR
  Plugin[Product plugin] --> SDK[OpenClaw SDK extraction hook]
  SDK --> Codex[Codex app-server / OAuth runtime]
  Codex --> Result[Structured JSON or controlled error]
  Result --> Plugin

Acceptance criteria

• A plugin can call the provider without OPENAI_API_KEY or another user model API key.
• The provider uses existing OpenClaw/Codex auth/runtime plumbing.
• Requests can include text and image inputs.
• Responses can be constrained to structured JSON or return a controlled parse/validation error.
• The turn is bounded: timeout, ephemeral state, no dynamic tools, read-only/no file mutation, and approval requests denied.
• Secrets, OAuth tokens, refresh tokens, and raw internal errors are not returned or logged.
• Provider metadata advertises capabilities so plugins can detect availability.
• Tests cover no-tools/no-approval behavior, timeout, structured error handling, and the no-user-key path with a mocked app-server runtime.

Prior art

OpenClaw already has a Codex media-understanding provider on main, which is close to the desired runtime posture for images. This issue is about making the structured extraction use case available as a generic plugin-consumable SDK capability rather than a product-specific route in the Codex extension.