Summary

OpenClaw should support forwarding non-image attachments as native provider file/document inputs when the selected model/backend supports them, instead of only treating those files as local workspace artifacts, managed media links, or tool-readable paths.

Problem to solve

OpenClaw currently appears to treat images as first-class multimodal model inputs, but non-image files are not handled the same way.

In practice, files such as PDFs, DOCX, XLSX, CSVs, slide decks, forms, scans, and other document-like inputs are either staged locally, exposed as attachment references, routed through tools, extracted client-side, or in some paths dropped/ignored rather than sent to the underlying model as native file input.

This is insufficient because many modern model providers already support native file/document inputs. Users reasonably expect OpenClaw to preserve that capability when using those providers. Instead, OpenClaw’s current behavior forces the agent or client to perform local extraction, OCR, page rendering, or manual preprocessing.

That fallback is much worse for many real-world files. Text extraction loses layout and table structure. Local OCR can perform poorly on scans, blurry documents, forms, handwriting, receipts, archival files, and image-heavy documents. Rendering pages to images is expensive and lossy. Requiring the agent to discover and inspect local file paths also makes behavior inconsistent: the agent may choose the wrong tool, fail to inspect the file, hallucinate about missing content, or spend unnecessary time/tokens trying to recover information the model provider could have handled natively.

The issue is not limited to PDFs. The broader problem is that OpenClaw lacks a provider-agnostic file/document content abstraction comparable to image input.

Proposed solution

Add first-class provider-native file/document attachment support across OpenClaw’s session, gateway, UI, CLI/API, channel, and subagent paths.

Desired behavior:

1. When a user attaches a non-image file, OpenClaw should preserve the file bytes, filename, MIME type, size, and attachment metadata.
2. OpenClaw should determine whether the selected provider/model supports native file/document input for that MIME type.
3. If supported, OpenClaw should forward the attachment to the provider using that provider’s native file/document input format.
4. If unsupported, OpenClaw should fall back to the current workspace/tool-based behavior, but make that fallback explicit rather than silent.
5. OpenClaw should surface a clear user/developer-facing indication of what happened: “sent natively to model,” “exposed as workspace file,” “converted/rendered,” or “unsupported.”

Suggested abstraction:

Add a content type such as FileContent or DocumentContent, alongside existing text/image content.

Example shape:

interface DocumentContent {
  type: "document" | "file";
  data?: string;        // base64-encoded data or data URL
  fileId?: string;      // provider-side uploaded file id, when applicable
  url?: string;         // external file URL, when supported
  mimeType: string;
  filename?: string;
  sizeBytes?: number;
}

Model/provider capability metadata should indicate support for native file input:

input: ("text" | "image" | "document" | "file")[]
supportedMimeTypes?: string[]
maxFileBytes?: number

Provider mappings could include:

• OpenAI Responses API: map supported files to input_file
• Anthropic: map supported documents to document content blocks
• Gemini: map supported documents/files to inlineData or provider file handles
• Unsupported providers: do not silently drop files; return a clear unsupported/fallback state

This should apply to:

• Normal interactive sessions
• Web/Control UI uploads
• Gateway chat.send
• CLI/API attachment paths
• Channel attachments where OpenClaw has access to file bytes
• sessions_spawn and other subagent/delegation flows
• Codex/OpenAI-backed sessions where the backend path can support provider-native file input

The important distinction is that local extraction should remain a fallback, not the default behavior when the provider can accept the file natively.

Alternatives considered

Local extraction via tools

This is useful as a fallback, but it should not be the only path. It is lossy, provider-blind, tool-dependent, and inconsistent. It performs especially poorly on scans, tables, forms, diagrams, slide decks, and documents where visual layout matters.

Client-side extraction before sending

This pushes complexity onto every client and integration. Each client has to implement its own parsers, OCR, error handling, MIME detection, and token budgeting. It also prevents users from benefiting from provider-native document understanding.

Rendering document pages to images

Page rendering can help in some cases, but it is not equivalent to native file/document input. It can be expensive, loses document semantics, requires choosing resolution/page limits, and does not scale well for large files. It is better as a fallback than as the primary architecture.

RAG/vector indexing

RAG is useful for long-term retrieval over extracted text, but it is not a substitute for native file understanding. It does not solve layout-heavy, scanned, visual, tabular, or form-like documents unless the extraction stage is already high quality.

Agent-initiated provider API calls

The agent should not need to make its own OpenAI/Anthropic/Gemini API calls to recover a capability that the framework could provide. This adds credential, security, billing, and architecture complexity. File forwarding should be handled by OpenClaw’s provider/session layer.

Workspace-only attachment mounting

Mounting files into the workspace is useful for code/tool workflows, but it is a different capability from sending files as model inputs. Users should be able to choose models that support native file input and have OpenClaw pass the file through accordingly.

Impact

Affected users/systems/channels

This affects anyone using OpenClaw with non-image files as primary context, including:

• Users uploading PDFs, DOCX, XLSX, CSVs, slides, forms, receipts, statements, reports, contracts, scans, or archives
• Researchers reviewing papers, scans, and source documents
• Legal, finance, admin, and operations workflows involving contracts, invoices, statements, forms, or records
• Students and knowledge workers working with reports, worksheets, slide decks, and scanned materials
• Gateway clients, desktop clients, CLI/API integrations, and channel integrations
• Agent and subagent workflows where a parent session delegates work involving attached files
• Users of OpenAI, Anthropic, Gemini, and other providers that already support native file/document inputs

Severity

This is workflow-blocking for projects where non-image files are the main input. It is not just an annoyance or minor parity gap. For document-heavy workflows, OpenClaw may become unusable compared with using provider UIs or APIs directly, because the model’s native document understanding is not exposed through OpenClaw.

Frequency

For affected workflows, this happens always or very frequently. Every time a user attaches a supported non-image file and expects the model to analyze it natively, OpenClaw either requires a workaround, falls back to local/tool extraction, or may fail to provide the file as effective model context.

Consequence

The practical consequences include:

• Lower answer quality compared with provider-native file handling
• Failed or incomplete analysis of scans, forms, tables, diagrams, and layout-heavy documents
• Extra manual preprocessing and OCR work
• More brittle client integrations
• More token and compute waste from extraction/rendering workarounds
• Inconsistent agent behavior depending on which tools are available and whether the agent chooses to use them correctly
• Silent or confusing failures when files are accepted by the UI/API but not actually sent as useful model input
• Users abandoning OpenClaw for direct provider UIs/APIs when working with files

Requested priority

Please treat this as a first-class model input capability, not just an attachment storage or document extraction feature. OpenClaw already has native-feeling image forwarding; the same architectural pattern should exist for provider-supported file/document inputs, with explicit capability detection and transparent fallback behavior.

Evidence/examples

No response

Additional information

No response