Skip to content

[FEATURE]: Local OCR fallback for images when no vision model is available #48333

Closed as not planned

Description

@savasturkoglu1

Feature hasn't been suggested before.

  • I have verified this feature I'm about to request hasn't been suggested before.

Describe the enhancement you want to request

Follow-up to the image transcription feature (#48332). When no vision-capable model exists in the selected model's provider, images are still not readable.

This is the same goal as #46785 (RapidOCR-json) but with a bundled Tesseract engine instead of a separately installed OCR binary, so it works out of the box.

Request: a fully local OCR fallback that runs on the user's machine, so the image never leaves the device.

Findings from a prototype:

  • tesseract.js does not work in the compiled bun build --compile binary. The worker thread cannot resolve wasm-feature-detect and other modules because the on-disk node_modules paths are not present in the binary. The build succeeds but createWorker hangs.
  • tesseract-wasm is a better fit: its core WASM is ~1.8MB (vs ~43MB for tesseract.js-core), it exposes a synchronous createOCREngine({ wasmBinary }), and it worked in a source run. Its WASM is not exported from the package exports, so embedding it in the binary needs either vendoring the WASM or a build step that copies it next to the binary.
  • Language data (eng.traineddata.gz, ~14MB) can be bundled and loaded with loadModel, no network needed.
  • Image input needs decoding to RGBA before loadImage; photon-node is already a dependency and can decode.

Open questions: where to put the OCR engine (main thread vs worker), how to embed the WASM in the compiled binary, and whether OCR should be opt-in via config.

Activity

  1. neriousy commented on Oct 21, 2026

    @neriousy
    Member

    Thanks for the suggestion. An MCP server can add this today without changes to opencode: https://opencode.ai/v2/docs/mcp-servers. We'd rather keep it out of core so it works with whichever server people prefer. Closing as not planned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions