Skip to content

Show a PDF page or an image to the AI as a picture: File Search view_page (#1384) - #1402

Merged
helpcodeai merged 3 commits into
mainfrom
feat/file-search-view-page
Nov 1, 2026
Merged

helpcodeai merged 3 commits into
mainfrom
feat/file-search-view-page

Conversation

@helpcodeai

Copy link
Copy Markdown
Contributor

Depends on #1401 (103 file types). This branch is based on it, so until #1401 merges its two commits show here too.

Summary

Found in the production test from Claude.ai: the AI can read the text of a PDF, but not what is drawn. A technical drawing's title block, a stamp or a scanned table has no text layer, and Claude.ai does not display embedded PDFs. The new read-only tool <prefix>_view_page returns one page as a JPEG image block, which every MCP client shows to the model.

ee/ authorization: the owner explicitly authorized changes under deploy/ee/file-extractor/ and packages/backend/src/ee/file-search/ for File Search; nothing else under ee/ is touched.

Changes

  • Extractor POST /render: same token, upload limits and slots as /extract, run in the same isolated child process (timeout and memory limit).
    • PDF pages are rendered with pypdfium2.
    • Images (png, jpg, gif, webp, tiff incl. multi-page, bmp) are decoded with Pillow, restricted to those decoders, so an EPS in disguise is never decoded. Above 100 million pixels a file is refused (decompression bomb).
    • Output: longest side 1600 px, JPEG at most 1.5 MB.
    • Errors: unsupported, page_out_of_range, encrypted, too_large, timeout, damaged.
  • Worker POST /internal/render: reads the file through the same index lookup and source read as /internal/fetch, within the plan's file size.
  • Engine file_view: checks scope and folder rules, PDF or image only. Returns an image block plus {path, page, pageCount, note} ("use page N+1 for the next page").
  • Tool <prefix>_view_page (path, page), read-only and idempotent. Mentioned in _guide, the description drafts, the 11 File Search catalog entries and the docs.
  • Backfill on backend boot: existing File Search connectors get missing built-in tools (here _view_page).
    • Runs in the background, idempotent, never touches existing tools.
    • share_file is created switched off.
    • File Share connectors and never-activated drafts are skipped.
    • Never blocks boot.

Cloud and self-hosted alike. Image size +32 KB, no new dependency (Pillow was already in the image).

Type

  • New feature

Testing

  • Added/updated unit tests
Directory Command Result
deploy/ee/file-extractor ./test.sh exit 0; host 227 passed, 7 skipped; hardened no-network container 229 passed, 5 skipped; HTTP stage renders a PDF page
packages/backend npx tsc --noEmit -p tsconfig.json clean
packages/backend npx jest src/file-search src/ee/file-search src/adapters/catalog src/mcp-server --maxWorkers=4 51 suites passed, 2 skipped; 15,399 tests
root node scripts/adapter-count.mjs --check, node scripts/generate-llms-txt.mjs --check OK, fresh

New specs:

  • tests/test_render.py (35 tests: bombs, disguised EPS, out-of-range page, encrypted, damaged, size cap, auth, cleanup)
  • file-search.engine.spec.ts (scope, unsupported type, plan size, success shape)
  • file-search-backfill.service.spec.ts (6 tests)

Related Issues

Relates to #1384, #1381, #1379.

POST /render takes the same token, form and size limits as /extract and
runs in the same isolated child process. PDF pages are rendered with
pypdfium2, images with a fixed set of Pillow decoders after a pixel-count
check; the JPEG is at most 2400 px and 1.5 MB. Errors are unsupported,
page_out_of_range, encrypted, too_large, timeout or damaged.

Refs #1379
<prefix>_view_page checks the path against the caller's scope, accepts
PDF and image files only, and has files-worker read the file and the
extractor render it; the JPEG leaves as MCP image content, which clients
such as Claude.ai show while they do not show PDFs. Saving a connector's
description now also creates built-in tools it is missing.

Refs #1379
Existing File Search connectors only got their tools at activation, so
view_page would never reach them. Once per boot, in the background, every
activated FILE_SEARCH connector in every workspace gets the built-in tools
it lacks, 50 connectors at a time; existing tools, File Share connectors
and drafts are left alone, and errors are logged without stopping boot.

Refs #1379
@helpcodeai
helpcodeai force-pushed the feat/file-search-view-page branch from fc40a8a to 9cb9654 Compare November 1, 2026 18:41
@helpcodeai
helpcodeai enabled auto-merge (squash) November 1, 2026 18:41
@helpcodeai
helpcodeai merged commit b3350de into main Nov 1, 2026
14 checks passed
@helpcodeai
helpcodeai deleted the feat/file-search-view-page branch November 1, 2026 18:56
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 1, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant