Skip to content

refactor(pdf): move document extraction to plugin#71278

Merged
vincentkoc merged 3 commits into
mainfrom
refactor/pdf-document-extract-plugin
Apr 25, 2026
Merged

refactor(pdf): move document extraction to plugin#71278
vincentkoc merged 3 commits into
mainfrom
refactor/pdf-document-extract-plugin

Conversation

@vincentkoc

Copy link
Copy Markdown
Member

Summary

  • add a bundled document-extract plugin that owns local PDF extraction and pdfjs-dist
  • add a generic document extractor contract, public artifact loader, and SDK subpath
  • route core PDF extraction through enabled document extractor plugins with a clear disabled/unavailable error
  • update SBOM ownership, docs, and changelog for the dependency split

Why

This continues the SBOM dependency-risk split by moving PDF parsing out of core while preserving default PDF extraction behavior through an enabled bundled plugin.

Tests

  • pnpm plugin-sdk:api:check
  • pnpm deps:root-ownership:check && pnpm deps:sbom-risk:check
  • pnpm test src/media/pdf-extract.test.ts src/plugins/document-extractors.runtime.test.ts extensions/document-extract/document-extractor.test.ts src/agents/tools/pdf-tool.test.ts src/media/input-files.fetch-guard.test.ts
  • pnpm test:extension document-extract
  • pnpm exec tsgo -p tsconfig.core.json --incremental --tsBuildInfoFile .artifacts/tsgo-cache/core-pdf-document-extract-postrebase.tsbuildinfo
  • pnpm build
  • pnpm test:build:bundled-runtime-deps

Note: direct pnpm exec tsgo -p tsconfig.extensions.json ... still hits the pre-existing extensions/qa-lab/web/src/main.ts CSS side-effect import declaration issue on this checkout.

@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation gateway Gateway runtime scripts Repository scripts agents Agent runtime and tooling size: L labels Apr 24, 2026
@vincentkoc vincentkoc self-assigned this Apr 24, 2026
@openclaw-barnacle openclaw-barnacle Bot added the maintainer Maintainer-authored PR label Apr 24, 2026
@vincentkoc
vincentkoc marked this pull request as ready for review April 24, 2026 22:23
@greptile-apps

greptile-apps Bot commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR moves PDF extraction out of core into a bundled document-extract plugin, introducing a generic DocumentExtractorPlugin contract and a dispatch layer (extractDocumentContent) that routes through enabled plugins. The refactor is well-structured and preserves the existing extraction logic faithfully inside the plugin.

  • P1 (src/media/document-extractors.runtime.ts lines 51–61): Errors thrown by individual extract() calls are silently discarded (catch { continue; }). When pdfjs encounters a malformed or password-protected PDF, all extractors throw, extractDocumentContent returns null, and extractPdfContent raises "PDF extraction disabled or unavailable" — an incorrect diagnosis that replaces the real pdfjs error. The previous code let these errors propagate directly.
  • P2 (src/plugins/document-extractor-public-artifacts.ts line 66): Factory invocations (exported()) are not individually guarded, so a throwing factory silently disables all extractors in that module.

Confidence Score: 4/5

Safe to merge after addressing the error-swallowing regression that turns real extraction failures into a misleading 'disabled' message.

One P1 finding: the catch { continue; } in the extractor dispatch loop is a behavioral regression — real document-level errors from pdfjs are now silently absorbed and reported as 'extraction disabled or unavailable' instead of the actual cause. This is a present defect on the changed path that will make debugging extraction failures significantly harder. The P2 (unguarded factory call) is a hardening gap that compounds the same symptom.

src/media/document-extractors.runtime.ts (error swallowing in extractor dispatch loop), src/plugins/document-extractor-public-artifacts.ts (unguarded factory invocation)

Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/media/document-extractors.runtime.ts
Line: 51-61

Comment:
**Extraction errors silently swallowed, producing misleading failure message**

When an extractor's `extract()` call throws — for example pdfjs encountering a malformed or password-protected PDF — the `catch { continue; }` block discards the error and moves to the next extractor. With only one extractor registered, all extractors are skipped, `extractDocumentContent` returns `null`, and `extractPdfContent` re-raises "PDF extraction disabled or unavailable". This is factually incorrect: the extractor is enabled and reachable; it failed because of the document itself. The previous code let pdfjs errors propagate to the caller directly, giving a meaningful stack trace. At minimum the caught error should be surfaced (e.g., logged or re-thrown when no subsequent extractor succeeds).

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/plugins/document-extractor-public-artifacts.ts
Line: 66-69

Comment:
**Uncaught exception from factory function causes entire extractor loading to fail**

`exported()` is called without a try/catch. If a `create*DocumentExtractor` factory throws (e.g., a plugin that probes for native dependencies at construction time), the exception propagates out of `collectExtractorFactories``loadBundledDocumentExtractorEntriesFromDir``resolvePluginDocumentExtractors`, which is then absorbed by the `catch { return null; }` in `extractDocumentContent`. The end result is the same misleading "disabled or unavailable" error, and a factory bug in one plugin silently disables all extractors in that module. Wrapping the `exported()` call in its own try/catch and skipping (or logging) failing factories would isolate the blast radius.

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "refactor(pdf): move document extraction ..." | Re-trigger Greptile

Comment on lines +51 to +61
try {
const result = await extractor.extract(params);
if (result) {
return {
...result,
extractor: extractor.id,
};
}
} catch {
continue;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Extraction errors silently swallowed, producing misleading failure message

When an extractor's extract() call throws — for example pdfjs encountering a malformed or password-protected PDF — the catch { continue; } block discards the error and moves to the next extractor. With only one extractor registered, all extractors are skipped, extractDocumentContent returns null, and extractPdfContent re-raises "PDF extraction disabled or unavailable". This is factually incorrect: the extractor is enabled and reachable; it failed because of the document itself. The previous code let pdfjs errors propagate to the caller directly, giving a meaningful stack trace. At minimum the caught error should be surfaced (e.g., logged or re-thrown when no subsequent extractor succeeds).

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/media/document-extractors.runtime.ts
Line: 51-61

Comment:
**Extraction errors silently swallowed, producing misleading failure message**

When an extractor's `extract()` call throws — for example pdfjs encountering a malformed or password-protected PDF — the `catch { continue; }` block discards the error and moves to the next extractor. With only one extractor registered, all extractors are skipped, `extractDocumentContent` returns `null`, and `extractPdfContent` re-raises "PDF extraction disabled or unavailable". This is factually incorrect: the extractor is enabled and reachable; it failed because of the document itself. The previous code let pdfjs errors propagate to the caller directly, giving a meaningful stack trace. At minimum the caught error should be surfaced (e.g., logged or re-thrown when no subsequent extractor succeeds).

How can I resolve this? If you propose a fix, please make it concise.

Comment on lines +66 to +69
const candidate = exported();
if (isDocumentExtractorPlugin(candidate)) {
extractors.push(candidate);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Uncaught exception from factory function causes entire extractor loading to fail

exported() is called without a try/catch. If a create*DocumentExtractor factory throws (e.g., a plugin that probes for native dependencies at construction time), the exception propagates out of collectExtractorFactoriesloadBundledDocumentExtractorEntriesFromDirresolvePluginDocumentExtractors, which is then absorbed by the catch { return null; } in extractDocumentContent. The end result is the same misleading "disabled or unavailable" error, and a factory bug in one plugin silently disables all extractors in that module. Wrapping the exported() call in its own try/catch and skipping (or logging) failing factories would isolate the blast radius.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/plugins/document-extractor-public-artifacts.ts
Line: 66-69

Comment:
**Uncaught exception from factory function causes entire extractor loading to fail**

`exported()` is called without a try/catch. If a `create*DocumentExtractor` factory throws (e.g., a plugin that probes for native dependencies at construction time), the exception propagates out of `collectExtractorFactories``loadBundledDocumentExtractorEntriesFromDir``resolvePluginDocumentExtractors`, which is then absorbed by the `catch { return null; }` in `extractDocumentContent`. The end result is the same misleading "disabled or unavailable" error, and a factory bug in one plugin silently disables all extractors in that module. Wrapping the `exported()` call in its own try/catch and skipping (or logging) failing factories would isolate the blast radius.

How can I resolve this? If you propose a fix, please make it concise.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a78335602d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +59 to +60
} catch {
continue;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Propagate extractor failures for matching MIME handlers

When a matching document extractor throws (for example, a malformed/corrupt PDF in extractor.extract), this catch swallows the error and falls through as if no extractor existed; extractPdfContent then rethrows a generic "disabled or unavailable" message. That regresses diagnostics compared with the previous direct PDF.js path, because real parsing/runtime failures are now misreported as plugin disablement, making operator triage and user error handling inaccurate.

Useful? React with 👍 / 👎.

@vincentkoc
vincentkoc force-pushed the refactor/pdf-document-extract-plugin branch from c885d2b to 5c89266 Compare April 25, 2026 00:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling docs Improvements or additions to documentation gateway Gateway runtime maintainer Maintainer-authored PR scripts Repository scripts size: XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant