Skip to content

fix(media): guard document extractor metadata#89789

Closed
vincentkoc wants to merge 1 commit into
mainfrom
fuzz-tool-schema-next90-20260603
Closed

fix(media): guard document extractor metadata#89789
vincentkoc wants to merge 1 commit into
mainfrom
fuzz-tool-schema-next90-20260603

Conversation

@vincentkoc

Copy link
Copy Markdown
Member

Summary

  • Guard document extractor metadata reads before MIME matching and extraction.
  • Skip malformed/unreadable extractor rows instead of letting one bad plugin entry prevent later healthy extractors.
  • Preserve method-style extractor this binding when invoking valid extractors.

Verification

  • Red repro before the fix: focused Vitest failed with document extractor mimeTypes getter exploded before the healthy PDF extractor ran.
  • node_modules/.bin/oxfmt --check src/media/document-extractors.runtime.ts src/media/document-extractors.runtime.test.ts
  • node_modules/.bin/oxlint src/media/document-extractors.runtime.ts src/media/document-extractors.runtime.test.ts
  • git diff --check
  • OPENCLAW_VITEST_FS_MODULE_CACHE_PATH=/tmp/openclaw-vitest-next90 node_modules/.bin/vitest run src/media/document-extractors.runtime.test.ts -t "keeps trying healthy document extractors after unreadable metadata" --pool=forks --maxWorkers=1 --maxConcurrency=1 -> 1 file / 1 test, 2 skipped
  • OPENCLAW_VITEST_FS_MODULE_CACHE_PATH=/tmp/openclaw-vitest-next90 node_modules/.bin/vitest run src/media/document-extractors.runtime.test.ts --pool=forks --maxWorkers=1 --maxConcurrency=1 -> 1 file / 3 tests
  • .agents/skills/autoreview/scripts/autoreview --mode branch --base origin/main -> clean, no accepted/actionable findings

Behavior addressed: malformed document extractor metadata no longer crashes document extraction before healthy plugin extractors can run.
Real environment tested: local macOS worktree on branch fuzz-tool-schema-next90-20260603, direct Vitest binary with one fork for the focused and full document-extractor runtime tests.
Exact steps or command run after this patch: OPENCLAW_VITEST_FS_MODULE_CACHE_PATH=/tmp/openclaw-vitest-next90 node_modules/.bin/vitest run src/media/document-extractors.runtime.test.ts --pool=forks --maxWorkers=1 --maxConcurrency=1.
Evidence after fix: full src/media/document-extractors.runtime.test.ts passed with 1 file / 3 tests; focused regression passed with 1 file / 1 test, 2 skipped.
Observed result after fix: unreadable extractor metadata is skipped, the healthy PDF extractor still runs, and method-style extractor receiver binding is preserved.
What was not tested: full pnpm check:changed; Blacksmith Testbox is unauthenticated locally, and AWS Crabbox changed-gate setup failed before lease creation with coordinator HTTP 401 on /v1/runs and /v1/leases.

@vincentkoc vincentkoc self-assigned this Jun 3, 2026
@openclaw-barnacle openclaw-barnacle Bot added size: S maintainer Maintainer-authored PR labels Jun 3, 2026
@clawsweeper

clawsweeper Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge. Reviewed June 21, 2026, 6:25 AM ET / 10:25 UTC.

Summary
The PR guards document-extractor metadata reads, skips malformed extractor rows before MIME matching, preserves extractor receiver binding, and adds a focused regression test.

PR surface: Source +25, Tests +47. Total +72 across 2 files.

Reproducibility: yes. by source inspection. Current main reads extractor.mimeTypes before the extraction try/catch, and the PR's regression test models a throwing metadata getter followed by a healthy extractor.

Review metrics: none identified.

Stored data model
Persistent data-model change detected: vector/embedding metadata: src/media/document-extractors.runtime.test.ts. Confirm migration or upgrade compatibility proof before merge.

Merge readiness
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Refresh the draft branch so GitHub can compute a clean merge result.
  • Rerun the focused document-extractor runtime tests and the appropriate changed-surface gate on the refreshed head.

Risk before merge

  • [P1] GitHub currently reports the draft PR as conflicting, so maintainers need a refreshed branch before trusting the final merge result.
  • [P1] The PR body has focused local test and lint proof, but the broad changed-gate/Testbox proof was blocked and should be rerun before landing.
  • [P1] The linked document-extract user report also tracks registration/build and inspect diagnostics; this dispatcher guard should not be treated as the complete fix for that broader issue.

Maintainer options:

  1. Decide the mitigation before merge
    Refresh the branch against current main, keep the dispatcher guard if the merge result is still narrow, and land it only with focused document-extractor tests plus the appropriate changed-surface gate.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge

  • [P2] The remaining action is maintainer refresh and validation of a protected draft/conflicting PR, not a narrow automated repair task.

Security
Cleared: No concrete security or supply-chain concern found; the diff changes only TypeScript runtime guard logic and a colocated Vitest regression test.

Review details

Best possible solution:

Refresh the branch against current main, keep the dispatcher guard if the merge result is still narrow, and land it only with focused document-extractor tests plus the appropriate changed-surface gate.

Do we have a high-confidence way to reproduce the issue?

Yes by source inspection. Current main reads extractor.mimeTypes before the extraction try/catch, and the PR's regression test models a throwing metadata getter followed by a healthy extractor.

Is this the best way to solve the issue?

Yes for the narrow runtime-boundary hardening. Loader-level artifact isolation, inspect taxonomy, and installed-package build fixes remain separate related work for the broader document-extract report.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 72b9bc730370.

Label changes

Label justifications:

  • P2: This is a bounded document-extraction robustness fix with limited blast radius, not an emergency runtime outage by itself.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Not applicable: The external-contributor proof gate does not apply because this is a maintainer-member PR; the PR body includes focused local runtime test output as validation context.
Evidence reviewed

PR surface:

Source +25, Tests +47. Total +72 across 2 files.

View PR surface stats
Area Files Added Removed Net
Source 1 30 5 +25
Tests 1 47 0 +47
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 2 77 5 +72

What I checked:

  • Root policy read: Root AGENTS.md was read fully and applied to the protected-label, read-beyond-diff, plugin-boundary, and validation requirements for this PR. (AGENTS.md:1, 72b9bc730370)
  • Scoped plugin policy read: The touched media runtime consumes plugin document extractor entries, so src/plugins/AGENTS.md loader and public-contract boundary guidance was read and applied. (src/plugins/AGENTS.md:1, 72b9bc730370)
  • Live PR state: GitHub reports the PR as draft with the maintainer label and mergeStateStatus DIRTY / mergeable CONFLICTING, so it should not be closed or treated as land-ready cleanup. (824257c9aafb)
  • Current-main failure path: Current main reads extractor.mimeTypes before entering the extractor try/catch, so unreadable metadata can abort iteration before a later healthy extractor is considered. (src/media/document-extractors.runtime.ts:40, 72b9bc730370)
  • PR implementation path: The PR adds readDocumentExtractor(), wraps id/mimeTypes/extract reads, skips unreadable entries, and calls extract with the original receiver. (src/media/document-extractors.runtime.ts:22, 824257c9aafb)
  • PR regression coverage: The added test places a throwing mimeTypes getter before a healthy PDF extractor and verifies the healthy extractor still returns while preserving receiver state. (src/media/document-extractors.runtime.test.ts:88, 824257c9aafb)

Likely related people:

  • vincentkoc: Current blame ties the media dispatcher path to Vincent Koc, and earlier history shows Vincent moved document extraction into the bundled plugin/runtime path and authored this narrow guard. (role: feature introducer and recent area contributor; confidence: high; commits: 0a7b0096477f, e3cba98f3904, 1856d1f530d4; files: src/media/document-extractors.runtime.ts, src/media/document-extractors.runtime.test.ts, src/plugins/document-extractor-public-artifacts.ts)
  • Shakker: Recent plugin-runtime history includes lazy capability plugin startup work touching the document extractor resolver area, which is adjacent to how extractor rows reach this dispatcher. (role: adjacent plugin-runtime contributor; confidence: medium; commits: 86bdeb056176; files: src/plugins/document-extractors.runtime.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal backlog priority with limited blast radius. labels Jun 3, 2026
@steipete

steipete commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Closing this fuzz-only hardening PR. The only failing state is a synthetic fixture using hostile in-process objects, throwing getters or callbacks, or similarly malformed metadata; there is no linked production report or supported external contract requiring that state.

This adds broad fail-soft behavior and substantial regression surface across consumers. If a real plugin or provider demonstrates the failure, please open a focused issue with the observed payload and stack so validation can live at the single owning registration/parsing boundary. Thanks for exploring it.

@steipete steipete closed this Jul 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintainer Maintainer-authored PR P2 Normal backlog priority with limited blast radius. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants