-
-
Notifications
You must be signed in to change notification settings - Fork 80.7k
[Feature] Content-based prompt injection scanning on tool output #79168
Copy link
Copy link
Open
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.ClawSweeper found a clear likely implementation shape for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:needs-security-reviewClawSweeper marked this issue as needing security-sensitive review.ClawSweeper marked this issue as needing security-sensitive review.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:securitySecurity boundary, credential, authz, sandbox, or sensitive-data risk.Security boundary, credential, authz, sandbox, or sensitive-data risk.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Description
Metadata
Metadata
Assignees
Labels
P2Normal backlog priority with limited blast radius.Normal backlog priority with limited blast radius.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.ClawSweeper found a clear likely implementation shape for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.ClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.ClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:needs-security-reviewClawSweeper marked this issue as needing security-sensitive review.ClawSweeper marked this issue as needing security-sensitive review.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.ClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.ClawSweeper found a high-confidence source-level issue reproduction.impact:securitySecurity boundary, credential, authz, sandbox, or sensitive-data risk.Security boundary, credential, authz, sandbox, or sensitive-data risk.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.Very strong issue quality with high-confidence source-level or clear reproduction.
Type
Fields
Priority
None yet
Problem
Tool output (exec results, web fetches, file reads) is not scanned for prompt injection patterns before being fed to the model. The current mitigation is XML wrapping of tool output, which provides structural separation but no content-level defense against indirect injection.
Originally filed as part of #65824 (item 4) and #65816. Re-audited against 2026.5.7 (eeef486).
What 2026.5.7 gets right
Two meaningful changes since the original filing:
before_tool_call/after_tool_callhook surface — makes it possible to build content scanning as a user-space plugin. Thebefore_tool_callhook (hook-runner-global-CCAcWVdN.js:569-574) is a modifying hook withfailurePolicyByHook: "fail-closed"(line 842), meaning a plugin can block tool execution. Theafter_tool_callhook (line 593-597) is a void hook (fire-and-forget) — useful for logging but cannot modify or block tool results.bash-tools-CtaEWCRn.jsadds targeted detection of shell variable injection in exec preflight. Narrow but useful.Neither constitutes built-in content scanning of tool output for prompt injection patterns, but the hook surface significantly lowers the bar for building it externally.
Source evidence (5.7 dist/ file names)
hook-runner-global-CCAcWVdN.js:569-574—runModifyingHook("before_tool_call", event, ctx, { ... })— modifying hook, can block tool execution. Fail-closed policy at line 842.hook-runner-global-CCAcWVdN.js:593-597—runVoidHook("after_tool_call", event, ctx)— void hook, fire-and-forget. Cannot modify or block tool results from reaching the model.selection-BeP8qtCb.js:1402-1424— tool execution metadata (name, duration, result size) is passed to the hook with error handling (ctx.log.warn(...)on hook failure). The hook receives the tool result but cannot intercept it.selection-BeP8qtCb.js:654—/** Track tool execution start data for after_tool_call hook. */— structured metadata available to scanners.native-hook-relay-Dd9iHtsW.js:19—if (!hookRunner?.hasHooks("after_tool_call")) return— early exit when no plugins registered (zero overhead when unused).types-B6qmUh0I.js:410—"after_tool_call"is registered in the typed hook enum.The architectural gap
after_tool_callis void-only (line 597:runVoidHook). A content scanner on this hook can log and alert but cannot prevent the scanned content from reaching the model. For true blocking, content scanning would need either:tool_result_filtermodifying hook that intercepts between tool completion and model submission, orafter_tool_calla modifying hook that can rewrite or redact tool outputThe
before_tool_callhook can block tool execution preemptively but cannot scan tool results since results don't exist yet at that point.Acknowledged trade-off
We recognize robust content scanning is genuinely hard to do without false positives — Unicode normalization, encoding tricks, indirect instruction patterns, and the moving target of injection techniques all make this a research-grade problem. This may be a deliberate deferral rather than an oversight.
If built-in scanning is out of scope, an official reference plugin using the new hook surface would be valuable to the community.
Workaround
cc-taint-check.py— regex scanner with Unicode normalization that blocks high-severity injection patterns. Conservative and narrow precisely because of the false-positive difficulty. Uses theafter_tool_callhook for alerting (cannot block, only log).Proposed solution
Built-in or officially-supported content scanning on tool output, applied at the tool-result layer:
tools.contentScanning.enabled: truewith tunable sensitivityIf built-in is out of scope, a reference plugin demonstrating scanning via
after_tool_call(for alerting) andbefore_tool_call(for preemptive path-based blocking) that the community can fork and customize would still close much of the gap.Impact
High, mitigated. THREAT-MODEL-ATLAS.md rates indirect prompt injection (T-EXEC-002) as High residual risk. The new hook surface reduces the barrier to user-space mitigation, but most users won't build their own scanner. The void-only nature of
after_tool_callmeans even users who do build a scanner cannot block suspicious results — only alert on them.Environment