Skip to content

scan_output() taints clean sessions and forces side-effect tool review #158

Open

Description

@chirag-gamer

I was following the agent checklist in sdk/README.md with a default Guard: scan every model response with guard.scan_output(text), then check side-effect tools before running them. On current dev (cf0b0c7), scanning an ordinary benign model response marks the shared session as tainted, so the very next side-effect call that was allowed before the scan now comes back as REVIEW. That makes the taint review fire on sessions where nothing untrusted was ever ingested.

Observed behavior

scan_output() scans model output with Source.TOOL_OUTPUT, and afterwards guard._maybe_mark_session_tainted_from_scan() adds scan:tool_output to the shared context's taint triggers. This happens even when the output is completely benign and no retrieved or external content entered the session.

The README's own checklist sets up the conflict:

  • Step 5 says to scan agent output: "Scan agent output. guard.scan_output(text)."
  • The same page explains that "A tainted session plus a side-effect tool returns REVIEW."

So a host doing exactly what the docs say ends up with every side-effect tool held for review after the first model response, benign or not. It also dilutes the signal of the review itself: if the session is always tainted, REVIEW stops meaning "untrusted data reached the session".

Minimal reproduction

Run on dev at cf0b0c7, Python 3.13, Windows 11, installed via uv sync --all-extras --dev:

from unplug import Guard
from unplug.api.enums import Action, Source

# Control: a clean session allows a representative side-effect tool.
guard = Guard()
assert guard.context.is_session_tainted is False
baseline = guard.check_tool_call("shell", {"command": "echo hello"})
assert baseline.action == Action.ALLOW

# Documented host step: scan an ordinary model response.
result = guard.scan_output("The answer is 42.")
assert result.safe is True
assert result.action == Action.ALLOW

# Benign output scan mutated shared taint state.
assert guard.context.is_session_tainted is True
assert any("scan:tool_output" in t for t in guard.context.taint_triggers)

# The same call is now held for review solely because of that output scan.
review = guard.check_tool_call("shell", {"command": "echo hello"})
assert review.action == Action.REVIEW
assert any(f.subcategory == "session_taint_side_effect" for f in review.findings)

All assertions pass on current dev. Observed trigger: ['scan:tool_output'].

The streaming path behaves differently

The equivalent chunked output scan is isolated and does not touch shared state:

stream_guard = Guard()
stream_result = stream_guard.scan_stream(["The answer is 42."], source=Source.TOOL_OUTPUT)
assert stream_result.safe is True
assert stream_guard.context.is_session_tainted is False

So non-streaming and streaming output scans have different session side effects for the same content. scan_stream() uses isolated=True internally; scan_output() does not.

Positive controls still work

I checked that real ingestion paths still taint and gate correctly, so this is not a broken tool gate:

  • guard.notify_taint_source("web_fetch") taints the session and check_tool_call("shell", ...) returns REVIEW.
  • guard.scan("Some RAG chunk content here.", source=Source.RETRIEVED) taints via scan:retrieved and the same side-effect call returns REVIEW.

Expected behavior

Output scanning should keep its leakage/redaction protection, but scanning my own model response should not mark the shared session as influenced by untrusted data. Session taint should come from actual untrusted ingestion: retrieved/external content or configured taint-source tools.

One possible direction, not a mandate: separate "scan this text for leakage before it leaves" from "the session has been influenced by untrusted data". Regression coverage would need both cases: benign output leaving the session clean, and real ingestion still tainting. Happy either way if maintainers would rather treat the current behavior as intended CaMeL-style conservatism and document it instead.

Environment: Python 3.13.13, Windows 11, unplug built from dev at cf0b0c7.

Activity

  1. chirag-gamer commented on Aug 24, 2026

    @chirag-gamer
    ContributorAuthor

    assign this to me.

  2. chiruu12 commented on Sep 7, 2026

    @chiruu12
    Collaborator

    Still open. Before assigning, what's your approach to keeping scan_output() from marking the session tainted? Does the taint move off Source.TOOL_OUTPUT, or does _maybe_mark_session learn which sources count?

  3. chirag-gamer commented on Sep 8, 2026

    @chirag-gamer
    ContributorAuthor

    checked the current dev paths and I would not remove Source.TOOL_OUTPUT globally or change its provenance mapping.

    My approach is to make the distinction at the API boundary:

    • guard.scan_output() / scan_output_request() mean “scan the agent/model response before it leaves.” They should keep using the output pipeline for secrets, leakage, URLs, canaries, redaction, and the shared policy/context, but must not mutate session taint.
    • Generic guard.scan(text, source=Source.TOOL_OUTPUT) remains the path for genuinely untrusted tool results. That behavior should continue to taint, including the existing ToolOutputOrchestrator use case.
    • Source.TOOL_OUTPUT is still useful scanner provenance and is also used by boundary handling, so changing/removing it from _maybe_mark_session_tainted_from_scan() would be too broad: it would stop real externally produced tool output from tainting as well.
    • Retrieved scans and explicit notify_taint_source("web_fetch") remain unchanged and continue to drive the side-effect REVIEW gate.

    Concretely, I would remove only the non-isolated call to:

    self._maybe_mark_session_tainted_from_scan(Source.TOOL_OUTPUT)

    from scan_output_request(). I would not make scan_output_request() fully isolated: it should still use the normal shared context for output scanning, registered secrets, canaries, policy, and other existing output-pipeline behavior. The change is specifically to prevent a clean model response from being treated as untrusted ingestion.

    That also aligns the local non-streaming output path with the current streamed path: scan_stream(..., source=Source.TOOL_OUTPUT) already scans in isolation and does not taint the shared session.

    The regression coverage I would add is:

    1. benign scan_output() leaves context.is_session_tainted false and a representative side-effect tool remains ALLOW;
    2. AgentHooks.scan_agent_output() has the same clean-session behavior;
    3. generic guard.scan(..., source=Source.TOOL_OUTPUT) still taints, preserving the actual external-tool-output path;
    4. retrieved scans and notify_taint_source() still taint and make side-effect tools return REVIEW;
    5. scan_stream(..., source=Source.TOOL_OUTPUT) remains non-tainting.

    This keeps the taint signal tied to actual untrusted ingestion rather than to the documented output-leakage check. Does that match the intended separation?

  4. chirag-gamer commented on Oct 13, 2026

    @chirag-gamer
    ContributorAuthor

    ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions