You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 2e08ad7
Browse filesBrowse the repository at this point in the historyBrowse files
Python: Samples: terminate only on high/critical matches in ATR validation middleware (#9433)
ATRValidationMiddleware raised MiddlewareTermination on any ATR match,
including low and medium severity. Those rules are broad enough to match
ordinary tool arguments (for example `curl https://example.com` or
`cat ~/.bashrc`), so the sample stopped legitimate tool calls.
The middleware now terminates only when the highest-severity match is
high or critical (BLOCKING_SEVERITIES). That is the severity at which
ATR's reference verdict (determineOutcome in src/verdict.ts) is always
deny or ask; "ask" means a person should confirm, and this sample has no
confirmation step (approval_mode="never_require"), so both stop the call.
Below high, the cut is a false-positive trade-off: ATR also returns ask
for some medium matches, but the same rules match ordinary commands.
Medium and low matches are logged as a warning with rule id, severity
and confidence, and the tool runs. The docstring and README say that
ATR is not a dangerous-command filter (`rm -rf ~` matches only a medium
rule) and that production code should also require tool approval.
detect_attack() now returns the highest-severity pyatr match instead of
only its rule id, so the middleware can read severity and confidence.
Also update the README row to describe this and drop the mention of a
deny-list fallback that the sample does not have.
Co-authored-by: eeee2345 <[email protected]>
Copy file name to clipboardExpand all lines: python/samples/02-agents/middleware/README.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,7 +11,7 @@ This folder contains focused middleware samples for `Agent`, chat clients, tools
11
11
|[`agent_loop_middleware_todos.py`](./agent_loop_middleware_todos.py)| Demonstrates `AgentLoopMiddleware` with a `should_continue` predicate built from a `TodoProvider` via `todos_remaining`, so the agent keeps working while open todos remain. |
12
12
|[`agent_loop_middleware_judge.py`](./agent_loop_middleware_judge.py)| Demonstrates `AgentLoopMiddleware.with_judge`: a ChatClient judge re-runs the agent until it decides the original request was answered, with `criteria` shared between the agent and the judge. |
13
13
|[`agent_loop_middleware_report.py`](./agent_loop_middleware_report.py)| Demonstrates composing two `AgentLoopMiddleware` on one agent: an inner `todos_remaining` loop that drafts a report todo-by-todo, wrapped by an outer report-style `with_judge` loop that re-runs it until an editor chat client judges the report publication-ready. |
14
-
|[`atr_validation_middleware.py`](./atr_validation_middleware.py)| Demonstrates deterministic validation at the tool-execution boundary: a `FunctionMiddleware` that inspects the validated tool arguments and raises `MiddlewareTermination` before the tool runs when they match an attack rule. Loads the open, MIT-licensed Agent Threat Rules ruleset and runs the real engine locally (`pip install pyatr`), with a built-in deny-list fallback when it is not installed. |
14
+
|[`atr_validation_middleware.py`](./atr_validation_middleware.py)| Demonstrates deterministic validation at the tool-execution boundary: a `FunctionMiddleware` that raises `MiddlewareTermination` before the tool runs when the arguments match a high- or critical-severity Agent Threat Rules rule, and logs lower-severity matches as warnings. It is not a dangerous-command filter, so use tool approval in production. Runs the open, MIT-licensed ruleset locally (`pip install pyatr`). |
15
15
|[`chat_middleware.py`](./chat_middleware.py)| Shows class-based and function-based chat middleware that can observe, modify, and override model calls. |
16
16
|[`class_based_middleware.py`](./class_based_middleware.py)| Shows class-based agent and function middleware. |
17
17
|[`decorator_middleware.py`](./decorator_middleware.py)| Demonstrates middleware registration with decorators. |
Copy file name to clipboardExpand all lines: python/samples/02-agents/middleware/atr_validation_middleware.py
+53-16Lines changed: 53 additions & 16 deletions
Original file line number
Diff line number
Diff line change
@@ -36,7 +36,22 @@
36
36
point that validates a tool call right before it executes. ATRValidationMiddleware is a
37
37
FunctionMiddleware that inspects the validated tool arguments in
38
38
``FunctionInvocationContext.arguments`` and raises ``MiddlewareTermination`` BEFORE calling
39
-
``call_next()`` when the arguments match a known attack pattern, so the tool never runs.
39
+
``call_next()`` when the arguments match a high- or critical-severity attack rule, so the tool
40
+
never runs.
41
+
42
+
Why that threshold: ATR's reference verdict is "deny" or "ask" for every high- or
43
+
critical-severity match, where "ask" means a person should confirm the call. This sample has no
44
+
confirmation step (``approval_mode="never_require"``), so it treats both as stop. Below high, the
45
+
cut is a false-positive trade-off, not ATR's verdict: ATR also returns "ask" for some
46
+
medium-severity matches, but those rules also match ordinary commands such as
47
+
``curl https://example.com``, so medium- and low-severity matches are logged as warnings and the
48
+
tool runs.
49
+
50
+
Limits: ATR targets agent attacks such as prompt injection, tool-argument tampering, and
51
+
exfiltration; it is not a filter for dangerous shell commands. ``rm -rf ~`` matches only a
52
+
medium-severity rule, so it runs with a warning, and some destructive commands match no rule at
53
+
all. This middleware does not replace tool approval (``approval_mode="always_require"``) for
54
+
tools with side effects.
40
55
41
56
Detection is delegated to Agent Threat Rules (ATR) -- an open, MIT-licensed detection ruleset
42
57
for AI-agent threats such as prompt injection, tool-argument tampering, and exfiltration. The
@@ -48,6 +63,10 @@
48
63
49
64
logger=logging.getLogger(__name__)
50
65
66
+
# ATR's verdict for a high or critical match is deny or ask; with no confirmation step here, both
67
+
# terminate the run. Medium and low matches are logged and the tool runs.
0 commit comments