Skip to content

feat(oversight): bind approvals to the call they approve - #419

Merged
sattyamjjain merged 2 commits into
mainfrom
feat/bound-approvals
Nov 1, 2026
Merged

sattyamjjain merged 2 commits into
mainfrom
feat/bound-approvals

Conversation

@sattyamjjain

@sattyamjjain sattyamjjain commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

Stacked on #410 (fix/fail-closed-verdicts), since it changes the approval flow #410 rewrote. Retarget to main after #410 merges.

Summary

An approval now binds to the exact call it approves: these arguments, once, and for a limited time. Until now the approver saw only the escalation's reason and rule (core.py:2747 on b21aec7), so it could not see what it approved, and authorize_once was a set of tool names, checked and then discarded without a lock and copied by with_metadata().

Raised in agno-agi/agno#10600 (approved, then the arguments changed, and the substituted call ran 3 of 3 times), crewAIInc/crewAI#6859, google/adk-python#7148 (an in-memory claim let two replicas both run the call), agentarea/agentarea#650 and openai/openai-agents-python#4896.

  1. The request carries the call. OversightRequest gains args_digest, args_preview, agent_id, session_id, policy_digest and expires_at, all with defaults, so a 0.10.28 approver keeps working. args_digest is the sha256 of canonical JSON of the bound arguments (agent_airlock.approvals.args_digest): sorted keys, a non-finite float spelled out, and a {"__unserializable__": type(value).__name__} marker for anything JSON can't encode. args_preview goes through AuditLogger._redact_args, and policy_digest through _compute_policy_digest(), which works on unfrozen policies too.
  2. A decision about other arguments is refused. An OversightResponse can echo args_digest; one that differs from the request's is refused as policy_indeterminate. No echo is read as before.
  3. authorize_once(tool_name, *, args_digest=None, ttl_seconds=None) is spent under a lock (GrantStore), and with_metadata() clones share the store instead of copying it. The reauth guard and the action-contradiction gate both spend through it, so a grant bound to args A never admits args B.
  4. ApprovalLedger, with InMemoryApprovalLedger and SQLiteApprovalLedger (the BEGIN IMMEDIATE pattern from agent_commerce_caps.py). SecurityPolicy(approval_ledger=...) claims a matching approval before asking the approver, atomically, so replicas sharing the file never both run one approval. Expiry is now >= expires_at.
  5. SecurityPolicy.standing_grants: an operator's standing approval of one exact tool with exact target arguments. Only host code makes one, the call's audit record names it (standing_grant), revoke_standing_grant(grant_id) stops it, and an approver's GRANT never turns into one.

Two refusals I added beyond the list, both on the same theme:

  • Arguments changed while the approver decided. Pre-execution ends by working the digest out again from the live values. A shared dict edited in the meantime is refused as policy_indeterminate and the tool never runs. This is the agno#10600 shape inside one process.
  • Arguments that can't be read (nested deeper than 32 levels, or containing themselves) are refused as policy_indeterminate without asking, since no approval can bind to them.

@requires_human_oversight fills the same request fields and raises ValueError on a mismatched echo or arguments changed after the request, the way it already treats a mismatched request_id.

Where I went past or around the work order

  • The ledger key is (tool_name, args_digest), so claim(tool_name, args_digest, call_id) and grant(tool_name, args_digest, *, expires_at). The digest covers the arguments only, so keying on it alone would let an approval for transfer(to=x, amount=5) be claimed by any other tool called with the same arguments.
  • The digest tags what JSON would merge. The work order's marker alone would give every pydantic model the same digest, so an approval for Order(amount=10) would admit Order(amount=10_000). Models, dataclasses, bytes, sets, non-string keys and non-finite floats are encoded by value. The __unserializable__ marker is kept for host objects model JSON can't build, such as a framework's injected context.
  • A dict spelled like a tag is written as pairs (second commit). {"__bytes__": "6869"} was encoded exactly as b"hi", and {"__unserializable__": "Session"} as a host Session object. Model JSON can build those dicts, so a call could reuse another call's approval digest or match its standing grant. A mapping with a key shaped like __x__ is now encoded as key-value pairs, which no tag can be. I found it while building the decision record on top of this.
  • Revocation sits outside the policy digest. Adding a grant after freeze() moves the digest, so the frozen policy refuses every call. Revoking one only narrows the policy, so it doesn't, and freeze() carries revocations over. Otherwise revoking on a frozen policy would refuse every call.
  • New digest keys appear only when set, so every existing policy keeps its 0.10.28 digest (checked for SecurityPolicy() and STRICT_POLICY).
  • Not done: item 6, the optional reviewer callable, which the work order puts in a separate PR. Ledger approvals don't bind to an agent: any agent making that exact call can claim one.

Changed

  • An escalation whose arguments can't be read is refused without asking the approver (it used to be asked).
  • AirlockContext._authorized_once is a GrantStore, not a set. in still works, and consume_grant still spends a set a duck-typed context carries.
  • SecurityPolicy.check_reauthorization and ActionContradictionGate.check_action take an optional args_digest.
  • Audit records of a call a standing grant admitted carry standing_grant. Every other record is unchanged.

Test Plan

  • pytest tests/test_bound_approvals_0_10_29.py: 45 passed in 0.40s. The concurrency tests passed 30 of 30 repeated runs.
  • make lint: ruff check and ruff format clean; mypy shows only the 3 known local errors at integrations/langchain.py:330 (langchain-core 1.6.9 is installed locally, CI doesn't have it)
  • pytest -m "not docker" with --cov=agent_airlock --cov-fail-under=82: 8171 passed, 50 skipped, 5 deselected; coverage 90.10%
  • mkdocs build --strict and scripts/check_docs_api.py: clean
  • scripts/check_bare_imports.py: all 205 modules import with the installed core
  • Digest of SecurityPolicy() and STRICT_POLICY identical to fix(security): refuse indeterminate and late approvals distinctly #410's branch

The new module covers every case the work order lists: a grant for args A doesn't admit B, A admits once and a second A is refused; asyncio.gather of two identical calls with one grant runs one; two processes sharing a SQLite ledger, one claim wins; a grant at now == expires_at is refused; a response echoing another digest is indeterminate; the request carries the digest and a masked preview with no raw secret; a grant spent through a with_metadata() clone is gone from the original.

Mutation check

Each mutation runs in its own interpreter over tests/test_bound_approvals_0_10_29.py (43 pass on the fixed code).

Mutation Tests that go red
Grants are tool names only (as 0.10.28) 2
No lock around the grant spend 3
Expiry read as now > expires_at 2
with_metadata copies the grants (as 0.10.28) 1
The request carries no digest or preview (as 0.10.28) 2
A mismatched digest echo is read as a grant 1
No re-check after the approval 1
Unreadable arguments go to the approver 8
The ledger is not asked 2
A SQLite claim without BEGIN IMMEDIATE 1
Standing grants not read 2
Revocation ignored 2
Standing grants outside the digest 2
A grant matches by == (True == 1) 2
Models read as their type only 1
The decorator does not re-check after approval 1
Tag-shaped keys not escaped (first commit) 1

The SQLite mutation survived at first: the two-process test's children import the unpatched module. test_two_connections_racing_for_one_approval_one_wins races two connections in one process, where only SQLite's write lock stands between them, and catches it.

CHANGELOG lines owed to the next release (not in this PR)

Added:

  • agent_airlock.approvals: args_digest, StandingGrant, and the ApprovalLedger protocol with InMemoryApprovalLedger and SQLiteApprovalLedger. SecurityPolicy(standing_grants=..., approval_ledger=...) admits an escalated call by an operator's standing grant or by an approval claimed from the ledger (one call per approval, across processes with SQLite) before asking the approver. authorize_once takes args_digest and ttl_seconds.

Security:

Changed:

  • An escalation whose arguments nest deeper than 32 levels or contain themselves is refused as policy_indeterminate without asking the approver.

🤖 Generated with Claude Code

sattyamjjain and others added 2 commits November 1, 2026 10:15
OversightRequest carries the sha256 of the call's bound arguments, a
masked preview, the agent and session ids, the policy digest and an
expiry; a response echoing another digest, or a granted call whose
arguments changed while the approver decided, is refused as
policy_indeterminate. authorize_once takes args_digest and ttl_seconds,
is spent under a lock and shared by with_metadata() clones.
SecurityPolicy gains standing_grants (operator-made, exact tool and
target arguments, audited, revocable) and an approval_ledger whose
SQLite claim runs in BEGIN IMMEDIATE so replicas never both run one
approval. Raised in agno-agi/agno#10600 and google/adk-python#7148.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
A plain dict such as {"__bytes__": "6869"} was written exactly as b"hi",
and {"__unserializable__": "Session"} as a host Session object. Model
JSON can build those dicts, so a call could reuse another call's
approval digest or match its standing grant. A mapping with a key
shaped like a tag is now written as key-value pairs.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@sattyamjjain
sattyamjjain changed the base branch from fix/fail-closed-verdicts to main November 1, 2026 04:45
@sattyamjjain
sattyamjjain merged commit 181cc59 into main Nov 1, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant