feat(gatekeeper): runtime fail-closed enforcement middleware (M0–M5) - #76

Extends M0's slice (nothing thrown away):
- GateCost{PureCode,Bounded,Network,Llm} + IToolGate.Cost; UseAgentEvalToolGate REJECTS Network/Llm gates
  at construction (those belong in shadow mode).
- ToolGateAction{Allow,Block,Mutate} + verdict Mutate factory; ToolGatePolicy{WarnOnly,ReplaceResult,Terminate}.
- Terminate: sets FunctionInvocationContext.Terminate + records action="Block"/terminate=true (counted by
  the shipped GlassBoxEvidence).
- MutateArgs: rewrites context.Arguments in place (AIFunctionArguments) + before/after evidence — CONFIRMED
  empirically to reach the function.
- ArgumentPatternGate (reuses the ReDoS-safe, fail-closed ForbiddenPatternScanner; rejects unbounded regex);
  SequenceGate (single-run ordered A-then-B block).

Empirical seam findings (baked into the plan §4b): ThrowOnFail DROPPED — a throw at the function-invocation
seam is caught by FICC and never reaches the caller (hard-fail is an M2 run/chat-gate concern); the Terminate
sibling-hole-fill and ToolExecution-timing dual-benefit remain out of scope (not reachable / deferred to M2).

+7 M1 tests (Terminate, MutateArgs-echo, cost-rejection, ArgumentPattern block + unbounded-throw, SequenceGate,
MCP-shaped custom AIFunction gated by name). net8/9/10 green; 286 MAF+Guardrails regression-clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KyCZ6Em2Rc5aYdaMHuwtuy

…ail; streaming scope)

The run-level gate over AIAgentBuilder.Use(runFunc, runStreamingFunc):
- AgentRunScope (AsyncLocal, Begin()-factory, carries session/agent/trace) — the bridge that lets inner
  tool/session gates read the run context.
- UseAgentEvalGate(pre, post, EvalGatePolicy=WarnOnly, trace?) — reuses the shipped IChatGate/GateVerdict/
  EvalGatePolicy/EvalGateRefusalException over the input (run-pre = incoming-attack detection) and response
  (run-post) TEXT; records gate.run-pre.*/gate.run-post.* evidence (GlassBoxEvidence counts it). Wires BOTH
  branches; register outermost. Block enforcement: WarnOnly->record; ThrowOnFail->throw (PROPAGATES at the run
  boundary, unlike the FICC-swallowed tool seam); Redact->short-circuit a refusal/redacted AgentResponse.
- Streaming: run-pre only; a run-post gate under Redact/ThrowOnFail throws NotSupportedException at stream
  start (never buffer the stream). PERF-01: the AgentRunScope is re-established PER MoveNextAsync segment
  (an AsyncLocal set once atop an async iterator does not survive the first yield) - proven by a streaming
  text-then-toolcall test where a post-yield tool call still sees the scope.

Types re-grounded to 1.12.0: AgentResponse / AgentResponseUpdate (not AgentRun*), AgentSession (not
AgentThread), session.StateBag. +6 tests net8/9/10; 292 MAF+Guardrails regression-clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KyCZ6Em2Rc5aYdaMHuwtuy

- SessionContextGate: base for run-level gates needing the session; reads AgentRunScope.Current and
  FAILS CLOSED (Block) when no scope/session is available (SEC-02 — a control that can't read its context
  must deny). Registered as run-pre gates.
- OperatorAuthGate: allow-list auth over a caller-set session StateBag operator identity
  (gatekeeper.operator); missing identity => Block (MAF surfaces no framework identity).
- RateLimitGate: N runs per rolling window per session. SEC-04: authoritative counter is in-process
  (ConditionalWeakTable keyed by session identity, per-session lock) NOT the StateBag, which has no atomic
  increment and would let a read-modify-write counter be raced/bypassed. Injectable TimeProvider for
  deterministic tests.

+4 tests net8/9/10 (auth fail-closed / allowed / unauthorized; rate within-over-reset with a fake clock).
296 MAF+Guardrails regression-clean.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KyCZ6Em2Rc5aYdaMHuwtuy

…n + 9 major)

Adversarial review (4 finders -> verify) of M0-M2c; all confirmed findings fixed:
- CRITICAL fail-open: ArgumentPatternGate serialized args with DEFAULT JSON escaping, so injection
  metacharacters (< > & ' +) were escaped and a pattern like "<script"/"' OR"/"&&" NEVER matched ->
  gate returned Allow -> payload executed. Now serializes with UnsafeRelaxedJsonEscaping. +theory test.
- Tool gate: a gate THROWING in InspectAsync was swallowed by FICC -> tool ran (fail-OPEN). Now
  fail-CLOSED (blocks + records) regardless of policy. +test.
- Honest evidence: a WarnOnly block recorded action="Block" so GateBlockCount counted a call that RAN.
  Now records action="Warn" (not counted); enforced blocks stay "Block". (tool + run gate)
- SequenceGate: enumeration race (reason built outside the lock) + cross-run state leak (never reset).
  Now per-run state keyed by AgentRunScope (weak table) with all access under the per-run lock.
- Run gate streaming: run-pre gates ran BEFORE any AgentRunScope existed -> SessionContextGate pre-gates
  failed closed on every stream. Now the scope wraps the streaming pre-gate evaluation too.
- RateLimitGate: documented the fixed/tumbling window (was "rolling") and the object-identity keying
  bypass on persisted-session reload.

+8 tests (injection metachars x3, throwing-gate fail-closed, mutate-then-block, streaming auth allow +
fail-closed). net8/9/10: 303 MAF+Guardrails green, 0 regressions.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KyCZ6Em2Rc5aYdaMHuwtuy

…e bypass (self-introduced)

Round-2 convergence review found 3 issues (2 = same critical regression from pass 1):
- CRITICAL: SequenceGate keyed per-run state by AgentRunScope OBJECT identity, but the streaming run gate
  minted a NEW scope per MoveNextAsync segment (the PERF-01 workaround) — so an ordered attack (read_secrets
  then send_email in different stream pumps) got a fresh empty set each segment and was NEVER blocked. Fix:
  AgentRunScope.Enter() re-asserts ONE scope instance across all segments (stable identity); StreamCore now
  begins one scope and re-enters it per pump. +streaming SequenceGate regression test (reads=1, sends=0).
- MAJOR: the run gate's RunGatesAsync did not guard gate InspectAsync against exceptions (the tool gate got
  that in pass 1) — a throwing run-pre gate now fails CLOSED (records + refuses) regardless of policy. +test.

net8/9/10: 29 gatekeeper tests, 305 MAF+Guardrails green, 0 regressions.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KyCZ6Em2Rc5aYdaMHuwtuy

…aring test, doc)

Round-3 convergence review (0 critical/major; verifying r2 fixes) confirmed 3 minors, all fixed:
- RateLimitGate: validate window > TimeSpan.Zero (a non-positive window reset the counter every call,
  silently disabling the limit — fail-open). Mirrors the maxRuns guard.
- Streaming SequenceGate test was NOT load-bearing (passed even without the r2 fix, because both tools
  fell back to the shared key). Replaced with a cross-run isolation test: run 1 fires the trigger, run 2
  (fresh run/scope) fires only the guarded tool and must be ALLOWED — fails without the stable per-run scope.
- SequenceGate: documented that it orders calls across SEPARATE invocations, not a same-iteration
  concurrent batch (the seam exposes one call at a time, no sibling names) — use ForbiddenToolGate for
  strict same-batch protection.

net8/9/10: 29 gatekeeper, 305 MAF+Guardrails green.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KyCZ6Em2Rc5aYdaMHuwtuy

…mpletes M2)

- GateMetadataReader.StageFromKey (Core, additive): extracts the {stage} token from a
  gate.{stage}.{seq}.{policy} key (mirrors PolicyFromKey; returns parts[1]).
- DoctorCommand.CheckDoubleGatingAsync (CLI): warns when the SAME gate policy recorded verdicts under
  BOTH a chat-seam stage (pre/post) and an agent-seam stage (tool/run-*) — the policy is likely gating
  the same run twice. Wired into the doctor flow beside CheckDoubleWrappingAsync. (The ToolExecution
  double-record half is moot: the tool-gate timing dual-benefit was deferred to a later milestone.)

+3 tests (StageFromKey extraction, double-gating warns at both seams, no-warn when seams differ).
net8/9/10: 725 Cli+Tracing+Gatekeeper green, 0 regressions.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KyCZ6Em2Rc5aYdaMHuwtuy

…gates)

New bridge project AgentEval.RedTeam.Gatekeeper (references BOTH AgentEval.MAF gate seams AND
AgentEval.RedTeam probes — no cycle; keeps the core MAF package free of the red-team dep). Closes the
eval -> red-team -> enforcement loop: the SAME oracle that scores an offline attack now blocks a live call.

- ProbeEvaluatorGate : IToolGate — adapts a DETERMINISTIC IProbeEvaluator; attack-succeeded => Block,
  Resisted/Inconclusive => Allow. Caller declares GateCost (IProbeEvaluator carries none); Network/Llm
  rejected at construction + a known-LLM-type deny-list (fixes SafetyMetricGate's unguarded-inline-LLM
  shape — no judge/registry call on the hot tool path). Scans a synthesized surface (tool name + args)
  with UnsafeRelaxedJsonEscaping so injection metachars aren't escaped away.
- CanaryToolGate : IToolGate + CanaryLure — graduates a red-team CanaryTool into a production honeypot:
  CanaryLure.Tools advertises it as a visible schema-only lure the model can be tempted by; the gate
  blocks the emit (wire with Terminate => Block+Terminate+evidence). The stub throws if ever executed.

+7 tests incl the moat proof: an advertised destructive canary is lured, the model calls it, the gate
blocks the emit and the body never runs. (Note: the CLI has no --sut/in-process-agent on-ramp yet; the
attack-the-gate scan runs via MAFAgentAdapter in code — a CLI seam is future work.)

net8/9/10: 3702 MAF+RedTeam+Gatekeeper green, 0 regressions.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KyCZ6Em2Rc5aYdaMHuwtuy

…d-team bridge (8 findings)

Adversarial review of M3 found the "adapt ANY IProbeEvaluator" promise silently failed OPEN for evaluators
whose detection isn't on the string overload. All confirmed findings fixed:
- FAIL CLOSED on the enforcement path: ProbeEvaluatorGate now maps Resisted=>Allow, Succeeded+Inconclusive
  =>Block (cannot-determine => deny). This converts every silent no-op — a behavioral oracle (detection on
  the AgentResponse overload) or a probe-dependent one (reads probe.ExpectedTokens, blanked by the sentinel)
  — into a LOUD block instead of a leak. Inverts the grading convention deliberately: a runtime gate that
  can't prove a call safe must not run it. +test (Inconclusive => blocked).
- Generic LLM rejection: replaced the brittle 2-name deny-list with reflection for an IChatClient field, so
  ANY LLM-backed evaluator (JudgeBacked/Decomposed/... not just LLMJudge/Likert) is rejected even if declared
  PureCode. +test.
- Honeypot cannot be observe-only: added IToolGate.MinimumPolicy (default WarnOnly); CanaryToolGate overrides
  it to ReplaceResult; UseAgentEvalToolGate throws if a gate is registered below its enforcement floor (a
  canary under WarnOnly would silently breach). +test.
- CanaryLure doc corrected: the stub throw is SWALLOWED by FICC, so the lure alone gives NO protection — the
  paired CanaryToolGate is mandatory.
- Made the moat proof LOAD-BEARING: register the REAL destructive body and assert it runs ungated but is
  blocked gated (was a tautology — only the lure stub was wired).

net8/9/10: 3443 Gatekeeper+RedTeam green, 0 regressions.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KyCZ6Em2Rc5aYdaMHuwtuy

…torGate enforcement floor

Round-2 review (3 majors) found the round-1 LLM guard under-covered its own claim:
- HoldsChatClient only checked DIRECT IChatClient fields, but DecomposedGraders composites hold their LLM
  leaves as IProbeEvaluator CHILDREN (CompositeEvaluator/OutcomeFilter/ConjunctionGate) — so a composed LLM
  grader declared PureCode slipped through and would run a judge inline on the hot path. Made the guard
  REACHABILITY-aware: it now reads field VALUES and recurses through IProbeEvaluator children (arrays /
  collections too) with a reference-visited cycle guard. +test (LLM leaf wrapped two composites deep).
- ProbeEvaluatorGate forgot to override IToolGate.MinimumPolicy, so its "only Resisted allows" promise could
  be silently downgraded to WarnOnly (flagged call runs). Now MinimumPolicy=ReplaceResult. +test.

net8/9/10: 13 M3 tests, 3445 Gatekeeper+RedTeam green, 0 regressions.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KyCZ6Em2Rc5aYdaMHuwtuy

…uarantine)

The off-hot-path judge that runs the LLM/network checks the inline gates reject, then arms a fail-closed
gate for a LATER run instead of blocking the one it observed:
- IShadowJudge / ShadowVerdict / ShadowJudgeContext — a judge over an IMMUTABLE snapshot (response text +
  input + session; never the live AgentTrace, whose read paths assume writes are done — avoids the PERF-05
  serialization race entirely by not touching it).
- ShadowJudgePump — an OWNED (IAsyncDisposable, not fire-and-forget) background pump: a BOUNDED channel +
  single consumer. Enqueue is non-blocking; a full queue drops + reports via onError (the returning run is
  never slowed). A compromised verdict SetValue's the quarantine reason into the session StateBag; every
  verdict goes to the onVerdict sink (the output store), never the live trace. A throwing judge is reported,
  not fatal.
- UseAgentEvalShadowJudge — wraps the run seam (runFunc + a streaming variant that accumulates then enqueues
  after the full response); returns the response unchanged and immediately.
- QuarantineGate : SessionContextGate — the arming CONSUMER: fail-closed run-pre gate that blocks a run whose
  session was quarantined by an earlier shadow verdict.

+4 tests incl the closed loop (compromised => armed => next run fails closed), clean-verdict-no-quarantine,
snapshot+sink, throwing-judge-reported. net8/9/10: 322 MAF+Guardrails+Gatekeeper green, 0 regressions.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KyCZ6Em2Rc5aYdaMHuwtuy

…ngs)

Adversarial review of the shadow-judge pump found lifecycle/concurrency defects; all fixed:
- MAJOR: DisposeAsync/CompleteAndDrainAsync awaited the consumer UNBOUNDED, and _cts.Cancel() sat AFTER
  that await — so a hung network judge (the documented use case) hung dispose forever and the cancel token
  was dead. Now a shared DrainAsync does a BOUNDED WaitAsync(drainTimeout); on timeout it cancels in-flight
  judgements (Cancel is reachable WHILE a judge is in flight) and stops waiting. +drainTimeout ctor param.
  +test (hanging judge => Dispose completes within the timeout).
- MINOR: a throwing onError callback propagated out of the catch and killed the consumer (pump silently
  dead => quarantine never armed again). Wrapped every onError in SafeReport (try/catch). +test (throwing
  onError + throwing judge on item 1 => item 2 still judged + armed).
- MINOR: Enqueue reported "queue full" even when the real cause was enqueue-after-completion. Now tracks a
  _completed flag and reports the accurate cause.
- MINOR: the streaming seam had zero coverage. +tests (streamed response judged+quarantined; a stream
  abandoned early enqueues nothing).

net8/9/10: 8 M4 tests, 326 MAF+Guardrails+Gatekeeper green, 0 regressions.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KyCZ6Em2Rc5aYdaMHuwtuy

…ath ObjectDisposedException)

Round-2 review found a second-order regression from the r1 bounded-drain: when the drain abandons a hung
consumer and DisposeAsync disposes _cts, the consumer's per-item re-read of _cts.Token throws
ObjectDisposedException (later buffered items then silently dropped as confusing errors instead of judged).
Fix: capture `var token = _cts.Token` ONCE before the consume loop — the CancellationToken struct stays
usable after the source is disposed. +load-bearing test (a judge that ignores cancellation blocks item 1,
Dispose abandons + disposes _cts, item 1 unblocks, item 2 still judges with no ObjectDisposedException).

net8/9/10: 9 M4 tests, 315 MAF+Gatekeeper green, 0 regressions.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KyCZ6Em2Rc5aYdaMHuwtuy

…ment sample

- docs/gatekeeper.md: the developer-facing walkthrough of the whole stack (tool gate / run gate / session
  gates / the moat / shadow judge), the fail-closed + honest-evidence principle, the cost model (inline
  rejects LLM/network; shadow judge is the release valve), the gate catalog, MinimumPolicy enforcement
  floor, and the doctor double-gating check. Wired into docs/toc.yml beside Guardrails.
- samples SafetyAndSecurity/04_GatekeeperEnforcement: a ★ no-credentials sample (scripted model, fully
  deterministic) demonstrating all four layers failing closed — forbidden tool blocked, a real
  ContainsToken red-team evaluator blocking a poisoned call, a canary honeypot held, and a shadow verdict
  quarantining the next run. Wired into the Safety & Security menu group; references AgentEval.RedTeam.Gatekeeper.

Verified end-to-end: every scenario reports blocked/held/refused.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KyCZ6Em2Rc5aYdaMHuwtuy

The MAF streaming-run delegate types session/options nullable; make StreamCore + StreamAndEnqueue accept
AgentSession?/AgentRunOptions? to match (both are optional on a run) — clears the CS8604 warnings.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Claude-Session: https://claude.ai/code/session_01KyCZ6Em2Rc5aYdaMHuwtuy