The definitive curated list of tools, frameworks, standards, and resources for governing AI agents in production.
AI agents that call tools, write code, query databases, and execute actions need the same controls as any other system touching production infrastructure: authentication, authorisation, audit trails, cost controls, policy enforcement, and compliance evidence.
Scope: Runtime governance of AI agents — policy enforcement, audit trails, access control, cost management, compliance tooling, and agent security. Not AI safety research, alignment theory, or general responsible AI ethics.
Why now: The EU AI Act enforcement timeline is live. NIST AI RMF is production-ready. The OWASP Agentic AI Top 10 documents real attack patterns. Claude Code, Copilot, Cursor, and autonomous agent frameworks are now standard tools in enterprise software teams. Governance is no longer optional.
Contributions welcome.
- Why Governance Matters
- International Standards
- Regulatory Frameworks
- Industry Standards and Guidance
- Open-Source Governance Toolkits
- Free Governance Tools
- Enterprise Governance Solutions
- Claude Code and MCP Governance
- Policy Engines and Authorisation
- Audit, Observability, and Cost Control
- Security, Red-Teaming, and Threat Models
- Model and Data Governance
- Agentic Architecture Patterns
- Government and Institutional Guidance
- Learning Resources
- Related Lists
AI agents with tool access operate with the same blast radius as a poorly-scoped IAM role. They can read files they shouldn't, call APIs they weren't meant to, run up unbounded costs, and take irreversible actions — all without a governance layer.
Prompt injection causes agents to execute attacker-controlled instructions via untrusted tool output. Excessive agency allows agents to take actions beyond their intended scope. Unbounded costs emerge when agents loop or call expensive APIs without budget controls. Audit gaps mean that when something goes wrong, there is no record of what the agent did or why. Compliance exposure under the EU AI Act, ISO 42001, and NIST AI RMF requires documented governance evidence.
A governed agent runs with least-privilege tool access, an immutable audit trail, budget enforcement, and policy checks that fire before any irreversible action.
- ISO/IEC 23053 - Framework for AI systems using machine learning. Defines key concepts, components, and lifecycle stages.
- ISO/IEC 23894 - Guidance on AI risk management. Companion to ISO 42001 for operationalising risk processes.
- ISO/IEC 42001:2023 - The international standard for AI management systems. Specifies requirements for establishing, implementing, maintaining, and continually improving an AI management system within an organisation. Certifiable.
- ISO/IEC 42005:2025 - Guidance on AI system impact assessment: how and when to assess the effects of an AI system on individuals and society, and how to document it. Annex A maps it onto ISO/IEC 42001.
- ISO/IEC TR 24028 - Overview of trustworthiness in AI. Covers accuracy, robustness, reliability, safety, security, and privacy.
- Blueprint for an AI Bill of Rights - White House principles for AI systems that affect Americans. Rescinded as US federal policy in January 2025 and retained here only as an archived reference, its five principles still shape procurement language and several state-level bills.
- EU Artificial Intelligence Act - EU regulation classifying AI systems by risk tier with mandatory requirements for high-risk systems. General-purpose AI model obligations effective August 2025. Full enforcement 2026.
- Executive Order 14110 on Safe, Secure, and Trustworthy AI - US federal requirements for AI safety testing, red-teaming, and disclosure for frontier models.
- General-Purpose AI Code of Practice - Voluntary compliance instrument published by the European Commission in July 2025 for providers of general-purpose AI models under Articles 53 and 55 of the AI Act. Chapters on transparency, copyright, and safety and security.
- NIST AI 600-1: Generative AI Profile - Companion profile to the AI RMF covering twelve risk areas specific to generative AI, including confabulation, information security, data privacy, and value-chain integration.
- NIST AI Risk Management Framework - NIST's voluntary framework for managing AI risk. Four functions: Govern, Map, Measure, Manage. Widely adopted as the US enterprise governance baseline.
- NIST AI RMF Playbook - Practical implementation guidance mapping each AI RMF subcategory to suggested actions, outcomes, and measurement approaches.
- UK AI Safety Institute - UK government body responsible for evaluating safety of advanced AI models. Publishes evaluation methodologies and results.
- AER-1 (AI Agent Execution Receipts) - IETF Internet-Draft (draft-zambo-aer1-26) defining a verifiable receipt format for AI agent work: each job's steps and saved results in a record anyone holding the link can recheck. Public conformance corpus (143 vectors, 146 checks) and five independent implementations.
- CISA Guidelines for Secure AI Development - US Cybersecurity and Infrastructure Security Agency guidance on secure AI system development and deployment.
- Cloud Security Alliance AI Safety Initiative - Enterprise guidance on AI security, governance, and trust. Includes the AI Controls Matrix and assessment tools.
- CSA MAESTRO - Seven-layer threat modelling framework for agentic AI from the Cloud Security Alliance, separating traditional per-layer threats from agentic ones arising from autonomy and non-determinism.
- ENISA AI Threat Landscape - EU Agency for Cybersecurity reports on AI-specific threats, risk assessments, and guidelines for EU organisations.
- MITRE ATLAS - Adversarial Threat Landscape for AI Systems. Tactics, techniques, and real-world case studies for attacks against ML and AI systems, modelled on ATT&CK.
- MITRE ATT&CK for AI - Machine learning attack techniques mapped to the ATT&CK framework for integration with existing threat intelligence programmes.
- OWASP Agentic AI: Threats and Mitigations - Threat-model reference for autonomous agents from the OWASP Agentic Security Initiative, with a taxonomy spanning agent design, memory, planning and autonomy, tool use, and deployment.
- OWASP Top 10 for LLM Applications - The ten most critical security risks for LLM-powered applications: prompt injection, insecure output handling, training data poisoning, model denial of service, and supply chain vulnerabilities.
- AffixIO - Agentic Pay Kit (npm
affixio) with host-side signed yes/no ACTION attestation before agent pay and tool gates (x402BeforePay / Agentic Pay / KYA; not person KYC). (npm) (docs) - A-FORGE - Governed execution engine for AI agents: file, service, and git mutations run only after an arifOS verdict, under per-run action-class budgets with HMAC-verified action ingress and receipted runs; the engine never self-authorizes.
- Agent Action Runtime - Open-source execution layer for AI agent actions with policy controls, human approval, retries, post-action verification, recovery, and structured execution receipts.
- agent-airlock - Apache-2.0 Python library that checks agent tool calls before they run. Pydantic V2 strict validation with no type coercion, strips arguments that are not in the tool's signature, and returns structured fix hints the model can retry against. Adds RBAC, rate limits, per-model cost budgets, capability gating, and per-CVE guards for MCP servers. Adapters for LangChain, LangGraph, OpenAI Agents SDK, Claude Agent SDK, PydanticAI, CrewAI, Google ADK, and FastMCP.
- Agenthof - Config-driven control plane governing AI agents through logged model/tool/exec doors: per-call credential injection, per-agent least-privilege, and a hash-chained audit ledger (Apache-2.0, Go).
- Agentlas OS - Apache-2.0 local-first Agent Operation Environment (AOE) with explicit permission scopes, least-privilege tool access, verification gates, and local execution receipts across Claude Code, Codex, Gemini CLI, Cursor, and local models.
- AgentLock - Pre-action authorization for AI agent tool calls. Deny-by-default gate with five decision types, session-level behavioral scoring, Ed25519 signed receipts, and hash-chained audit. Published adversarial benchmark with regression data.
- Agent Passport System - Apache-2.0 protocol for agent identity, scoped delegation, runtime enforcement, and signed action receipts. Includes TypeScript and Python SDKs and an MCP server with 150 governance tools.
- attenu-guard - In-process permission enforcement across AI-agent delegation chains: a sub-agent gets a strict subset of its parent's authority, denials happen before the tool body runs, and the audit ledger verifies offline. Python + TypeScript, 17 framework adapters. Apache-2.0;
pip install attenu-guard. - Busabase - MIT-licensed database and workspace for AI agents with a governed mutation boundary: scoped MCP and OpenAPI credentials can propose ChangeRequests but cannot force them into canonical data, while exact diffs, submitter attribution, comments, reviews, commits, and history remain inspectable. Self-hostable and local-first.
- Casbin - Access control library supporting ACL, RBAC, ABAC, and multi-tenant models. Language-agnostic with production implementations in Go, Rust, Python, Java, and Node.js.
- Cedar - AWS policy language and engine for fine-grained authorisation. Formally verified semantics, expressive syntax, and high throughput for per-request agent permission decisions.
- Chock - Governance-as-code for AI coding agents. Policies committed to the repo compile to pre-tool-use hooks (Claude Code, Cursor), git hooks, and a CI gate, so rules travel with every clone and fork. Per-agent coverage report marks a policy "enforced" only where an installed mechanism is witnessed, and "advisory" where it is prose. Apache-2.0;
pip install chock. - Clay Seal Identity - Short-lived attested agent credentials (SPIFFE JWT/X.509, proof-of-possession, Biscuit capability tokens) with offline verification and optional MCP FastMCP tool authorization. MIT;
pip install clayseal-identity. - CorvinOS - Self-hosted agentic OS with hash-chained tamper-evident audit log (GDPR Art. 30/32), per-user consent gate (deny-by-default), EU AI Act Art. 50 bot-disclosure, and GDPR Art. 17 erasure orchestrator — all as load-bearing architecture constraints.
pip install corvinos. Apache-2.0. - DeadBolt - Apache-2.0 execution gate for AI agent tool calls, with embedded Rust, a local HTTP sidecar, and a stdio MCP proxy; checks policy, expiring leases, operator approval, and kill state before dispatch, and records decisions for actions routed through trusted executors.
- decision-os-min - Python execution-governance runtime: Ed25519-signed action-bound decisions, one-time capability spend, a PEP that refuses unsigned effects, optional OPA/Cedar as replaceable PDPs, hash-chained audit. PolyForm-Noncommercial-1.0.0.
- Guardrails AI - Input and output validation framework for LLM responses. Define schemas, validators, and automated correction actions that enforce structure and safety constraints at inference time.
- Helio - Apache-2.0 governance proxy for MCP agents. Every tool call passes a policy engine, evidence grounding checks, approval workflows, and cross-tool spend budgets that deplete one shared pot across several tools and servers, then lands in an audit trail. Runs in front of unmodified agents and MCP servers; the runtime and dashboard ship in one package (
npx @gethelio/proxy init). - Hexis - Git-backed platform for skills, tools, and context for AI agents. Role-based access, reviewed changes, and an encrypted secrets vault govern what each person and their MCP client can use.
- Hlinor Agent Registry - Apache-2.0 Python library and CLI that gate AI agent actions against reviewed, versioned policy bundles: pre-dispatch allow/deny decisions, experimental request-bound Ed25519 approvals, tamper-evident hash-chained execution receipts, and a protocol-neutral MCP
tools/callvalidator fixture. (PyPI) - HOL Guard - Open-source local-first runtime governance layer for AI agents that evaluates supported tool actions before execution, applies policy checks for prompt injection, secret exposure, unsafe commands, package and MCP risks, and records approval or block receipts.
- Humanbound - Open source testing framework that scores agent behavior against a security policy, targeting the risks in the OWASP Top 10 for Agentic Applications (prompt injection and goal hijacking listed first), and turns failed tests into guardrail rules.
- InGravity - Open-source governance and safety layer for AI agents that evaluates proposed actions against deterministic policies, records decisions, supports human approval workflows, and exposes an MCP gateway.
- Kakunin - Compliance and identity infrastructure for AI agents. Issues X.509 certificates via AWS KMS, enforces per-agent action scope before execution, scores behaviour against a rolling baseline, and auto-revokes credentials when risk crosses a threshold, with a tamper-evident audit trail for MiCA and the EU AI Act. Platform AGPL-3.0; SDKs Apache-2.0.
- Kepil - AGPL-3.0 gate plus journal for agent actions. Every action is checked against a machine-readable per-job mandate before a model is called, fail-closed, and lands in an append-only hash-chained journal verified by a separate implementation in another language. Irreversible actions stop and wait for a person; the HTTP API, the MCP server and the n8n node cannot confirm one, enforced by a test. Python standard library only, no dependencies, state in plain JSON files.
- Kodama - Apache-2.0 API-first, multi-tenant runtime control plane in Go for containerized coding agents and pipelines. Enforces OIDC RBAC, short-lived HMAC execution tokens bound to active action deadlines (revoked while waiting), role-inbox human approval gates, envelope-encrypted secret resolution at activation, an authenticating model gateway and MCP proxy that keep upstream provider keys out of agent containers, forced PostgreSQL row-level security, and gitolite-style
pre-receiveGit access and approval rules (kodama git serve). - KYDE Gateway - Drop-in OpenAI-compatible proxy for OpenAI, Anthropic, Gemini, Copilot, local models, and others that records every agent action into an Ed25519-signed, hash-chained ledger and enforces DLP and per-MCP-tool policies before requests reach the upstream. Source-available BSL-1.1.
- LiteLLM - Proxy layer for LLM API calls with per-key budgets, rate limiting, spend tracking, and model routing across all major providers.
- LlamaFirewall - Meta's open-source guardrail framework for agents. Composes PromptGuard 2 jailbreak detection, chain-of-thought alignment checks, and CodeShield static analysis behind a single policy engine.
- LlamaGuard - Meta's open-source content safety model for classifying LLM inputs and outputs against safety policies.
- MAREF - Open-source agent governance operating system with TLA+ formal verification, 10-state Gray Code governance state machine, per-agent Ed25519 identity, circuit breaker with HALT absorbing state, and LoRA/ontology dual drift detection. Covers 10/10 OWASP Agentic Top 10 risks. Apache 2.0.
- MARGINAL - Apache-2.0 local-first runtime governor for AI coding agents. Observes repeated tool work in Shadow Mode and gates narrow enforcement on verified evidence, explicit consent, and integrity checks.
- MREA - Multi-role agent framework separating Architect, Auditor, and Implementer roles, with risk classification and a human approval gate before implementation. MIT licensed.
- Markus - Open-source AI workforce platform for building and running teams of AI agents with role-based governance, human-in-the-loop task review, persistent memory, and inter-agent communication. Apache-2.0.
- Microsoft Agent Governance Toolkit - Runtime security for AI agents across LangChain, CrewAI, AutoGen, OpenAI Agents, Semantic Kernel, and 15+ frameworks. Covers all 10 OWASP Agentic Top 10 risks with policy evaluation under 0.1ms.
- NeMo Guardrails - NVIDIA's toolkit for adding programmable guardrails to LLM-based systems via Colang configuration language.
- OnlyRead - A Chrome extension that keeps AI agents read-only in your browser. Agents can read the sites you pick but can't change anything on them, enforced on the network requests.
- OpenAIDR - Inventories AI coding agents, MCP servers, and skills; detects risky actions; enforces policies at supported agent hooks; and tracks usage and estimated cost. Client components are Apache-2.0; model-backed checks use OpenSecura's hosted service.
- OpenFirma - Call-level enforcement boundary for autonomous AI agents. Intercepts every outbound call (HTTPS, syscalls, browser automation) and evaluates Cedar policies locally before execution. Fully deterministic, no SDK integration required, agent never holds credentials.
- Open Policy Agent - CNCF-graduated general-purpose policy engine using the Rego language. Decouples policy from application logic; increasingly used for agent tool authorisation.
- plumb-line - Apache-2.0 provenance primitive (JS/Python) that propagates mock or low-confidence taint through derived values, plus five Claude Code skills that audit a repository for unverified claims and source-truth gaps.
- PolicyStrata - Local-first policy regression testing and runtime decision gates for LLM data-agent stacks, with CI checks across model-visible tools, semantic validation, SQL compilation, database controls, and result release. MIT licensed.
- Presidio - Microsoft's PII detection and anonymisation SDK. Identifies and redacts sensitive data in text before it reaches an LLM or audit log.
- Provenza - Self-hosted AI governance platform providing policy enforcement, prompt masking (PII/NER), shadow AI monitoring, and MLOps lifecycle support. Apache-2.0.
- SealRun - Verifies an AI run's exact bytes against a signed manifest before launch and refuses to start if tampered; records a hash-chained audit journal with optional BSV anchoring. Stdlib-only Python, zero dependencies; research preview.
- solvi - Apache-2.0 Python runtime for agent and model decisions: hard checks and policy rules allow, deny or escalate tool calls, conformal risk control calibrates when an answer may be given without a person, and decisions go to a hash-chained store with offline replay and audit reports.
- SteerPlane - Open-source runtime control plane for AI agents: deterministic loop detection, per-session cost ceilings with mid-stream termination, and a hierarchical deny/allow/rate-limit policy engine, enforced via a Python decorator or an OpenAI-compatible gateway proxy. Framework integrations for LangChain, CrewAI, AutoGen, and the OpenAI Agents SDK. No model in the enforcement path. MIT licensed;
pip install steerplane/npm install steerplane. - systemprompt-template - Self-hosted governance layer for Claude Code and MCP agents. Authentication, authorisation, audit trail, cost controls, and policy enforcement in a single compiled Rust binary. Source-available BSL-1.1.
- ThumbGate - Local-first PreToolUse enforcement engine for AI coding agents. Runs in the agent's hook system to hard-block secret exfiltration, destructive deletes, and supply-chain attacks before the tool call executes. Turns thumbs-down feedback into auto-promoted prevention rules. Works with Claude Code, Cursor, Codex, Gemini CLI, Amp, Cline, and OpenCode. MIT licensed, npm installable.
- UNITARES - Self-hosted MCP/HTTP server (Python 3.14; Postgres and Redis via Docker Compose) that keeps an attributed record for operators running multiple agents: identity and lineage, claims and evidence, agent review that keeps objections on file, and outcomes linked to the check-ins and predictions they grade. Runs alongside tool-call gates rather than acting as one. Plugins for Claude Code and Codex; adapters for Hermes Agent and OpenAI-compatible clients. Apache-2.0 licensed.
- YBK Enterprise AI Asset Governance (元冰可) - Self-hosted control plane that registers MCP servers, skills, agents, and AI apps as governed assets on a shared lifecycle state machine (draft → trial → online → offline → archived). Each MCP tool carries its own permission point, so a token's scope does not automatically authorise a tool, and a tool whose risk level was never declared resolves to
unknown— allowed on read-only paths, refused withRISK_UNREGISTEREDon write paths until it clears a domain-then-security approval. On-behalf-of token chains record who delegated to whom, and spend is attributable along three axes: department, agent, and skill. Exposes REST, MCP, and plugin-bridge entry points over one auth model. Node 22, no build step, no dependency beyond the standard library; node:sqlite for metering, JSON collections or SQLite for business data. MIT. - YYLO - Command-line orchestrator for coding agents with typed task, validation, merge, and release-readiness boundaries. Task start freezes the protected target SHA and creates a dedicated branch/worktree per task; the merge queue owns risk-based review (low risk has no semantic reviewer, high risk two sequential reviewers on one frozen candidate, stopping as REVIEW_FINDINGS_EXHAUSTED rather than an unbounded review loop). Workflow runs retain declared receipt hashes and terminal manifests as receipt-backed repository changes. Orchestrates Pi and Codex subagents. MIT licensed, npm installable as @yylo/cli.
Interactive tools that answer common governance questions without a signup. Each is also callable by AI agents over HTTP (POST /api/v1/tools/{tool}/run with an X-Agent-Id header), so the same check an engineer runs in a browser runs in CI or inside a coding agent.
- AI ROI Calculator - Models the cost and payback of an AI rollout with explicit assumptions, for the business case behind a governance budget.
- CLAUDE.md Scorer - Scores a CLAUDE.md against a structured rubric and reports what is missing. Usable in a hook to fail CI when agent instructions degrade.
- EU AI Act Risk Classifier - Walks the Annex III decision tree and returns your system's risk category (prohibited, high-risk, limited, minimal) with the specific articles that apply. Deterministic, not legal advice.
- EVE Authority Lab - Scans a declared agent surface (an OpenAPI spec, MCP server config, IAM policy, or agent tool definitions) and reports every consequential action the agent can take, where its authority exceeds the job, and the control that closes each gap. Verdicts are deterministic and LLM-free, findings are split into what is observable in the artifact versus what a static surface cannot establish, and the result is a signed report a third party verifies offline. No signup. By EVE NeuroSystems.
- GitHub Actions Permissions Generator - Produces a least-privilege
permissions:block for a workflow instead of the default write-all token. - llms.txt Generator and Validator - Generates or lints an llms.txt so AI crawlers and agents get a machine-readable index of your site.
- systemprompt.io Reports MCP Server - Paid remote MCP server (
https://systemprompt.io/api/v1/mcp/systemprompt-reports/mcp) returning decision-ready EU AI Act compliance reports (Annex III classification, article-by-article obligation gaps, Annex IV checklist, remediation plan). Paid, per call; discovery and the usage guide are free.
- AgenticRail - Hosted pre-execution gate for AI agent step order. The caller declares the intended sequence in advance, any step presented out of order is refused before it executes, and the sequence is sealed on completion. Every decision, permit and refusal alike, is written to an Ed25519-signed, hash-chained receipt that verifies offline against published keys. Closed source, free tier with a public demo key.
- Certiv - Endpoint-native, pre-execution security and governance layer for AI agents. An endpoint agent inspects agent actions and tool calls on the device and enforces policy before they execute, with an audit trail of allowed and blocked actions.
- CoreBase - Governance layer for agents that read and write to live business systems; databases, REST and GraphQL APIs, MCP servers, and 50+ SaaS apps. Enforces rules on every tool call, holds risky actions for human sign-off, and logs executed and blocked calls alike. Per-tenant Postgres row-level security, embeddable widget, and an API.
- Credo AI - Comprehensive AI governance platform covering risk assessment, compliance mapping (EU AI Act, NIST AI RMF, ISO 42001), model cards, and ongoing monitoring across the AI lifecycle.
- Datadog LLM Observability - Production monitoring for LLM applications: latency, cost, quality scoring, and trace capture integrated with existing Datadog infrastructure.
- EVE CoreGuard - Deterministic, pre-execution governance for regulated AI decisions. Returns ALLOW/BLOCK/MODIFY before an action runs - a zero-LLM verdict identical for identical inputs and attributable to a named rule - and emits a cryptographically signed per-decision certificate that a third party verifies offline and replays deterministically. Versioned policy packs for ECOA/Reg B fair lending, SR 26-2 model risk, HIPAA, and the EU AI Act. SaaS, VPC, or on-prem. By EVE NeuroSystems.
- Execlave - Runtime governance platform for AI agents. Before each action the agent calls a pre-execution check that allows it, blocks it, or holds it for human approval under the organisation's policies (19 enforced policy types, including access control, tool-call rules, PII access, prompt-injection scanning, Rego, and cost caps). A kill switch pauses an agent and blocks its further calls. Every decision is written to an append-only, hash-chained audit log with signed checkpoints, and the evidence exports as signed EU AI Act compliance reports. Managed cloud hosted in the EU, or self-hosted with Docker Compose or Helm. Proprietary, with MIT-licensed JavaScript and Python SDKs and CLI; free non-commercial tier.
- Gram - Open-source AI control plane for connecting agents to MCPs with role-scoped access, policy enforcement, threat detection, and observable access events.
- Harden Agentic Integrity Foundation (AIF) - Pre-execution check for coding agents that runs on the developer laptop, outside the agent, and checks each command, file edit and push against the session's intent before it runs; one policy across agents, starts watch-only, decisions made locally.
- HiddenLayer - AI detection and response platform. Monitors AI models for adversarial attacks, data extraction attempts, and policy violations.
- Lumenova AI - AI lifecycle governance: risk assessment, explainability monitoring, and compliance reporting focused on model transparency and regulatory evidence.
- Meilynx - Inline proxy for LLM and agent traffic that applies governance rules to each request, permits or blocks specific tools and MCP calls per policy at the boundary, and records each decision in a tamper-evident, hash-chained audit trail. The verifier that checks a Meilynx audit chain is open source (meilynx-verify, Apache-2.0) and runs offline, with no Meilynx system in the loop.
- Microsoft Entra Agent ID - Directory identities for AI agents in Microsoft Entra, with lifecycle management, access reviews, entitlement management, and conditional access applied to agents as to users.
- OneTrust AI Governance - Inventory, risk assessment, and compliance controls for AI systems embedded in broader data governance and privacy programmes.
- Patronus AI - Automated evaluation and monitoring for LLMs in production. Detects hallucinations, toxicity, PII leakage, and custom policy violations.
- Penholder - Preventive human-approval write-gate for AI agents. Intercepts an agent's write to a system of record (Postgres or a governed spreadsheet) and holds it as a durable PENDING proposal that commits only after a human approves — fail-closed on conflict, with an append-only, hash-chained, tamper-evident provenance log. Framework-agnostic and enforced at the write boundary; public network MCP endpoint at api.penholder.ai/mcp.
- Praesidia - Agent governance platform with per-connection tool permissions, rule-based guardrails, human approvals and signed decision records for supported agent and MCP actions routed through Praesidia; proprietary service with a free Developer plan and open-source SDKs and audit verifier.
- Proofpane - Runtime governance gateway for AI coding agents (Claude Code, Cursor, Codex) and automation platforms. Enforces policy allow/deny/human-in-the-loop and DLP redaction in the execution path, and writes a hash-chained audit that exports as an offline-verifiable, Ed25519-signed evidence pack mapped to NIST AI RMF, ISO 42001, EU AI Act, GDPR, and SOC 2.
- Protect AI - MLSecOps platform covering model scanning, supply chain security, and runtime protection for AI and ML systems.
- Safeguard - Software supply chain security platform with agent governance features: an MCP Guardrail proxy in front of MCP servers and agent identity that tracks which agent holds what access, at what level. Commercial with a free signup tier.
- Scalekit - Agent integration layer that connects agents to 600+ apps through per-user OAuth with scoped permissions, exposes only the selected tools to each agent via virtual MCP servers, and logs every tool call for audit. SOC 2 Type II and HIPAA BAA; commercial with a free tier.
- Standard Compute - Hosted LLM gateway that checks every model call against company, team and key rules before it is sent: provider allow-lists that lower levels can only narrow, a Europe region rule and a zero-data-retention requirement, plus per-key model allow-lists and a fixed monthly budget per key, with an admin audit log of every change (before and after values). The Teams platform tier is free.
- systemprompt.io - Self-hosted AI governance infrastructure: a single compiled Rust binary on your own systems that governs, logs, and cost-controls every AI interaction across every provider and client (Claude, Codex, Gemini, or your own agents). Authentication, authorisation, audit trail, policy enforcement, and a provider gateway behind one /v1 endpoint. Air-gap capable; the only entry in this section you run entirely on your own infrastructure. Source-available BSL-1.1 via systemprompt-template.
- ValGuard - Hosted validation layer that checks LLM and agent responses and tool calls against deterministic rules (JSON schemas, tool-call allowlists, PII checks, token and cost budgets) before downstream code acts on them, with shadow mode to measure a rule before enforcing it. On Growth plans and above, each validated request can carry an HMAC-signed, metadata-only record of which validators ran and what they decided. Closed source, free tier.
- agent-approval-gate - PreToolUse hook enforcing spend, account-creation, and fund-movement gates for autonomous agents. Blocked calls become queued approval tickets; includes an installer and a probe-based verifier for both block and allow paths.
- agent-audit-kit - Apache-2.0 offline scanner for MCP server configs, agent settings and instruction files. Flags tool poisoning, unsafe permissions and known-vulnerable MCP server versions, writes SARIF, runs as a GitHub Action, and keeps a public CVE-to-rule ledger.
- Agentic Control Plane - Policy enforcement on every tool call across Claude Code, Codex, Cursor, and MCP clients from one install. Allow, ask, or deny before execution; per-agent and per-workspace spend caps; PII redaction; and an audit trail that resolves each call back to the human who started the run. MIT open core (GatewayStack); runs hosted or fully local.
- AgentLeak - Open-source privacy-leakage benchmark for multi-agent LLM systems (IEEE Access 2026): 7 internal communication pathways, 1,000 scenarios, 4,979 validated traces; 68.8% inter-agent leakage vs 27.2% in final outputs — output-only audits miss 41.7%.
- Aident Loadout - Hosted remote MCP server (
https://loadout.aident.ai/mcp) and CLI that give Claude Code, Codex, Cursor, and other MCP clients mediated access to connected apps and 400+ Skills. OAuth runs in the browser and API keys stay in Aident Vault, so the model receives results rather than raw credentials; agents can check an action's cost and risk before it runs, and an Audit log records what ran, from which agent, whether it succeeded, and what it cost. Closed source with a free pay-as-you-go plan; the agent skill is MIT (aident-skill). - Anthropic Cookbook - Reference implementations and patterns from Anthropic including agent architectures, tool use, and safety patterns.
- arifOS - AGPL-3.0 constitutional policy kernel for MCP-enabled agents. An 8-verb governed MCP surface judges every action against 13 governance floors (F1-F13) before execution, returning ALLOW/HOLD/DENY/VOID/SABAR verdicts and writing immutable audit receipts;
pip install arifos, or point any MCP client at the public no-auth remote endpointhttps://mcp.arif-fazil.com/mcp. - awesome-claude-code - The canonical Claude Code community list covering tooling, hooks, slash-commands, agent skills, and workflows.
- awesome-claude-code-security - Curated list focused on Claude Code hardening: MCP server security, secrets scanning, prompt injection detection, and red-teaming frameworks.
- cc-safety-net - Pre-execution hook for 16 coding-agent CLIs, including Claude Code, Codex, Cursor, Gemini CLI and GitHub Copilot CLI. It blocks destructive Git and filesystem commands, such as
git reset --hard,git push --forceandrm -rfon dangerous targets, and reads of secrets such as SSH keys and.envfiles, before the tool call runs. It parses the command, so wrapping it inbash -corpython -cdoes not hide it. It is not a sandbox. MIT. - Claude Code Documentation - Anthropic's documentation on the permissions model, CLAUDE.md configuration, MCP server setup, and hook system.
- claude-code-guardrails-lite - Claude Code plugin (mod,
tool.callcheck) that denies reading.envsecret files, git force pushes, and recursivermon/,~or$HOMEbefore the tool call runs. Pattern-based, about 75 lines of dependency-free JavaScript, fails closed on internal errors. MIT. - Enforcer Governor (GVNR) - Local guard that decides allow, deny, or ask before each agent action, through a Claude Code PreToolUse hook, an OpenAI- or Anthropic-shaped proxy, or an MCP server. Dollar-per-minute spend limits per agent and across agents are on by default, with caps on new agents and errors per minute, loop detection, and command rules (pipe-to-shell refused; tree deletes, force-pushes, credential access and deploys escalated to a human). Every decision is written to a hash-chained receipt naming the operator. Source-available under FSL-1.1-ALv2.
- eu-ai-act-compliance-mcp - MCP server for EU AI Act compliance checking and risk classification of AI systems.
- HANRIA - Free hosted check of a proposed agent action against a written mandate; returns permit, deny or escalate with an Ed25519 signed receipt that verifies offline. The caller enforces. MCP, A2A 1.0 and HTTP endpoints, no auth.
- Helio - Proxy that governs tool calls from any MCP client. Declarative YAML policies match on tool name, annotations, input parameters, and cumulative state; irreversible actions route to approval workflows; cross-tool spend caps and rate limits apply at the gate; and every call lands in an audit trail with a bundled dashboard. Apache-2.0.
- Kepil MCP server - Seven tools for creating orders, running steps, asking whether an action is permitted by the mandate, and verifying the journal. Confirming an irreversible action is deliberately not among them: a model that could approve itself would make the rest decoration, and a test fails if a confirm tool ever appears in the list.
- lazaretto-guard - Claude Code PreToolUse hook that reads a remote MCP server's tool descriptions for poisoning (pseudo-system blocks, directives at local files, secret-carrying parameters, cross-server shadowing, invisible characters) before
claude mcp addconnects it, and checks pinned package installs against published malware advisories. On a finding it asks for permission and names the rule and line; it never blocks. The server read runs on a hosted scanner with a free key, and a URL carrying credentials is never sent. MIT. - mcp-airlock - Governance proxy in front of an MCP server. Each tool gets a risk tier per environment: read-only, forced dry run, dry run then human confirmation over a signed approve link, or straight through. The confirmation token is HMAC-signed and self-contained, so the proxy keeps no approval store, any replica can finish a confirmation, and each token is spent exactly once; a replay, a tampered argument or a bad signature is refused with a named rule. Blast-radius limits cap how many objects one call and one principal may touch per hour, every call writes a JSONL audit record before and after, and secrets are redacted on the way out. Python, MIT.
- MCP Guard - MIT-licensed stdio MCP proxy that applies YAML allow, block, and approval rules before forwarding routed tool calls to the server. Unrouted calls and direct credentials require separate controls.
- MCP Specification - The Model Context Protocol specification. Understanding the protocol is prerequisite to governing it.
- Mneme - Compiles a repository's architectural decision records into deterministic checks that run on a Claude Code PreToolUse hook, blocking an agent's proposed edit before it is written. Also integrates with Codex CLI and the Claude Agent SDK. MIT.
- optout - CLI that checks the data controls of the AI coding tools on a machine: training, telemetry, feedback-upload and transcript settings for Claude Code, Codex, Gemini CLI, Cursor (privacy mode read from its local state DB), VS Code, Copilot and Aider.
fixwrites the documented opt-outs. Account-side training toggles it can't read are kept as dated, expiring attestations.driftexits non-zero when a setting gets weaker, andtermsdiffs vendor privacy pages. Python, no dependencies, AGPL-3.0. - OrcaPromptVault - Dated archive of what coding agents are actually told and handed: 119 system prompts and 16 tool-schema files from 43 products, 44 of them recorded off the wire by a local proxy with the command that reproduces each one. Claude Code declares 35 tools in an interactive session and 29 under
claude -p, on the same version - the tool surface a policy gate has to cover is not a constant, and this is where to check it per version and per entrypoint. AGPL-3.0. - Provenrail Guard - Claude Code plugin that gates tool calls on a PreToolUse hook. Denies destructive commands (
rm -rf,terraform destroy,git push --force,DROP TABLE,chmod 777) and leaked credentials before they run, escalates lower-confidence cases (.envaccess, deploys, migrations) to a human prompt recorded as oversight, and signs every allow, deny and approval into a hash-chained record thatpr verifyor a browser verifier can recompute offline. Policy is declared in a repo-local.provenrail.json. MIT. - ProxyKey - Credential proxy with a remote MCP server for Claude Code, Cursor and other MCP clients. Real API keys stay AES-256-GCM encrypted on the proxy; the agent issues, rotates and revokes virtual passes with per-pass IP binding, rate limits and request logs, and the toolset has no operation that returns a real key. Free hosted service; the encryption module is published at proxykey-crypto.
- Sanctuary Framework - MCP server that gives an agent encrypted, identity-bound memory and a Tier-1 approval gate on irreversible operations, with an append-only audit log the operator can query. On macOS it enforces a signed operator policy, with a per-account allow/deny demo and reboot survival. On Ubuntu 24.04 (x86-64), the separate Castle Wall Linux 0.1.0-1 package (a GitHub pre-release, not part of the npm package) enforces a signed per-account policy through five reboots on three fresh servers. Other Linux distributions and Windows are unproven. Apache-2.0.
- shim-cli - Local hooks for Claude Code, Codex CLI, GitHub Copilot CLI and VS Code that detect secrets and personal data in prompts and tool results. In Claude Code, tool results are masked before the model reads them; the other clients get a report, a redacted prompt (Copilot CLI) or an opt-in block. Keeps entity names and counts, never values. Apache-2.0, installable from PyPI as
shim. - Snyk agent-scan - Scanner for MCP servers, agent configurations, and agent skills; detects tool poisoning, tool shadowing, and prompt injection in tool descriptions, and can proxy MCP traffic. Formerly Invariant Labs MCP-Scan.
- Sqwakie - Peer-to-peer coordination for Claude Code and Codex agents across a team's machines, as MCP tools and a CLI. Every message and board change is a signed event in an append-only team log (ed25519 key per machine); inbound requests are checked against an owner-signed roster, messages from other agents reach a model wrapped as information rather than instructions, token-shaped secrets are redacted before anything leaves the machine (best effort), agent admin actions are posted to the team's
#generalchannel, and each machine's person keeps a kill switch. Free plan for small teams; source-available under FSL-1.1-ALv2 (each version becomes Apache-2.0 after two years). - systemprompt-core - The MCP governance runtime. 30-crate Rust workspace handling authentication, authorisation, rate limiting, and logging for MCP server interactions. Published on crates.io under
systemprompt-*. - ThumbGate - PreToolUse hook-based enforcement layer that gates Claude Code's tool calls locally before execution. Hard-blocks secret exfiltration, destructive deletes, and supply-chain attacks. Self-improving rules from captured thumbs-down feedback.
- ToolHive - Runs MCP servers in isolated containers with declared permissions, secrets management, and a signed server registry. Apache-2.0, with Kubernetes and CLI deployment modes.
- toolmoat - MIT, dependency-free CLI for MCP setups in Claude Code, Claude Desktop, Cursor, VS Code, Codex and Gemini CLI. A stdio proxy pins MCP tool definitions, holds changed tools for approval, blocks credentials in tool-call arguments and keeps an audit log; a static scanner checks skills, plugins, hooks, permissions and MCP configs, with SARIF output.
- Verax - MCP server between an agent and its tools: each call passes a policy gate and leaves a signed decision record before it runs, an effect row is reconciled against it after, approval waits for an operator on the same machine, and the ledger stays there. Apache-2.0.
- whatileaked - Scans local Claude Code, Codex and Cursor transcripts for credentials already sent to the model, using the gitleaks rule set. No network, zero dependencies.
- provio - One
provio.yamlpolicy (native DSL, Rego or Cedar) checked before every tool call from Claude Code, Codex, Gemini CLI, Cursor, Windsurf, MCP clients and agent SDKs; every decision recorded in a hash-chained, tamper-evident ledger with signed receipts.
- Agent Gorgon - User-space runtime policy guard for agent process trees that observes process, file, and network activity, records JSONL decisions, and can attempt SIGSTOP or SIGKILL on reviewed triggers; audit-only mode rehearses policy without signaling. Apache-2.0.
- Allowly - Hosted policy decisions and signed receipts for AI-agent actions; the caller enforces, and receipts verify offline with workspace Ed25519 keys using open-source Python (
allowly-receipt-format) and TypeScript (@allowly/verifier) verifiers, with a CC BY 4.0 specification and interoperability map. - amu-governance - Lineage-gated access control library for AI-agent shared memory. Gates retrieval on the full column-level derivation of a cached result, not just content tags, to block cross-department leakage through legitimately-computed results. MIT license,
pip install amu-governance. Author-submitted; accompanying paper published in IEEE Access (2026). - AWS Verified Permissions - Managed Cedar policy service on AWS. Centralised policy storage with sub-millisecond evaluation latency for agent action authorisation.
- Casbin - Multi-model access control library supporting ACL, RBAC with hierarchy and domain, ABAC, and RESTful models in 10+ languages.
- Cedar Policy Language - AWS-designed authorisation language with formally verified semantics. Human-readable syntax built for per-request authorisation decisions at high throughput.
- CIRVIX AgentControl - Open-source (Apache-2.0) runtime authorization for AI-agent tool calls explicitly routed through its returned SDK wrappers or MCP gateway, with authorization decisions recorded in a hash-chained audit trail.
- GOPAL - Library of OPA/Rego policies encoding AI-governance regulations as executable allow/deny checks, covering the EU AI Act, NIST AI RMF, UK GDPR Arts 22A-22D, aviation (ICAO/FAA/EASA), FERPA, and financial-services rules. Loads into an OPA server for request-path queries or runs under
opa eval. Apache-2.0. - HashiCorp Sentinel - Policy-as-code framework for Terraform, Vault, Consul, and Nomad. Useful for governing infrastructure provisioned by AI agents.
- OPA Rego Playground - Browser-based environment for writing and testing OPA/Rego policies without local setup.
- Ory Keto - Open-source permission server implementing Google Zanzibar's relation-based access control model for fine-grained agent tool permissions.
- Submilli - Secure runtime for code-mode agents. Each program runs in its own WebAssembly sandbox, and you decide what it may do: semantic permissions over operations, resources and argument values, default deny, with optional human approval. Runs are durable by default: after a restart they resume where they left off, without repeating finished steps. Apache-2.0.
- aGiTrack - Runtime audit trail for terminal coding agents. Wraps Claude Code, Codex, or OpenCode and commits each agent turn to git, recording the prompt, backend, model, and that turn's input, output, cache-read, and cache-write token counts in the commit message, so the trace lives in version control rather than a separate log store. Sub-agent tokens are counted separately. Apache-2.0.
- antiburn - Local desktop app that checks coding agent sessions for common causes of token burn: sessions too deep, premium models in subagents, high reasoning effort, broken caching, and unused MCP servers, skills, and tools. Shows usage-limit meters and per-session context, cost, and tool breakdowns. Supports Claude Code, Codex, Cursor, Copilot, Pi, and more. No account or server. MIT.
- Arize Phoenix - Open-source LLM and agent tracing, evaluation, and dataset tooling built on OpenTelemetry. Self-hostable, with span-level replay of agent runs.
- Asqav - Records each agent action as a hash-chained receipt signed server-side with ML-DSA-65, so the agent never holds the signing key. Receipts verify offline or through a public endpoint without an account. Python and TypeScript SDKs with adapters for LangChain, CrewAI, LlamaIndex, Haystack, Strands, smolagents, and the OpenAI Agents SDK. Elastic License 2.0.
- Bifrost - Go-native, OpenAI-compatible AI gateway with multi-provider routing, automatic failover, load balancing, guardrails, MCP gateway support, and built-in logs, metrics, and tracing.
- CausalLayer MCP - Deterministic AI liability attribution engine exposed as a remote MCP server. Given a structured incident, returns a CausalCertificateV1: a signed, hash-chained, Bitcoin-anchored receipt allocating fault between AI vendor, deployer, and end-user. Four-factor scoring, Shapley-inspired multi-agent attribution, and jurisdiction-aware regulatory mapping (EU AI Act, NIST AI RMF, AU AI Ethics).
- ClawBench - Open-source live-web benchmark for evaluating browser and computer-use agents on 283 tasks across 163 websites, with request interception and execution evidence useful for governance audits.
- Clay Seal Receipts - Verifiable execution receipts for AI agent actions: policy wrap with shadow mode, signed offline-verifiable receipts so audit trails do not depend on the vendor being online. MIT.
- Etch - Signed audit chain for AI agent decisions across MCP client tools. Events carry a hybrid Ed25519 + FIPS 205 SLH-DSA-SHA2-128f envelope and Merkle-chain into epochs anchored to both Sigstore Rekor and Bitcoin OpenTimestamps. Offline reference verifier ships as
world-model-mcpon PyPI; MIT verifier, BSL 1.1 hosted. - Evidently AI - Open-source ML and LLM monitoring. Detects data and model drift, generates monitoring reports, and evaluates LLM output quality.
- Helicone - Open-source LLM observability proxy. Request logging, cost tracking, caching, and rate limiting via a single proxy endpoint. Self-hostable.
- HELM AI Kernel - Post-decision tamper-evident audit for MCP tool calls. Every ALLOW/DENY/ESCALATE decision produces a cryptographically signed receipt, bundled into an offline-verifiable EvidencePack, so you can prove what an agent executed under which policy independently of mutable logs. Targets EU AI Act Article 12 and SOC 2 evidence needs.
- LangFuse - Open-source LLM observability. Full trace capture with spans, generations, scores, and costs. Self-hostable with integrations for LangChain, LlamaIndex, OpenAI, and Anthropic SDKs.
- MemTether - Cross-client AI memory hub with a tamper-evident evidence chain. Every memory decision (write, correct, retire, conflict adjudication) is recorded in a hash-chained audit log; memtether court --verify recomputes the full chain and memtether court --export produces a browser-verifiable evidence package aligned with EU AI Act Article 12 record-keeping. Supersession chains (corrections never delete), bi-temporal timestamps, human conflict adjudication via memtether conflicts --review, and Q-Value ranking. 23 client adapters. 334 tests. Apache-2.0, pip installable.
- Nobulex - Cryptographic receipt layer for AI agents. Ed25519-signed, JCS-canonical bilateral receipts (pre/post execution), hash-chained, independently verifiable. EU AI Act Article 12 compliance.
pip install nobulex/npm install @nobulex/core. - OpenClaw Monitor - Open-source monitoring dashboard for OpenClaw AI agents: token usage and session tracking across models, with 7-day usage trends. Self-hosted Vue 3 + ECharts web UI. MIT.
- OpenLLMetry - OpenTelemetry-based instrumentation SDK for LLM applications. Traces LLM calls with standard OTel spans and integrates with existing observability stacks.
- OpenTelemetry - CNCF standard for distributed tracing, metrics, and logs. The vendor-neutral substrate for building agent observability pipelines.
- Portkey - AI gateway with unified API for 250+ LLMs, request tracing, semantic caching, load balancing, and budget controls.
- Provena - Governs context inputs rather than agent actions: SHA-256 hash-chained audit trail, provenance validation (required-field presence), and freshness checking (staleness thresholds plus temporal-language detection) for RAG, tool, memory, and MCP context sources. Policy engine (log/warn/block), multi-agent trail aggregation, and EU AI Act Art. 10/12/13/14 compliance reports. Adapters for LangChain, LlamaIndex, CrewAI, AutoGen, OpenAI Agents SDK, and Google ADK. Zero core dependencies.
pip install provena. Apache-2.0. - Provenrail - Hash-chained, Ed25519-signed records of agent tool calls, model calls, and guardrail decisions, written to an off-box sink and verifiable without trusting the agent or the vendor. Two independent verifier implementations (a Python CLI and an in-browser JavaScript verifier) are held in lockstep by a frozen conformance-vector suite; anchors carry RFC 3161 trusted timestamps and transparency-log inclusion proofs with witness cosignatures, and both verifiers also validate OpenTimestamps Bitcoin proofs. Records stay on your own infrastructure. MIT client, SDK, verifier and spec; AGPL-3.0 server.
pip install provenrail/npm install provenrail. - Traccia - OpenTelemetry-native observability, governance, and compliance for AI agents and LLM applications. Integrates with OpenAI Agents SDK, CrewAI, Langchain, Claude Code, and more.
- Tracekit - Policy gate for agent tool calls (allow, deny, or hold for a person to approve the exact arguments) with a signed, hash-chained record of every call, decision and approval that exports per run and verifies offline. The record holds up against a compromised agent only when the signer runs as its own system user, in a sidecar or centrally, not in the local dev setup. MIT.
- Tuning Engines - AI control and evidence plane for model, MCP, skill, and agent traffic. Provides governed routing, policy decisions, approval workflows, cost analytics, trace ingestion, and runtime state references.
- Weights and Biases Weave - Tracing and evaluation for LLM applications with strong integrations for LangChain, LlamaIndex, OpenAI, and Anthropic.
- WhyLabs AI Observatory - AI observability platform monitoring LLM applications for drift, data quality issues, and policy violations in production.
- YYLO Benchmark - Evaluation and evidence layer for coding-agent runs, providing two deliberately separate evaluation lanes: isolated attempts execute in private fresh-repository attempt workspaces, while explicitly authorized production-touching workflows execute through a reviewed boundary. Evidence retains the initial workspace receipt plus post-execution repository manifest hashes linked through terminal, evidence, and state; evaluator profile, prompt, rubric, output, generation, and provenance hashes; and evidence/evaluation IDs behind every aggregate. Candidate and judge costs remain separate evidence. Evaluator profiles support deterministic commands and configurable LLM judges. MIT, npm installable as @yylo/benchmark.
- AgentDojo - Benchmark from ETH Zurich's SPY Lab measuring both utility and security of tool-using agents under indirect prompt injection, with pluggable attacks and defences.
- AI Incident Database - Searchable database of 700+ documented AI system failures and harms in deployment. Essential for building realistic threat models and risk assessments.
- AI-Infra-Guard - Tencent Zhuque Lab's self-hosted AI red-teaming platform. Unifies AI infrastructure CVE scanning, MCP server and agent skill auditing (tool poisoning, credential leaks, privilege escalation), and jailbreak evaluation behind one interface; skill-scan ships as a standalone pip-installable CLI for CI/CD.
- AVE (Agentic Vulnerability Enumeration) - Open standard (Apache 2.0) assigning stable IDs to behavioral vulnerability classes in agentic AI components such as MCP servers, agent skills and plugins, with AIVSS scores and crosswalks to MITRE ATLAS and the OWASP MCP and Agentic Top 10 lists.
- awesome-ai-agent-attacks - Curated timeline of 160+ documented AI agent security incidents, breaches, and vulnerabilities (2024-2026). Every entry dated, sourced, and categorized by attack pattern.
- Batesian - Apache-2.0 Go CLI that tests deployed MCP and A2A endpoints for authorization, task isolation, sessions, OAuth audience and scope, and callback weaknesses; exports JSON and SARIF findings.
- Darkmoon - GPL-3.0 autonomous AI penetration testing platform. Agentic reasoning drives real exploit execution across web, API, cloud, identity, CI/CD, IaC, Active Directory, and Kubernetes, producing proof-based findings; a privacy gateway keeps real hosts, addresses, and credentials out of the LLM context.
- Garak - NVIDIA's LLM vulnerability scanner. Probes deployed models for prompt injection, jailbreaks, data leakage, hallucination, and toxicity.
- Ironheights - Apache-2.0 static scanner for OpenClaw/ClawHub agent skills; checks SKILL.md and bundled scripts for prompt injection, credential reads, remote-exec installs and persistence before install, and pins installed versions to detect later changes. Disclosure: submitted by its author.
- llm-redteam-banking-checklist - Apache-2.0 Python CLI and GitHub Actions workflow that turns recorded LLM red-team findings for banking deployments into a release gate: five go/no-go gates (open criticals, high-finding owners, coverage, kill-switch test, monitoring) and a markdown evidence pack. Disclosure: submitted by its author.
- LLM Guard - Security toolkit for LLM interactions with input and output scanners for prompt injection, PII, toxicity, and sensitive data. Archived by its maintainers in July 2026; listed as a still-usable reference implementation rather than a maintained dependency.
- Orca AI Incident Archive - Open database of real-world AI agent security incidents, month by month since January 2025; every record has an attack-chain diagram and at least one primary source. A
real_harmfield separates cases with a confirmed victim from research demonstrations, so threat models can weigh harm that happened against harm that was shown possible. JSON/CSV exports, CC BY 4.0. - PromptBench - Microsoft's unified evaluation framework for adversarial robustness of LLMs. Tests models against adversarial prompts at character, word, sentence, and semantic levels. Archived; the benchmark corpus remains useful, the code is no longer maintained.
- promptmap - Automated prompt injection testing tool. Systematically tests LLM-integrated applications for injection vulnerabilities.
- PyRIT - Microsoft's Python Risk Identification Toolkit for automated red-teaming of generative AI systems including multi-turn and orchestrated agent attacks.
- Red Team Model Router - OpenAI-compatible hosted API serving abliterated and reduced-refusal open models for authorized red-team harnesses. Disclosure: submitted by an operator of the service.
- Skill Doctor - Apache-2.0 offline scanner for agent skill files (SKILL.md, AGENTS.md, rules files) and MCP server manifests. YARA-X rules, Unicode and entropy checks, and a diff between declared and actual capabilities cover eleven threat classes, including prompt injection, data exfiltration, and tool poisoning. Deterministic text, JSON, or SARIF reports can gate CI. Detection accuracy has not yet been benchmarked.
- Skill Safe - Free pre-installation security review for agent skills and tool configurations, with traceable findings for prompt injection, exfiltration, unsafe actions, and permission risks.
- SkillHawk - Zero-dependency static scanner for Agent Skills, SKILL.md and MCP configs; flags risky instructions without executing the target.
- Sunglasses - MIT-licensed local scanner for prompt injection, credential leaks and data exfiltration in text and files. Runs as a CLI, a Python API or an MCP server. Disclosure: submitted by its maintainer.
- VERITAS Omega Agent Trust Lab - Open-source blind calibration lab for testing whether agent-assurance decisions survive forged results, parameter substitution, nonce replay, correlated evaluators, evidence deletion, and missing telemetry.
- Datasheets for Datasets - Microsoft Research framework for documenting dataset provenance, composition, collection process, and recommended uses.
- DVC - Git-like versioning for ML datasets and models. Reproducible pipelines, experiment tracking, and audit trail for training data and model artifacts.
- Great Expectations - Data quality validation framework. Define expectations for training and inference data and alert when data drifts outside governance bounds.
- Hugging Face Model Cards - Implementation guide and templates for model cards on the Hugging Face Hub.
- MLflow Model Registry - Centralised model store with versioning, stage transitions, and approval workflows.
- Model Cards - Google's framework for documenting AI model characteristics, performance, and limitations. De-facto standard for transparent model disclosure.
- Sigstore - Cryptographic signing infrastructure for software artifacts. Enables verification that a model came from a trusted build process.
- SLSA - Supply-chain Levels for Software Artifacts applied to ML models and training pipelines. Defines four assurance levels from basic to hermetic builds.
- 12-Factor Agents - Adaptation of the 12-factor app methodology for LLM agents. Covers configuration, state management, logging, and disposability in agentic contexts.
- Anthropic: Building Effective Agents - Anthropic's published guidance on safe agentic systems: minimal footprint, human-in-the-loop for high-stakes actions, and preference for reversible over irreversible actions.
- awesome-agentic-patterns - Curated collection of production agent patterns including sandboxing, credential management, human-in-the-loop workflows, and multi-agent coordination.
- Bounded Agents - Reference implementation of the Agentic Principal Chain (APC), an external authorisation architecture that attenuates delegated scope and budgets across multi-agent chains, evaluates tool actions against session history, and enforces composition restrictions outside the model. Includes the paper.
- Governed Autonomy Doctrine - Versioned reference architecture for runtime governance of autonomous agents, organised as five architectural planes: agent identity and lifecycle, execution and tool governance, policy and compliance, human oversight and audit, and multi-agent trust and delegation. RFC 001 proposes twenty-nine testable, mechanism-agnostic conformance criteria across those planes and is open for comment through 31 October 2026. Publishes mappings to the NIST AI RMF, the OWASP Top 10 for Agentic Applications, and the IETF SCIM agent governance draft.
- HumanLayer - SDK for building human-in-the-loop workflows for AI agents. Wraps tool calls with approval gates, audit trails, and escalation paths.
- Impri - Self-hostable, framework-agnostic approval inbox for AI agents: agents propose actions via MCP or REST, a human approves/edits/rejects from web, Slack, Telegram or phone, then the agent executes. MIT-licensed open-core.
- Least privilege as an import contract - Single-egress architecture for agent systems: all outbound calls originate from one package behind policy checks and mandatory human approval, an import-linter contract enforced in CI keeps it single, and a static AST gate closes the authorisation gap the import contract cannot see. Includes the incident that motivated the layering and the limits of each layer.
- Lilian Weng: LLM-Powered Autonomous Agents - Comprehensive survey of agent architectures including planning, memory, tool use, and oversight mechanisms.
- Google Secure AI Framework - Google's framework for securing AI systems with six core elements covering foundations, detection, response, and standardisation.
- NIST AI Resource Center - Central hub for NIST AI governance resources including AI RMF, TEVV guidance, and sector-specific playbooks.
- OpenSSF AI/ML Security Working Group - Open Source Security Foundation working group on security for AI and ML supply chains. Produces guidance on securing training pipelines and model artifacts.
- Partnership on AI - Multi-stakeholder organisation producing research and guidance on responsible AI development and deployment practices.
- UK NCSC: Guidelines for Secure AI System Development - Co-authored by NCSC (UK), CISA (US), ACSC (Australia), and 15 other national cybersecurity agencies. Practical security guidance across the AI development lifecycle.
- AgentCraft Claude Code safety checklist - Checks to run before installing a Claude Code skill, plugin or mod, or opening a cloned repo, each tied to a static review of 5,036 skills, 2,688 plugins, 891 mods and 41,142 project skills.
- EU AI Act Compliance Checker - Interactive tool for assessing whether a specific AI system falls under EU AI Act obligations and which requirements apply.
- IAPP AI Governance Professional (AIGP) - Certification covering AI risk assessment, policy development, and compliance implementation. The most widely recognised credential for AI governance practitioners.
- OWASP LLM AI Security and Governance Checklist - Practical checklist for teams deploying LLM-powered systems in production.
- RansomLeak Agentic AI Security exercises - Hands-on browser exercises for the OWASP Agentic AI Top 10 and MCP Top 10, plus an EU AI Act course covering oversight and incident reporting duties.
- SANS Institute AI Security Resources - SANS training and research on AI/ML security covering adversarial attacks, model security, and secure deployment practices.
- Singapore AI Governance Readiness Checklist - Free evidence-oriented checklist mapping IMDA's four agentic AI governance dimensions to 24 production-readiness prompts covering risk bounds, accountable ownership, lifecycle controls, and end-user responsibility.
- State of AI Governance Report - Annual enterprise survey of AI governance programme maturity, common gaps, and implementation patterns from Credo AI.
- awesome-ai-agents - Categorized list of agent frameworks and tools, with observability, safety and security sections.
- awesome-mcp-servers - Comprehensive directory of MCP server implementations.
- AwesomeResponsibleAI - Academic and policy resources for responsible AI covering ethics, standards, and regulatory frameworks.