Curated list of AI agent security tools, papers and datasets: runtime controls, sandboxes, scanners for MCP servers and skills, static analysis, red teaming, incident records, threat models and standards.
An AI agent is a model that can read untrusted content and call tools with real privileges. Securing one means controlling what it may touch, isolating what it runs, scanning what it loads, and measuring how it fails. Every entry below exists on GitHub and is maintained; descriptions are the projects' own words, shortened. Maintained by Muhammad Basit Ali; projects by the maintainer are listed in their sections like any other entry. See contributing.md for the inclusion criteria and SECURITY.md to report a listed project that has turned malicious.
- Runtime controls and authorization
- Sandboxes
- Scanners for MCP servers and skills
- Static analysis
- Red teaming and evaluation
- Datasets and incident records
- Threat modelling and taxonomies
- Standards and guidance
- Papers
- Talks
- Related lists
- Contributing
- Related projects
Brokers, gateways, policy engines and guardrails that sit between an agent and the tools, credentials and data it uses.
- jentic-one - Self-hosted execution layer that connects agents to APIs, scopes what they can touch and keeps credentials out of the agent's hands.
- hermes-vault - Local-first credential broker, scanner and encrypted vault for the Hermes agent.
- ADR - Observability, security benchmarking and threat detection for enterprise AI agents, from Uber.
- invariant - Guardrails and policy rules for agent tool calls and traces.
- NeMo Guardrails - Toolkit for adding programmable guardrails to LLM-based conversational systems.
- Guardrails - Framework for input and output validators around LLM calls.
- OpenAI Guardrails - Python guardrails library from OpenAI.
- PurpleLlama - Meta's tools for assessing and improving LLM security, including Prompt Guard, Llama Guard, LlamaFirewall and CyberSecEval.
- DefenseClaw - Security governance for agentic AI, from Cisco AI Defense.
- mcp-gateway - Plugin-based gateway that proxies other MCP servers and adds security plugins.
- mcp-guardian - Manage, proxy and secure MCP servers with message approval and logging.
- agentgateway - Proxy for AI agents and MCP servers with policy, observability and multiplexing.
- Docker MCP Gateway - Docker's MCP gateway and
docker mcpCLI plugin for running servers in containers. - ToolHive - Platform for running and managing MCP servers with isolation and policy, including a Kubernetes operator.
- mcp-context-protector - Security wrapper for MCP servers from Trail of Bits that pins tool descriptions and guards context.
- Vigil - Detects prompt injections, jailbreaks and other risky LLM inputs with YARA rules and models.
- Hyperconsciousness - Developer-alpha encrypted, append-only knowledge store with scoped, expiring grants for agent access through MCP.
- agent-airlock - Deny-by-default contract layer for AI agent tool calls, with strict argument validation, ghost-argument stripping and self-healing retries, applied as one decorator.
- AEGIS - Windows-first desktop oversight for AI agents with local monitoring, security reviews, opt-in action policies and MCP gateways.
Isolation for the code and processes an agent runs.
- nono - Micro-sandboxes for agent runtimes with zero setup and zero latency.
- Wassette - Security-oriented runtime that runs WebAssembly components via MCP with capability-based permissions.
- onecli - Sandboxed agent harness for teams with built-in secret management.
- sandbox-runtime - OS-level filesystem and network restrictions for arbitrary processes without a container.
- E2B - Secure sandboxed environments for running agent-generated code.
- microsandbox - Local-first microVM runtime and library for untrusted code.
- Firecracker - MicroVMs for serverless workloads, used underneath several agent sandboxes.
- gVisor - Application kernel that isolates containers from the host kernel.
- bubblewrap - Low-level unprivileged sandboxing tool used by Flatpak and agent runtimes.
- nsjail - Process isolation with Linux namespaces, cgroups, rlimits and seccomp-bpf filters.
- agent-sandbox - Kubernetes
Sandboxresource and controller for isolated, stateful, singleton workloads such as agent runtimes, delegating isolation to runtimes like gVisor or Kata Containers.
Tools that inspect MCP servers, agent skills, plugins and configuration before an agent loads them.
- SkillSpector - Scanner for agent skills that detects prompt injection, data exfiltration and supply-chain risks before installation.
- skill-scanner - Security scanner for agent skills from Cisco AI Defense.
- agent-scan - Security scanner for AI agents, MCP servers and agent skills, maintained by Snyk.
- mcp-scanner - Scans MCP servers for threats and security findings.
- a2a-scanner - Scans agents that speak the A2A protocol for threats and security issues.
- agentshield - Scanner for agent configurations, MCP servers and tool permissions, available as a CLI, GitHub Action and GitHub App.
- AI-Infra-Guard - Red teaming platform with agent, skill, MCP and infrastructure scans and jailbreak evaluation.
- mcp-shield - Security scanner for MCP servers.
- MCPSafetyScanner - Agent-driven MCP safety auditing and remediation, with an accompanying paper.
- agent-config-audit - Audits agent configuration files (Claude Code settings,
.mcp.json, Cursor rules, hooks, plugins) for risky permissions, secrets, unpinned servers and prompt injection. - agent-security-skills - Claude Code plugin and skill pack for agent security reviews: threat modelling, config audits, MCP server review and incident lookup.
- Ironheights - Open-source local-first security scanner for OpenClaw agent skills that catches malicious ClawHub skills, prompt injection, credential theft and tampering before install.
- agent-audit-kit - Static security scanner for MCP-connected agent pipelines that finds misconfigurations, hardcoded secrets, tool poisoning, rug pulls and tainted data flows, and runs fully offline.
Rules and analysers that find insecure agent code and workflows before they run.
- agentic-semgrep-rules - Semgrep rule pack for agent code in Python, TypeScript and JavaScript: model output reaching shells, SQL, URLs and files, over-broad tools, MCP servers without auth, leaked keys.
- agentic-radar - Security scanner that maps agentic workflows and reports their weaknesses.
- Semgrep - Static analysis engine the rule packs above run on.
- Gitleaks - Secret scanner for repositories, files and agent configuration.
- aibom - Generates an AI bill of materials by scanning source code.
- whatileaked - Scans local Claude Code, Codex and Cursor transcripts for credentials already sent to the model, using the gitleaks rule set.
Attack frameworks, benchmarks and vulnerable targets for measuring how an agent fails.
- garak - LLM vulnerability scanner with probes for injection, leakage and jailbreaks.
- PyRIT - Python Risk Identification Tool for red teaming generative AI systems.
- promptfoo - Tests and red-teams prompts, agents and RAG pipelines with declarative configs and CI integration.
- DeepTeam - Framework for red teaming LLMs and AI agents.
- Giskard - Evaluation and testing library for LLM agents.
- AgentDojo - Dynamic environment to evaluate prompt injection attacks and defences for LLM agents.
- InjecAgent - Benchmark for indirect prompt injection in tool-integrated agents.
- Agent Security Bench - Benchmark of attacks and defences for LLM-based agents.
- Agent-SafetyBench - Safety benchmark for LLM agents across risk categories and environments.
- SafeArena - Benchmark for assessing the harmful capabilities of web agents.
- inspect_evals - Collection of evaluations for the Inspect framework, including agent safety tasks such as AgentHarm.
- HarmBench - Evaluation framework for automated red teaming and robust refusal.
- JailbreakBench - Open robustness benchmark for jailbreaking language models.
- Cybench - Capture-the-flag tasks for evaluating the cybersecurity capabilities and risks of agents.
- agentshield-benchmark - Open benchmark for agent security tools covering prompt injection, exfiltration, tool abuse and provenance.
- mcp-injection-experiments - Code to reproduce MCP tool poisoning attacks.
- Damn Vulnerable MCP Server - Deliberately vulnerable MCP server for learning and testing scanners.
- Agentic Security - Vulnerability scanner and red teaming kit for agent workflows and LLMs, with multi-step jailbreaks, fuzzing and multimodal attacks.
Agents that use models to find vulnerabilities; useful as targets and as references for how tool-using agents are built and constrained.
- Strix - AI penetration testing agent that finds and fixes application vulnerabilities.
- Shannon - AI pentester for web applications and APIs that analyses source code and executes exploits to prove findings.
- pentest-ai - AI pentester that re-runs each exploit and ships a replayable proof for every finding.
- open-kritt - Self-hosted vulnerability research tool that orchestrates agents to find and validate issues in code.
- AutoCVE - Agent-driven platform for source code auditing, vulnerability verification and report generation.
- Pentest-Swarm-AI - Autonomous penetration testing with a swarm of specialised agents.
- Anthropic-Cybersecurity-Skills - Structured cybersecurity skills for agents, mapped to MITRE ATT&CK, NIST CSF, MITRE ATLAS and D3FEND.
Structured records of attacks, incidents and vulnerabilities.
- ai-agent-incidents - Open dataset of publicly documented AI agent security incidents, mapped to OWASP and MITRE ATLAS, with a browsable site and RSS feed.
- AI Incident Database - Source of the AI Incident Database, which catalogues harms caused by AI systems.
- AVID - AI Vulnerability Database of reported model and system vulnerabilities.
- tensor-trust-data - Prompt injection attacks and defences collected through the Tensor Trust game.
- jailbreak_llms - Dataset of in-the-wild prompts, including jailbreak prompts, from Reddit, Discord and websites.
- prompt-injections - Collection of prompt injections used by the Giskard scan.
- prompt-injection-defenses - Catalogue of practical and proposed defences against prompt injection.
Ways to describe an agent system and enumerate what can go wrong.
- agent-threat-model - Describe an agent architecture in YAML and get a STRIDE and OWASP Agentic threat model with a control checklist, diagram and SARIF.
- Agent-Wiz - CLI for threat modelling and visualising agents built with LangGraph, AutoGen, CrewAI and other frameworks.
- crosswalk - Cross-reference between the OWASP GenAI Security Project documents and other security frameworks.
- asi - Threat models, verification standards and controls for agent architectures built on OWASP AISVS and NIST AI RMF.
Published frameworks and specifications that agent security work maps to. GitHub repositories are linked so every entry can be checked.
- OWASP Top 10 for LLM Applications - The OWASP GenAI Security Project's Top 10 for LLM applications and related agentic guidance.
- OWASP Agentic Skills Top 10 - OWASP project on the top risks of agent skills.
- OWASP AI Exchange - OWASP's AI security and privacy guide with controls and threat coverage.
- OWASP Machine Learning Security Top 10 - OWASP's top ten security issues for machine learning systems.
- MITRE ATLAS data - Tactics, techniques and case studies of the MITRE ATLAS knowledge base in machine-readable form.
- Dioptra - NIST's test platform for characterising AI technologies, companion to the NIST AI Risk Management Framework.
- Model Context Protocol - The MCP specification, including its authorization and security best practices documents.
- CoSAI agentic secure design - Coalition for Secure AI workstream on secure design patterns and threat models for agentic systems, an OASIS Open Project.
Research with code. Each entry links to the paper's official repository.
- llm-security - Code and demonstrations for "Not what you've signed up for", the paper that introduced indirect prompt injection against application-integrated LLMs.
- llm-attacks - Code for "Universal and Transferable Adversarial Attacks on Aligned Language Models".
- camel-prompt-injection - Code for "Defeating Prompt Injections by Design", which separates control and data flows around the model.
- SecAlign - Code for "SecAlign: Defending Against Prompt Injection with Preference Optimization".
- StruQ - Code for "StruQ: Defending Against Prompt Injection with Structured Queries" (USENIX Security 2025).
- ToolEmu - Emulation framework for identifying the risks of tool-using agents (ICLR 2024).
- R-Judge - Benchmark for safety risk awareness of LLM agents (EMNLP Findings 2024).
Talk and workshop material that is published in a repository. Recordings are added only when a public page for the talk can be verified; please propose them with a link.
- AI Village workshops - Workshop materials published by the AI Village, the DEF CON village for AI security.
- llm_verification - CTFd plugin from the AI Village for running LLM prompt attack challenges at hacker CTFs.
- awesome-llm-security - Tools, documents and projects about LLM security.
- awesome-mcp-security - Security research, writeups and tools for the Model Context Protocol.
- awesome-ai-safety - Papers and articles on AI quality and safety.
- awesome-blackhat-arsenal - Security tools featured at Black Hat Arsenal events.
See contributing.md. Every entry must be a public, maintained project with documentation and a one-line description in the project's own words; archived and unmaintained projects are removed. Run python3 scripts/check_links.py before opening a pull request.
Project hub: https://basitalisandhu.github.io/ . More tools by the same author: https://github.com/basitalisandhu