Skip to content

Commit 5447098

Browse files
authored
Add integrations hub, extras, and security matrix (#46)
1 parent 9f15436 commit 5447098

25 files changed

Lines changed: 4969 additions & 329 deletions

File tree

‎sdk/README.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -215,7 +215,7 @@ Copy `unplug.example.toml` to `unplug.toml` to customize scanners, tool profiles
215215

216216
## Integrations
217217

218-
Framework hooks for LangGraph and Agno, plus framework-agnostic patterns: [`docs/INTEGRATIONS.md`](docs/INTEGRATIONS.md).
218+
Framework hooks for LangGraph, CrewAI, AutoGen, Haystack, and more: [`integrations/README.md`](integrations/README.md) (guides) · [`docs/INTEGRATIONS.md`](docs/INTEGRATIONS.md) (API reference).
219219

220220
Threat scanners live under `unplug.scanners` (canonical). The older `unplug.safeguards` path still works but emits deprecation warnings:
221221

@@ -243,7 +243,8 @@ See [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) for layering and optional extra
243243
| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | Hosted vs embedded vs sidecar architecture |
244244
| [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md) | Regex vs regex + ML eval results (neuralchemy, microsoft) |
245245
| [`docs/ML_INTEGRATION.md`](docs/ML_INTEGRATION.md) | Checkpoint layout, thresholds, long-text and streaming config |
246-
| [`docs/INTEGRATIONS.md`](docs/INTEGRATIONS.md) | LangGraph, Agno, framework-agnostic hooks |
246+
| [`integrations/README.md`](integrations/README.md) | Per-framework guides, extras, security matrix |
247+
| [`docs/INTEGRATIONS.md`](docs/INTEGRATIONS.md) | LangGraph, Agno, CrewAI, hooks API |
247248
| [`docs/AGENT_FLOW_SECURITY.md`](docs/AGENT_FLOW_SECURITY.md) | End-to-end agent hardening flow |
248249
| [`docs/HERMES_AGENT_SECURITY.md`](docs/HERMES_AGENT_SECURITY.md) | Context-file scanning for agent frameworks |
249250

‎sdk/docs/INTEGRATIONS.md‎

Lines changed: 46 additions & 84 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,15 @@
11
# Agent framework integrations
22

3-
Unplug ships **framework-agnostic hooks** first. LangGraph and Agno do not need to be installed to use the SDK: copy the patterns below or run the demos.
3+
> **Full guides:** [`integrations/README.md`](../integrations/README.md) — per-framework wiring, extras, and the [40-angle security matrix](../integrations/TESTING.md).
4+
5+
Unplug ships **framework-agnostic hooks** first. Install only the extra for your stack:
6+
7+
```bash
8+
pip install unplug-ai # core — no agent SDK deps
9+
pip install "unplug-ai[langgraph]" # LangGraph
10+
pip install "unplug-ai[crewai]" # CrewAI
11+
pip install "unplug-ai[integrations]" # all documented agent/RAG extras
12+
```
413

514
## Core: `AgentHooks`
615

@@ -21,115 +30,68 @@ tool_decision = hooks.before_tool_call("send_email", {"to": "...", "body": "..."
2130
| Hook | When to call |
2231
|------|----------------|
2332
| `scan_user_input` | Before LLM turn |
24-
| `wrap_retrieved_content` | After RAG / web fetch, before context insert |
25-
| `before_tool_call` | Before every tool execution (always local, even in server mode) |
26-
| `scan_agent_output` | Before returning response to user |
27-
| `scan_request_isolated` | Eval / probes (no session taint bleed) |
33+
| `wrap_retrieved_content` | After RAG / web fetch |
34+
| `before_tool_call` | Before every tool (always local) |
35+
| `scan_agent_output` | Before returning to user |
36+
| `scan_request_isolated` | Eval / probes (no session bleed) |
37+
38+
## Framework modules
39+
40+
| Framework | Extra | Module |
41+
|-----------|-------|--------|
42+
| LangGraph | `langgraph` | `unplug.integrations.langgraph` |
43+
| Agno | `agno` | `unplug.integrations.agno` |
44+
| CrewAI | `crewai` | `unplug.integrations.crewai` |
45+
| AutoGen | `autogen` | `unplug.integrations.autogen` |
46+
| Haystack | `haystack` | `unplug.integrations.haystack` |
47+
| LlamaIndex | `llama-index` | `unplug.integrations.llama_index` |
48+
| Pydantic AI | `pydantic-ai` | `unplug.integrations.pydantic_ai` |
49+
| Semantic Kernel | `semantic-kernel` | `unplug.integrations.semantic_kernel` |
50+
| Custom loop | *(none)* | `unplug.integrations.hooks` |
2851

2952
## LangGraph
3053

3154
```python
32-
from unplug import Guard
3355
from unplug.integrations.langgraph import langgraph_input_node, langgraph_tool_guard
34-
from unplug.integrations.hooks import AgentHooks
3556

36-
hooks = AgentHooks(Guard(mode="server")) # hosted API
3757
graph.add_node("unplug_gate", langgraph_input_node(hooks))
38-
3958
tool_guard = langgraph_tool_guard(hooks)
40-
decision = tool_guard("shell_exec", {"command": "..."})
41-
if not decision.allowed:
42-
...
43-
```
44-
45-
Demo (no `langgraph` package required):
46-
47-
```bash
48-
python examples/langgraph_hooks_demo.py
4959
```
5060

51-
## Agno
52-
53-
```python
54-
from agno.agent import Agent
55-
from unplug import Guard
56-
from unplug.integrations.agno import agno_pre_run_hook, agno_post_run_hook, agno_tool_hook
57-
from unplug.integrations.hooks import AgentHooks
58-
59-
hooks = AgentHooks(Guard())
60-
agent = Agent(
61-
pre_hooks=[agno_pre_run_hook(hooks)],
62-
# wire post_run / tool middleware per Agno version
63-
)
64-
```
65-
66-
Demo:
67-
68-
```bash
69-
python examples/agno_hooks_demo.py
70-
```
61+
Demo: `python examples/langgraph_hooks_demo.py`
7162

7263
## Haystack (RAG)
7364

74-
The retrieval path is the blind spot in most injection defenses: a poisoned
75-
document in the store carries its payload straight into the prompt. The
76-
`UnplugDocumentGuard` component sits between the retriever and the prompt
77-
builder. It scans each retrieved `Document`, drops or redacts on findings, and
78-
boundary-wraps survivors so injected text cannot impersonate system
79-
instructions. See [`RAG_DEFENSE.md`](RAG_DEFENSE.md) for the threat model.
80-
8165
```python
82-
from haystack import Pipeline
83-
from unplug.integrations.haystack import UnplugDocumentGuard
84-
85-
pipe = Pipeline()
86-
pipe.add_component("retriever", retriever)
87-
pipe.add_component("guard", UnplugDocumentGuard()) # Guard() under the hood
88-
pipe.add_component("prompt", prompt_builder)
89-
pipe.connect("retriever.documents", "guard.documents")
90-
pipe.connect("guard.documents", "prompt.documents")
66+
from unplug.integrations.haystack import UnplugDocumentGuard, scan_for_ingestion
9167
```
9268

93-
Defend at ingestion too, so poisoned content never reaches the store:
94-
95-
```python
96-
from unplug import Guard
97-
from unplug.integrations.haystack import scan_for_ingestion
98-
99-
decision = scan_for_ingestion(Guard(), document_text)
100-
if decision.index_ok:
101-
document.meta.update(decision.meta_update) # marks it pre-scanned
102-
document_store.write_documents([document])
103-
```
104-
105-
Install: `pip install unplug-ai[haystack]`. The scanning core (`scan_document`,
106-
`scan_for_ingestion`) is import-safe without Haystack; only the component
107-
requires it.
69+
Install: `pip install unplug-ai[haystack]`. See [`RAG_DEFENSE.md`](RAG_DEFENSE.md).
10870

10971
## Hosted vs local
11072

11173
| Setup | Guard init |
11274
|-------|------------|
113-
| Your VM + API key | `Guard(mode="server")` + `UNPLUG_API_KEY` |
114-
| Local sidecar | `Guard(mode="server", server_url="http://127.0.0.1:8000")` |
115-
| Embedded ML | `Guard()` + `active_model=tiny` |
75+
| Hosted API | `Guard(mode="server")` + `UNPLUG_API_KEY` |
76+
| Sidecar | `Guard(mode="server", server_url="http://127.0.0.1:8000")` |
77+
| Embedded | `Guard()` |
11678

117-
See [`DEPLOYMENT.md`](DEPLOYMENT.md).
79+
Tool enforcement always runs locally. See [`DEPLOYMENT.md`](DEPLOYMENT.md).
11880

119-
## Docker E2E (sidecar + examples)
81+
## Security testing
12082

12183
```bash
122-
cd sdk
123-
chmod +x scripts/docker_e2e.sh
124-
./scripts/docker_e2e.sh
125-
# or: make docker-e2e
84+
uv run pytest tests/security/test_agent_integration_matrix.py -v
12685
```
12786

128-
Builds `unplug-server` sidecar, waits for `/v1/health`, runs `local_sidecar_client.py` and `hosted_client.py`.
87+
See [`integrations/TESTING.md`](../integrations/TESTING.md) for all 40 angles.
88+
89+
## Docker E2E (sidecar + examples)
90+
91+
```bash
92+
cd sdk && make docker-e2e
93+
```
12994

130-
## What is not included yet
95+
## MCP
13196

132-
- Official PyPI extras `unplug-ai[langgraph]` / `[agno]` (hooks work without them)
133-
- Auto-instrumentation via `Guard.init()` (manual hooks only)
134-
- MCP / CrewAI / LlamaIndex wrappers (same `AgentHooks` pattern applies)
135-
- LlamaIndex `NodePostprocessor` equivalent of `UnplugDocumentGuard` (same `scan_document` core applies)
97+
For Claude / Cursor MCP server, use [`unplug-mcp`](https://github.com/UnplugAI/unplug-mcp) — see [`integrations/mcp/README.md`](../integrations/mcp/README.md).

‎sdk/integrations/README.md‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Agent framework integrations
2+
3+
Unplug ships **framework-agnostic hooks first**. You do not need LangGraph, CrewAI, or any agent SDK to use the core Guard — install only the extra for the stack you run.
4+
5+
## Quick start
6+
7+
```bash
8+
pip install unplug-ai # core Guard (regex scanners, no agent deps)
9+
pip install "unplug-ai[langgraph]" # + LangGraph when you wire graph nodes
10+
pip install "unplug-ai[integrations]" # all documented agent/RAG extras
11+
pip install "unplug-ai[all]" # integrations + ML + presidio + yara + …
12+
```
13+
14+
Every integration module lives under `unplug.integrations.*` and uses the same five hook points:
15+
16+
| Hook | When to call | Threats caught |
17+
|------|----------------|----------------|
18+
| `scan_user_input` | Before the LLM turn | Direct prompt injection, jailbreaks |
19+
| `wrap_retrieved_content` | After RAG / web fetch | Indirect injection in documents |
20+
| `before_tool_call` | Before every tool | Destructive shell/SQL, exfil, financial |
21+
| `scan_agent_output` | Before returning to user | Leaked secrets, harmful content |
22+
| `scan_request_isolated` | Eval / probes | Same scans without session taint bleed |
23+
24+
```python
25+
from unplug import Guard
26+
from unplug.integrations.hooks import AgentHooks
27+
28+
hooks = AgentHooks(Guard()) # or Guard(mode="server") for hosted API
29+
decision = hooks.scan_user_input(user_message)
30+
if not decision.allowed:
31+
raise RuntimeError(decision.message)
32+
```
33+
34+
## Supported frameworks
35+
36+
| Framework | Extra | Guide | Code module |
37+
|-----------|-------|-------|-------------|
38+
| **Custom loop** | *(none)* | [custom-loop](custom-loop/README.md) | `hooks.py` |
39+
| **LangGraph** | `langgraph` | [langgraph](langgraph/README.md) | `langgraph.py` |
40+
| **Agno** | `agno` | [agno](agno/README.md) | `agno.py` |
41+
| **Haystack** | `haystack` | [haystack](haystack/README.md) | `haystack.py` |
42+
| **LlamaIndex** | `llama-index` | [llama-index](llama-index/README.md) | `llama_index.py` |
43+
| **CrewAI** | `crewai` | [crewai](crewai/README.md) | `crewai.py` |
44+
| **AutoGen** | `autogen` | [autogen](autogen/README.md) | `autogen.py` |
45+
| **Pydantic AI** | `pydantic-ai` | [pydantic-ai](pydantic-ai/README.md) | `pydantic_ai.py` |
46+
| **Semantic Kernel** | `semantic-kernel` | [semantic-kernel](semantic-kernel/README.md) | `semantic_kernel.py` |
47+
| **MCP clients** | `mcp` | [mcp](mcp/README.md) | [unplug-mcp](https://github.com/UnplugAI/unplug-mcp) |
48+
49+
Demos (no framework install required for LangGraph / Agno patterns):
50+
51+
```bash
52+
cd sdk
53+
python examples/langgraph_hooks_demo.py
54+
python examples/agno_hooks_demo.py
55+
```
56+
57+
## Deployment modes
58+
59+
| Mode | Guard init | Tool enforcement |
60+
|------|------------|------------------|
61+
| Embedded (local) | `Guard()` | Local `check_tool_call` |
62+
| Hosted API | `Guard(mode="server", server_url=..., server_api_key=...)` | **Always local** — never delegate tool policy to the network |
63+
| Sidecar | `Guard(mode="server", server_url="http://127.0.0.1:8000")` | Local |
64+
65+
See [`docs/DEPLOYMENT.md`](../docs/DEPLOYMENT.md).
66+
67+
## Security testing
68+
69+
We maintain a **40-angle integration security matrix** exercised in CI. See [TESTING.md](TESTING.md) for the full list and how to run it locally:
70+
71+
```bash
72+
cd sdk
73+
uv run pytest tests/security/test_agent_integration_matrix.py -v
74+
```
75+
76+
## Contributing a new integration
77+
78+
1. Copy [`_template.md`](_template.md)
79+
2. Add hooks in `src/unplug/integrations/<name>.py` (no hard dependency at import time)
80+
3. Add optional extra in `pyproject.toml`
81+
4. Add matrix cases in `tests/security/test_agent_integration_matrix.py`
82+
5. Link from this README
83+
84+
## Related docs
85+
86+
- [`docs/INTEGRATIONS.md`](../docs/INTEGRATIONS.md) — API reference (compact)
87+
- [`docs/RAG_DEFENSE.md`](../docs/RAG_DEFENSE.md) — retrieval-path threat model
88+
- [`docs/DEPLOYMENT.md`](../docs/DEPLOYMENT.md) — local vs hosted vs sidecar

‎sdk/integrations/TESTING.md‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# Integration security matrix
2+
3+
Unplug integration tests cover **40 attack angles** across hook points and framework adapters. The matrix lives in:
4+
5+
```
6+
tests/security/test_agent_integration_matrix.py
7+
```
8+
9+
Run locally (regex-only core, no agent SDK installs required):
10+
11+
```bash
12+
cd sdk
13+
uv run pytest tests/security/test_agent_integration_matrix.py -v
14+
```
15+
16+
Run the full hook + adapter suite:
17+
18+
```bash
19+
uv run pytest tests/integration/test_integrations.py tests/unit/integrations/ -v
20+
```
21+
22+
## Matrix (40 angles)
23+
24+
| # | Category | Scenario | Expected |
25+
|---|----------|----------|----------|
26+
| 1 | User input | Benign question | Allow |
27+
| 2 | User input | Direct injection ("ignore previous instructions") | Block |
28+
| 3 | User input | System prompt exfiltration | Block |
29+
| 4 | User input | DAN-style jailbreak | Block / review |
30+
| 5 | User input | Fake system delimiter | Block |
31+
| 6 | RAG ingest | Poisoned document at write time | Block ingest |
32+
| 7 | RAG ingest | Clean document | Allow ingest |
33+
| 8 | RAG retrieve | Injection in retrieved chunk | Drop / redact |
34+
| 9 | RAG retrieve | Benign retrieved chunk | Wrap + keep |
35+
| 10 | RAG retrieve | Web-sourced external metadata | Flag external trust |
36+
| 11 | Tool | `rm -rf /` shell | Block |
37+
| 12 | Tool | SQL DROP TABLE | Block |
38+
| 13 | Tool | `git push --force` | Block |
39+
| 14 | Tool | File delete path | Block |
40+
| 15 | Tool | Benign search query | Allow |
41+
| 16 | Tool | High-value wire transfer | Block / review |
42+
| 17 | Output | API key in model output | Block / redact |
43+
| 18 | Output | Benign factual answer | Allow |
44+
| 19 | Output | Injection echoed in response | Block |
45+
| 20 | Session | Taint after web fetch blocks risky tool | Block |
46+
| 21 | Session | Isolated scan ignores prior taint | Allow benign |
47+
| 22 | Session | `reset_session` clears taint | Tool allowed after reset |
48+
| 23 | LangGraph | Input node benign | Pass state |
49+
| 24 | LangGraph | Input node injection | RuntimeError |
50+
| 25 | LangGraph | Tool guard destructive | Not allowed |
51+
| 26 | Agno | Pre-run benign | No raise |
52+
| 27 | Agno | Pre-run injection | RuntimeError |
53+
| 28 | Agno | Post-run leak | Block |
54+
| 29 | CrewAI | Task input injection | RuntimeError |
55+
| 30 | CrewAI | Tool guard destructive | Not allowed |
56+
| 31 | AutoGen | User message injection | RuntimeError |
57+
| 32 | AutoGen | Tool guard destructive | Not allowed |
58+
| 33 | LlamaIndex | Postprocessor drops poison node | Empty / filtered |
59+
| 34 | LlamaIndex | Postprocessor keeps benign node | Kept |
60+
| 35 | Pydantic AI | Input validator injection | RuntimeError |
61+
| 36 | Semantic Kernel | Prompt filter injection | RuntimeError |
62+
| 37 | Haystack | Ingest gate poison | Not index_ok |
63+
| 38 | Haystack | scan_document drops block | Dropped |
64+
| 39 | Hooks | wrap_retrieved blocked placeholder | Non-empty safe text |
65+
| 40 | Hooks | Secret-shaped user input | Block / redact |
66+
67+
## CI markers
68+
69+
| Marker | When |
70+
|--------|------|
71+
| *(none)* | Regex core — always runs |
72+
| `@pytest.mark.requires_haystack` | Haystack component tests |
73+
| `@pytest.mark.requires_ml` | ML recall-gate probes |
74+
| `@pytest.mark.requires_integrations` | Tests that import optional agent SDKs |
75+
76+
## Adding a case
77+
78+
1. Add a row to the table above.
79+
2. Add a parametrized entry or test method in `test_agent_integration_matrix.py`.
80+
3. Keep payloads in `_FIXTURES` at the top of the test file for reuse.

0 commit comments

Comments
 (0)