Skip to content

Commit 616d718

Browse files
committed
merge dev into docs move
2 parents 6d0cdae + 68b6095 commit 616d718

8 files changed

Lines changed: 89 additions & 25 deletions

File tree

‎CONTRIBUTING.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ uv sync --dev
1818

1919
| Extra | Enables |
2020
|-------|---------|
21-
| `ml` | ML span model (`Guard.with_tiny()`) |
21+
| `ml` | ML span model (`Guard(model="tiny")`) |
2222
| `yara` | YARA code/SQLi/XSS scanner |
2323
| `presidio` | Presidio PII scanner |
2424
| `litellm` | LLM judge for borderline cases |

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,7 @@ Try it without installing anything: [live demo](https://huggingface.co/spaces/Un
7474
| ML span model `Guard(model="tiny")` | **Preview** ([unplug-tiny-v1](https://huggingface.co/Unplug-AI/unplug-tiny-v1)) |
7575
| Sliding-window long documents + streaming scan | **Included** |
7676

77-
On the neuralchemy prompt-injection set, regex-only detection reaches **F1 0.58 / recall 0.41**, a fast first line, not sufficient alone. Adding the ML span model (`Guard(model="tiny")`) takes that to **F1 0.99 / recall 0.98**, and lifts recall on *indirect* injection from **0.05 → 0.91**. False-positive rate stays under 1% on the injection set (2.1% on a separate hard-benign corpus). Full tables, methodology, and honest caveats: [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md). Per-axis model metrics (including failure modes) are on the [model card](https://huggingface.co/Unplug-AI/unplug-tiny-v1).
77+
On the held-out `core/test` split of the neuralchemy prompt-injection set (942 rows), regex-only detection reaches **F1 0.52 / recall 0.35**, a fast first line and not sufficient alone. Adding the ML span model (`Guard(model="tiny")`) takes that to **F1 0.97 / recall 0.96**, and lifts recall on *indirect* injection from **0.05 to 0.91**. False positives on that split run at 5 of 390 benign rows, and 2 of 95 on a separate hand-written benign corpus. On broader public benign sets the model over-flags badly, up to 34% of a combined 3,227-prompt validation set, so tune the threshold for your own traffic rather than trusting the defaults. Full tables, methodology, and the axes where it fails: [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md). Per-axis model metrics are on the [model card](https://huggingface.co/Unplug-AI/unplug-tiny-v1).
7878

7979
**Language support.** Regex + normalization detection is tuned for English today.
8080
Ordinary non-English input (Cyrillic, CJK, etc.) is *not* treated as an evasion

‎assets/hf-org-card/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ pip install "unplug-ai[ml]"
3636
```python
3737
from unplug import Guard
3838

39-
guard = Guard.with_tiny() # auto-downloads unplug-tiny-v1
39+
guard = Guard(model="tiny",auto_download_model=True) # auto-downloads unplug-tiny-v1
4040
result = guard.scan(untrusted_text)
4141
if not result.safe:
4242
use(result.redacted_text) # attack removed, content preserved

‎docs/BENCHMARKS.md‎

Lines changed: 48 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,12 @@
11
# SDK benchmark results
22

3-
- **Date:** 2026-06-15 (ML rows); regex-only neuralchemy refreshed **2026-07-20** (Phase C)
3+
- **Date:** 2026-08-25
44
- **Guard:** `unplug-ai`, default scanners
55
- **Model:** `unplug-tiny-v1` (DeBERTa-v3-xsmall dual-head span model), `Guard(model="tiny")`
6-
- **Detection threshold:** risk ≥ 0.5 counts as flagged (block or review)
6+
- **Detection threshold:** risk >= 0.5 counts as flagged (block or review)
7+
- **Split:** `neuralchemy/Prompt-injection-dataset` `core/test` (942 rows). `unplug-tiny`
8+
was fine-tuned on `core/train`, so any score measured there is memorisation, not
9+
detection. Earlier revisions of this page reported the train split by mistake.
710
- **Methodology:** isolated single-turn sessions. Each sample is scanned in a fresh
811
`ExecutionContext` (`scan_request(..., isolated=True)`), so multi-turn trajectory
912
state never leaks between independent samples.
@@ -17,13 +20,28 @@ the regex layer alone misses (especially **indirect** injection).
1720

1821
| Dataset | Samples | Mode | F1 | Recall | FPR | Precision |
1922
| --- | ---: | --- | ---: | ---: | ---: | ---: |
20-
| neuralchemy/Prompt-injection-dataset | 4,391 | regex-only | 0.575 | 0.405 | 0.0052 | 0.992 |
21-
| neuralchemy/Prompt-injection-dataset | 4,391 | **regex + ML** | **0.987** | **0.981** | 0.0098 | 0.994 |
22-
| microsoft/llmail-inject (Phase1 subset, attacks) | 2,500 | regex-only | — | 0.052 | — | — |
23-
| microsoft/llmail-inject (Phase1 subset, attacks) | 2,500 | **regex + ML** | — | **0.907** | — | — |
23+
| neuralchemy `core/test` | 942 | regex-only | 0.519 | 0.351 | 0.0026 | 0.995 |
24+
| neuralchemy `core/test` | 942 | **regex + ML** | **0.974** | **0.958** | 0.0128 | 0.991 |
25+
| microsoft/llmail-inject (Phase1 subset, attacks) | 2,500 | regex-only | n/a | 0.052 | n/a | n/a |
26+
| microsoft/llmail-inject (Phase1 subset, attacks) | 2,500 | **regex + ML** | n/a | **0.907** | n/a | n/a |
2427

25-
- **Direct injection (neuralchemy):** recall **0.41 → 0.98**, F1 **0.58 → 0.99**, precision ~0.99 in both modes, false-positive rate stays under 1%.
26-
- **Indirect injection (microsoft):** recall **0.05 → 0.91**. Regex is structurally blind to indirect injection; the ML pass is what makes it detectable.
28+
- **Direct injection (neuralchemy):** recall **0.35 -> 0.96**, F1 **0.52 -> 0.97**.
29+
Precision holds near 0.99 in both modes. The ML pass costs false positives:
30+
FPR goes from 1 false positive in 390 benign rows to 5.
31+
- **Indirect injection (microsoft):** recall **0.05 -> 0.91**. Regex is structurally
32+
blind to indirect injection; the ML pass is what makes it detectable. We have not
33+
confirmed whether any of this subset overlaps `unplug-tiny` fine-tuning data, so
34+
read that row as an upper bound.
35+
36+
## What changed on 2026-08-25
37+
38+
This page used to report `core/train`: 0.987 F1 and 0.981 recall for regex + ML.
39+
That split is the model's own fine-tuning data. On the held-out `core/test` split the
40+
same build scores 0.974 F1 and 0.958 recall, so contamination was worth about 2.3
41+
points of recall.
42+
43+
Regex-only moved too, 0.405 recall down to 0.351, and regex has no training data at
44+
all. Part of the gap is that the two splits are not equally hard, not memorisation.
2745

2846
## False-positive rate on clean traffic
2947

@@ -59,6 +77,28 @@ recall/FPR knee) to keep this rate low without sacrificing recall.
5977
- The microsoft subset is attacks-only (recall, no FPR). neuralchemy carries both
6078
labels.
6179

80+
## Where it does badly
81+
82+
Detection numbers on corpora we did not pick are worse, and the false-positive rate on
83+
broad benign traffic is much worse. From the `unplug-tiny-v1` model card, which runs a
84+
frozen harness over public sets:
85+
86+
| Set | Recall | Doc FPR | F1 |
87+
| --- | ---: | ---: | ---: |
88+
| BIPIA indirect proxy (1,242) | 0.973 | 0.000 | 0.986 |
89+
| InjecGuard validation (144) | 0.896 | 0.208 | 0.775 |
90+
| Deepset full (662) | 0.829 | 0.188 | 0.784 |
91+
| spikee contextual (986) | 0.786 | 0.067 | 0.879 |
92+
| LLM-PIEval agentic (750) | 0.761 | n/a | 0.865 |
93+
| OOD direct injection (281) | 0.619 | 0.102 | 0.692 |
94+
| WildGuard benign (971) | n/a | 0.542 | n/a |
95+
| Combined public validation (3,227) | 0.810 | 0.341 | 0.717 |
96+
97+
One third of benign prompts in the combined public set get flagged. If your traffic
98+
looks like WildGuard rather than like our benign corpus, expect over-blocking, and
99+
tune the threshold before you put this in front of users. The model card carries the
100+
per-axis failure modes and marks three of its own gates as failing.
101+
62102
## Reproduce
63103

64104
```bash

‎docs/EVAL_PHASE_C.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ uv sync --all-extras --dev
1313

1414
# Datasets are already under benchmarks/data/. To refresh from Hugging Face:
1515
uv run python -m benchmarks.download --dataset all --out benchmarks/data
16-
# neuralchemy: full train export; microsoft: streaming Phase1 subset (default --limit 5000)
16+
# neuralchemy: core/test holdout; microsoft: streaming Phase1 subset (default --limit 5000)
1717

1818
# Smoke (built-in samples)
1919
uv run python -c "
@@ -30,6 +30,10 @@ uv run python -m benchmarks.run benchmarks/data/microsoft_indirect.jsonl --isola
3030
uv run python -m benchmarks.run benchmarks/data/neuralchemy.jsonl --ml --isolated --format json
3131
```
3232

33+
> **Superseded.** The neuralchemy rows below were measured on `core/train`, which is
34+
> `unplug-tiny`'s fine-tuning data. They are kept as a record of what that run reported.
35+
> Current numbers are on the `core/test` holdout in [`BENCHMARKS.md`](BENCHMARKS.md).
36+
3337
## Results (this run)
3438

3539
| Dataset | Samples | Mode | Precision | Recall | F1 | FPR |

‎sdk/README.md‎

Lines changed: 15 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -73,15 +73,21 @@ single-turn sessions:
7373

7474
| Dataset | Mode | Recall | F1 | FPR |
7575
| --- | --- | ---: | ---: | ---: |
76-
| neuralchemy (direct, 4,391) | regex-only | 0.41 | 0.58 | <1% |
77-
| neuralchemy (direct, 4,391) | **regex + ML** | **0.98** | **0.99** | <1% |
78-
| microsoft llmail (indirect, 2,500) | regex-only | 0.05 | — | — |
79-
| microsoft llmail (indirect, 2,500) | **regex + ML** | **0.91** | — | — |
80-
81-
Precision stays ~0.99 in both modes. The `<1%` FPR is on the injection set; on a
82-
separate hard-benign corpus (95 prompts) regex flags 0 and regex + ML flags 2,
83-
i.e. **2.1%** (one of which is a *review*, not a block). `inj_threshold` is tuned
84-
to the recall/FPR knee.
76+
| neuralchemy `core/test` (direct, 942) | regex-only | 0.35 | 0.52 | 1/390 |
77+
| neuralchemy `core/test` (direct, 942) | **regex + ML** | **0.96** | **0.97** | 5/390 |
78+
| microsoft llmail (indirect, 2,500) | regex-only | 0.05 | n/a | n/a |
79+
| microsoft llmail (indirect, 2,500) | **regex + ML** | **0.91** | n/a | n/a |
80+
81+
`core/test` is the held-out split. `unplug-tiny` was fine-tuned on `core/train`, so
82+
numbers measured there are memorisation rather than detection. Precision stays near
83+
0.99 in both modes. The FPR column counts benign rows inside the injection set; on a
84+
separate hand-written benign corpus of 95 prompts, regex flags 0 and regex + ML flags 2,
85+
one of which is a *review* rather than a block. `inj_threshold` is tuned to the
86+
recall/FPR knee.
87+
88+
On broader public benign sets the model over-flags badly, up to 34% of a combined
89+
3,227-prompt validation set. See [`docs/BENCHMARKS.md`](../docs/BENCHMARKS.md) before
90+
trusting the defaults.
8591

8692
**Language support.** Regex + normalization detection is tuned for English today.
8793
Ordinary non-English input (Cyrillic, CJK, etc.) is *not* treated as an evasion

‎sdk/benchmarks/download.py‎

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,10 +26,18 @@ def _export_jsonl(samples: list[Sample], path: Path) -> None:
2626
f.write(json.dumps(row) + "\n")
2727

2828

29-
def download_neuralchemy(out_dir: Path, *, limit: int | None = None) -> Path:
29+
# unplug-tiny was fine-tuned on the "train" split of this dataset, so scoring the
30+
# model against it measures memorisation. "test" is the held-out split the model
31+
# card gates on. Do not change this default.
32+
NEURALCHEMY_SPLIT = "test"
33+
34+
35+
def download_neuralchemy(
36+
out_dir: Path, *, limit: int | None = None, split: str = NEURALCHEMY_SPLIT
37+
) -> Path:
3038
from datasets import load_dataset
3139

32-
ds = load_dataset("neuralchemy/Prompt-injection-dataset", split="train")
40+
ds = load_dataset("neuralchemy/Prompt-injection-dataset", "core", split=split)
3341
samples: list[Sample] = []
3442
for i, row in enumerate(ds):
3543
if limit is not None and i >= limit:
@@ -90,12 +98,18 @@ def main() -> None:
9098
)
9199
parser.add_argument("--out", type=Path, default=None)
92100
parser.add_argument("--limit", type=int, default=None)
101+
parser.add_argument(
102+
"--neuralchemy-split",
103+
default=NEURALCHEMY_SPLIT,
104+
choices=["train", "validation", "test"],
105+
help="train overlaps unplug-tiny fine-tuning data; results are not comparable",
106+
)
93107
args = parser.parse_args()
94108

95109
out_dir = args.out or _repo_datasets_dir()
96110
paths: list[Path] = []
97111
if args.dataset in ("neuralchemy", "all"):
98-
paths.append(download_neuralchemy(out_dir, limit=args.limit))
112+
paths.append(download_neuralchemy(out_dir, limit=args.limit, split=args.neuralchemy_split))
99113
if args.dataset in ("microsoft", "all"):
100114
paths.append(download_microsoft_subset(out_dir, limit=args.limit or 5000))
101115
for p in paths:

‎sdk/demo/requirements.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
gradio>=6.22.0
1+
gradio>=6.24.0
22
--extra-index-url https://download.pytorch.org/whl/cpu
33
torch>=2.0
44
transformers>=4.44,<5.16

0 commit comments

Comments
 (0)