You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 616d718
Browse filesBrowse the repository at this point in the historyBrowse files
Copy file name to clipboardExpand all lines: README.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -74,7 +74,7 @@ Try it without installing anything: [live demo](https://huggingface.co/spaces/Un
74
74
| ML span model `Guard(model="tiny")`|**Preview** ([unplug-tiny-v1](https://huggingface.co/Unplug-AI/unplug-tiny-v1)) |
75
75
| Sliding-window long documents + streaming scan |**Included**|
76
76
77
-
On the neuralchemy prompt-injection set, regex-only detection reaches **F1 0.58 / recall 0.41**, a fast first line, not sufficient alone. Adding the ML span model (`Guard(model="tiny")`) takes that to **F1 0.99 / recall 0.98**, and lifts recall on *indirect* injection from **0.05 → 0.91**. False-positive rate stays under 1% on the injection set (2.1% on a separate hard-benign corpus). Full tables, methodology, and honest caveats: [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md). Per-axis model metrics (including failure modes) are on the [model card](https://huggingface.co/Unplug-AI/unplug-tiny-v1).
77
+
On the held-out `core/test` split of the neuralchemy prompt-injection set (942 rows), regex-only detection reaches **F1 0.52 / recall 0.35**, a fast first line and not sufficient alone. Adding the ML span model (`Guard(model="tiny")`) takes that to **F1 0.97 / recall 0.96**, and lifts recall on *indirect* injection from **0.05 to 0.91**. False positives on that split run at 5 of 390 benign rows, and 2 of 95 on a separate hand-written benign corpus. On broader public benign sets the model over-flags badly, up to 34% of a combined 3,227-prompt validation set, so tune the threshold for your own traffic rather than trusting the defaults. Full tables, methodology, and the axes where it fails: [`docs/BENCHMARKS.md`](docs/BENCHMARKS.md). Per-axis model metrics are on the [model card](https://huggingface.co/Unplug-AI/unplug-tiny-v1).
78
78
79
79
**Language support.** Regex + normalization detection is tuned for English today.
80
80
Ordinary non-English input (Cyrillic, CJK, etc.) is *not* treated as an evasion
-**Direct injection (neuralchemy):** recall **0.41 → 0.98**, F1 **0.58 → 0.99**, precision ~0.99 in both modes, false-positive rate stays under 1%.
26
-
-**Indirect injection (microsoft):** recall **0.05 → 0.91**. Regex is structurally blind to indirect injection; the ML pass is what makes it detectable.
0 commit comments