feat(benchmarks): gate document-embedded benign FPR by chiruu12 · Pull Request #202 · UnplugAI/Unplug

chiruu12 · 2026-10-07T08:47:41Z

Adds the benign slice #189 asked for, measuring the axis that actually moves.

#189 filed this as encoded benign text. It is not the encoding. Measured on checkpoint 19b7d6701bea, the same 95 benign rows from benign_ci.jsonl:

carrier alone, no payload                 0.0000
bare text                                 2/95   0.0211
plain text inside the document carrier   48/95   0.5053
base64 of that text in the same carrier  48/95   0.5053

Plain and base64 are the same number. Wrapping a short benign passage in an ordinary business document is what takes the rate from 0.0211 to 0.5053; encoding it adds nothing on top. A slice built on the encoding would pin a constant that does not move when the thing it names changes, so this commits the document shape instead.

Every flagged row comes from injection/span_model. The doc head is not the cause.

What is in here

benign_carrier_ci.jsonl, the 95 easy benign rows embedded in one dull operations summary. CARRIER_TEMPLATE lives in ci_gate.py so the corpus is reproducible from benign_ci.jsonl, and a test rebuilds it and compares.

The slice follows the hard slice's shape: CARRIER_FPR_RATCHET pinned at the measured 0.5053 so one more misfire fails, and CARRIER_FPR_TARGET at 0.0211, the rate those rows score bare, because wrapping a benign passage in a document does not make it less benign. The target is reported and does not gate.

The gate was measuring a configuration nobody ships

Worth reading before the rest. run_gate built its guards through evaluate's default, which is Guard() with no model. Without a checkpoint on disk the ML scanners never load, so every benign slice scores against regex alone:

easy_fpr  0.0000  with default Guard()      <- what CI has been measuring
easy_fpr  0.0211  with Guard(model="tiny")  <- the configuration that ships
EASY_FPR_CEILING  0.02

So the easy ceiling has been green on a path that skips the model, and the shipped path is over it. This PR does not change that ceiling, because moving it would bury the finding. make attack-gate now goes red on a machine with weights cached, and that red is the regression, not this branch. Filed separately as the ceiling is yours to call.

The gate now resolves weights once, reports which configuration produced the numbers, and refuses to pass a slice it could not score. --require-weights fails rather than reporting an unscored slice as fine, wired up as make attack-gate-ml and added to test-all-local. CI has no weights by design and stays green, with the carrier slice printed as n/a and the reason next to it.

Also

All four slices now score with isolated_requests=True: one guard per slice, a fresh ExecutionContext per sample. The old default rebuilt the guard per sample and reloaded the checkpoint 95 times. Identical counts on all four slices, and the harness suite goes from 6 minutes to 55 seconds.

.gitignore needed a negation for the new corpus. sdk/benchmarks/data/*.jsonl is ignored with one exception per committed file, so without it the gate would fail on a fresh clone with a missing corpus.

Checks

ruff check, ruff format --check, mypy (128 files) clean. 26 tests in test_attack_harness.py, 213 across tests/security plus test_evaluation.py. check_doc_drift.py no drift, uv lock --check clean. The five weights-backed tests carry requires_ml_weights and real_model_cache, so they skip in CI and run locally.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

github-actions · 2026-10-07T08:50:57Z

SDK Coverage •

File	Stmts	Miss	Cover	Missing
src/unplug
guard.py	383	26	93%	146, 206–207, 209, 218–219, 310, 323, 326, 331, 474, 476, 637, 775–779, 799–800, 802–805, 811, 830
TOTAL	7525	891	88%

Tests	Skipped	Failures	Errors	Time
1377	31 💤	0 ❌	0 🔥	58.294s ⏱️

greptile-apps · 2026-10-07T08:53:45Z

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge, with one remaining non-blocking concern about a test rejecting model improvements.

Findings

Better model fails tests ▶

Summary

Adds a committed corpus of benign passages inside business documents and measures its false-positive rate.

The gate measures benign prompts inside an ordinary business document.
The gate reports whether its corpus results used the tiny model.

_{Reviews (3) · Last reviewed commit: "fail a checkpoint that will not load, ke..." · Reviewed by Greptile}

greptile-apps · 2026-10-07T08:53:49Z

+        assert wrapped_fpr > bare_fpr * 5, (bare_fpr, wrapped_fpr)
+        assert encoded_fpr == wrapped_fpr, (wrapped_fpr, encoded_fpr)


Better model fails tests

These assertions require wrapped benign text to score more than five times worse than bare text, and encoded text to have exactly the same false-positive rate as wrapped text. If a model reduces wrapped false positives or improves one representation independently, this test fails even after the ratchet is updated. Keep these measurements as baseline evidence, or test an upper limit rather than requiring today's bad behavior.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

chiruu12 · 2026-10-14T15:42:25Z

Three of the four were real. Fixed in e95e372, rebased onto dev now that #201 has landed.

Wrong checkpoint gets scored. Right, and there was no clean way to fix it: nothing on Guard could tell you which checkpoint it had loaded, so the gate had to guess with a second lookup that reads UNPLUG_TEST_CHECKPOINT before UNPLUG_MODEL_PATH before the cache. Added Guard.active_checkpoint, and the gate now builds the release guard first and reports what that guard loaded. The same instance scores every slice, so the reported path is the measured path by construction rather than by agreement.

Empty carrier slice still passes. This one was bad. The report printed FAIL for the slice and the process exited 0, which is the exact failure the rest of this file is written to prevent. Three states were collapsed into two. A corpus with no usable rows now fails whatever the configuration, a slice that could not be scored for want of weights still does not fail the default run, and --require-weights is where a release refuses it. Two tests, one for an empty file and one for a corpus whose rows are all mislabelled, since those reach the same state by different routes.

Matrix report names wrong configuration. Also right. run_matrix already took a guard_factory and I was not passing it, so the matrix ran regex-only under a report claiming the whole run used the model. It takes the factory now. Checked first that it does not change the outcome: passed=True plain=3/3 regressions=[] either way.

Better model fails tests. Half right, and your half matters more than mine. wrapped > bare * 5 would have gone red the day someone fixed the span head, which makes it a test arguing for the bug. Dropped. Kept encoded == wrapped, because that equality is the finding this corpus exists for: if encoding ever starts changing the rate then the premise behind the slice is wrong, and a failing test is how I would want to hear it. The current rate stays pinned by CARRIER_FPR_RATCHET, which is what to lower when detection improves.

chiruu12 · 2026-10-15T15:18:44Z

Two of three, in c825537. The third is reviewing code that is no longer there.

Broken model still passes. Yes, and it is the same shape as the empty-slice one from the last round, which is not a good look two rounds running. _release_guard caught everything and returned None, so a checkpoint that was found and would not load reported "no checkpoint on disk", fell back to regex scanners and passed. It returns the load error alongside the guard now, and a found-but-broken checkpoint fails the gate with the error printed. Two tests: one for the broken case, one asserting absent weights still pass, since CI has none by design and that has to stay a pass or the gate gets switched off.

Matrix repeatedly loads weights. Measured it before changing anything: 45 scans, 19.2 seconds against 0.0. Your second option is the better one and I should have taken it last round. The matrix keeps its default guards, because it is a normalizer regression check and the normalizers sit below the model, so the model buys it nothing. The report now names the slice configuration and the matrix configuration separately instead of one run-level label that was only true of the slices. Gate run is back to 16.6 seconds.

Better model fails tests. This one is stale. wrapped > bare * 5 went in the last round, which is the change you asked for. Line 380 is now a bare-rate assertion that claims nothing, and I have dropped that too, so the only assertion left is encoded == wrapped. Keeping that one on purpose: the equality is the finding the corpus exists for, and if encoding ever starts moving the rate then the premise is wrong and I want a red test rather than a quiet one.

Unchanged and deliberate: EASY_FPR_CEILING stays at 0.02 while the model-backed rate is 0.0211, so a weights-backed run fails. That regression predates this branch and is tracked in #200.

Copilot AI balanced review requested due to automatic review settings October 7, 2026 08:47

Copilot AI reviewed Oct 7, 2026

github-actions Bot added the ci Build, workflows, or tooling label Oct 7, 2026

greptile-apps Bot reviewed Oct 7, 2026

View reviewed changes

chiruu12 added 2 commits October 7, 2026 14:34

gate document-embedded benign FPR

3fa8719

report the checkpoint that scored, fail an empty slice

e95e372

chiruu12 force-pushed the fix/189-carrier-benign-gate branch from 3af688e to e95e372 Compare October 14, 2026 15:42

greptile-apps Bot reviewed Oct 14, 2026

View reviewed changes

Comment thread sdk/benchmarks/attacks/ci_gate.py Resolved

Comment thread sdk/benchmarks/attacks/ci_gate.py Resolved

fail a checkpoint that will not load, keep the matrix cheap

c825537

chiruu12 merged commit 6e2bc37 into dev Oct 15, 2026
7 checks passed

chiruu12 deleted the fix/189-carrier-benign-gate branch October 15, 2026 15:23

chiruu12 mentioned this pull request Oct 15, 2026

The span head fires on benign text embedded in a document #189

Open

Navigation Menu

Repository navigation

feat(benchmarks): gate document-embedded benign FPR - #202

feat(benchmarks): gate document-embedded benign FPR#202
chiruu12 merged 3 commits into
devfrom
fix/189-carrier-benign-gate

chiruu12 commented Oct 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions Bot commented Oct 7, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Oct 7, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot Oct 7, 2026

Uh oh!

Uh oh!

chiruu12 commented Oct 14, 2026

Uh oh!

Uh oh!

Uh oh!

chiruu12 commented Oct 15, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		assert wrapped_fpr > bare_fpr * 5, (bare_fpr, wrapped_fpr)
		assert encoded_fpr == wrapped_fpr, (wrapped_fpr, encoded_fpr)

Conversation

chiruu12 commented Oct 7, 2026

What is in here

The gate was measuring a configuration nobody ships

Also

Checks

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Findings

Summary

Uh oh!

Uh oh!

Uh oh!

greptile-apps Bot Oct 7, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

chiruu12 commented Oct 14, 2026

Uh oh!

Uh oh!

Uh oh!

chiruu12 commented Oct 15, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Oct 7, 2026 •

edited

Loading

greptile-apps Bot commented Oct 7, 2026 •

edited

Loading