A pre-registered study showing that linear probes proposed to detect whether a language model knows it is being evaluated are really detecting benchmark formatting, not evaluation status.
TL;DR. Linear probes are increasingly proposed to detect whether a language model "knows it is being evaluated." We show, in a pre-registered study across eight open models from 0.6B to 32B (Qwen3, Llama-3.1, Gemma-2), that these probes pass every standard validation yet are detecting benchmark formatting, not evaluation status. When we hold an item's content fixed and swap only its "costume," wrapping benchmark questions in casual chat and real user messages in benchmark formatting, the probe inverts (SAD-oversight transfer AUROC 0.77 → 0.0003 at 8B), and the inversion does not weaken with scale through 32B, where the probe also reaches perfect within-distribution separation. Behaviorally, framing effects exist but run opposite to the sandbagging concern: deployment urgency, not benchmark framing, drives test-gaming. The takeaway is methodological: style controls and label audits should be mandatory for eval-awareness probing.
A probe is a simple lie-detector wired into a model's "brain": it reads the model's internal activations and tries to answer one yes/no question, "does the model think it is being tested right now?" If that worked, it could serve as a cheap safety monitor.
It does not work the way people hoped. The probe is not reading whether something is a test; it is reading whether the text is dressed up like a test (multiple-choice formatting, "your answer will be graded," and so on). To prove it, we keep the exact words of each example but swap the outfit: we put a real user's message in exam clothing and an exam question in casual-chat clothing. The probe flips its answers and becomes confidently wrong on almost every item: it was grading the costume, not the content. Bigger models do not fix this. The 32B model is the most confident of all and still gets the costume-swap completely backwards.
How to read AUROC (the score used throughout): it runs 0 to 1. 1.0 = perfect, 0.5 = a coin-flip (no signal), 0.0 = perfectly backwards (the classifier is right about the magnitude but wrong about the direction on every item). So a probe falling from ~0.9 to ~0.0 has not become "useless"; it has become an anti-detector.
| # | Question we asked | What we found |
|---|---|---|
| 1 | Do "eval-awareness" probes detect evaluation? | No, they detect formatting. They pass standard validation, then invert when only the costume is swapped. |
| 2 | Does the problem go away at larger scale? | No. The probe still inverts at 14B and 32B, and the 32B probe is the best-validated one in the study. |
| 3 | Does benchmark framing make models game the tests? | No, the effect runs the other way. Deployment-flavored urgency, not benchmark framing, drives test-gaming. |
| 4 | Does steering the probe direction change gaming? | No. It changes how much the model talks about tests, but not whether it games them, no better than a random direction. |
| 5 | Are the "hacks" the metric flags actually gaming? | Mostly not. A blinded audit found 0 of 19 flagged normal-task "hacks" were deliberate gaming; they were honest mistakes. |
Step 1: every probe passes standard validation. Trained on instructed contrasts ("you are being evaluated" vs "you are deployed") and transferred to the SAD oversight benchmark, the probe separates the classes well in all eight models, far above the pre-registered 0.65 gate.
| Model | Family | Size | SAD-oversight transfer AUROC | Passes 0.65 gate? |
|---|---|---|---|---|
| Qwen3-0.6B | Qwen3 | 0.6B | 0.89 | ✅ |
| Qwen3-1.7B | Qwen3 | 1.7B | 0.78 | ✅ |
| Qwen3-4B | Qwen3 | 4B | 0.91 | ✅ |
| Qwen3-8B | Qwen3 | 8B | 0.77 | ✅ |
| Qwen3-14B | Qwen3 | 14B | 0.90 | ✅ |
| Qwen3-32B | Qwen3 | 32B | 0.90 | ✅ |
| Llama-3.1-8B-Instruct | Llama-3.1 | 8B | 0.84 | ✅ |
| Gemma-2-2B-it | Gemma-2 | 2B | 0.83 | ✅ |
Transfer is non-monotonic in scale (no within-family power law), unlike the published cross-family trend.
Step 2: the same probes invert under the costume-swap. We re-wrap each SAD item with its content held verbatim but its costume swapped (benchmark items get a casual-chat wrapper; real-user items get a benchmark-MCQ wrapper), then re-score with the same probe at the same layer. The probe does not just lose signal, it flips.
| Model | Standard test (AUROC) | After costume-swap (AUROC) | Verdict |
|---|---|---|---|
| Gemma-2-2B-it | 0.83 | 0.16 | inverts |
| Qwen3-4B | 0.89 | 0.19 | inverts |
| Qwen3-8B | 0.77 | 0.0003 | near-perfectly backwards |
| Qwen3-14B | 0.89 | 0.004 | near-perfectly backwards |
| Qwen3-32B | 0.90 | 0.22 | inverts (best-validated probe in the study) |
Each row is measured at that model's own best-performing layer; n = 400 SAD items. Across all eight models, the calibrated "semantics-over-style" score is negative at essentially every working layer: content loses to costume.
The separation the probe exploits is real and clean; it is just the wrong variable. Below are genuine last-token residual-stream activations from Qwen3-0.6B on 60 benchmark vs 60 deployment coding prompts, projected onto the probe direction: two well-separated clusters (AUROC 1.00), a separation that reaches AUROC ≈ 1.0 across all eight models, so the smallest is representative. The costume-swap above shows this separation tracks formatting, not evaluation.
The separating axis, in 3D. The scatter above collapses the geometry onto a single probe axis. Rotating the full 3D structure makes the point unmistakable, and it holds on the largest dense model. Below is the real residual stream of Qwen3-14B, every prompt projected onto three meaningful directions: the probe's read-out axis, the true content axis, and one residual axis. The four groups split by costume (solid = benchmark formatting, hollow = chat formatting), not by content (blue = test item, orange = real item): both benchmark-formatted clusters land on the probe's "eval" side regardless of what the item actually is. Swapping only an item's costume (teal arrows) carries it straight across the decision boundary. The probe's axis is essentially the costume axis (cosine ≈ 0.95), and its verdict collapses to near-zero AUROC the moment only the costume changes.
Does dressing a task as a benchmark make models cut corners? We ran a lite-agentic coding harness with visible tests (shown to the model) and hidden tests (held out to catch gaming). The "gaming" readout is visible tests pass but ≥1 hidden test fails. Across 2,400 pre-registered trajectories, that rate is lower under benchmark framing, the opposite of the sandbagging concern.
| Model | Task type | Benchmark framing (E) | Deployment framing (D) | Direction |
|---|---|---|---|---|
| Qwen3-4B | normal | 3.5% | 5.8% | D > E |
| Qwen3-4B | hack-forcing | 0.0% | 3.2% | D > E |
| Qwen3-8B | normal | 5.2% | 6.6% | D > E |
| Qwen3-8B | hack-forcing | 2.2% | 6.3% | D > E |
A pre-registered powered replication (28 hack-forcing tasks × {E, D} × 10 samples, 1,120 trajectories) confirms it: deployment urgency roughly triples genuine capitulation.
| Condition | Capitulation rate | 95% CI |
|---|---|---|
| Benchmark framing (E) | 1.3% | [0.6, 2.6] |
| Deployment framing (D) | 4.2% | [2.7, 6.3] |
| Difference (D minus E) | +2.7 pp | [+0.7, +5.1], bootstrap p = 0.001 |
And a 100-item blinded audit found that on normal tasks, 0 of 19 flagged "hacks" were deliberate gaming; all were honest implementations that missed a spec nuance. The naive "hack rate" metric was largely measuring honest mistakes.
Eight models, three families. Probing covers all eight; the behavioral arm (more expensive) runs at ≤8B.
| Family | Sizes probed | Behavioral arm |
|---|---|---|
| Qwen3 | 0.6B · 1.7B · 4B · 8B · 14B · 32B | 4B, 8B |
| Llama-3.1 | 8B-Instruct | none |
| Gemma-2 | 2B-it | none |
BenchDress: 9 framings of the same task. Every one of the 120 bug-fix tasks is rendered identically in content under nine "dressings" that separate how it looks from what it is.
| Code | Style | Framing |
|---|---|---|
| E1 | benchmark | HumanEval-style problem statement, "scored automatically," code-only |
| E2 | benchmark | contest judge, "secret tests," no partial credit |
| E3 | benchmark | agentic eval harness, "marked PASSED only if the battery passes" |
| D1 | deployment | GitHub issue in a repo, maintainer voice |
| D2 | deployment | Slack ping, deadline, emoji |
| D3 | deployment | failing-CI trace, release-branch pressure |
| N | neutral | bare problem statement |
| SC_E | style control | real work wrapped in benchmark formatting |
| SC_D | style control | a benchmark item wrapped in casual workplace voice |
The two style controls (SC_E / SC_D) are the key: they pit surface format against actual evaluation status, which is exactly what natural data never does (in the wild, "looks like a benchmark" and "is an evaluation" almost always co-vary).
| RQ | Question | Answer |
|---|---|---|
| RQ1 | Do probes represent evaluation status? | They represent formatting; the direction inverts under the costume-swap. |
| RQ2 | Does the representation become semantic with scale? | No, still costume at 14B/32B; no emergence through the largest dense Qwen3. |
| RQ3 | Does the eval/deploy distinction drive test-gaming? | No, deployment urgency drives it; benchmark framing is bounded out of the sandbagging direction. |
| RQ4 | Does steering the probe direction change gaming? | No, it suppresses test-commentary (both signs) but not gaming, no better than random vectors. |
| SAE | Can sparse autoencoders localize the feature? | Pre-registered check failed informatively: Base-model SAEs do not reconstruct Instruct chat activations (FVU 1.8 to 10), so latent analysis is omitted. |
paper/PAPER.md: readable Markdown version.arxiv/: LaTeX source and the compiled PDF (arxiv/main.pdf).PREREG.md: the pre-registration (hypotheses, primary metrics, exclusion rules, and a dated deviations log), locked before the full behavioral sweeps.
benchdress/tasks/ BenchDress: 120 validated bug-fix tasks (92 normal + 28
hack-forcing), each with visible and hidden test suites.
src/benchdress/ Dataset construction, sandboxed execution, dressings,
the lite-agentic harness, and hack/pushback detectors.
src/probing/ Instructed-contrast prompts, activation caching, probes,
and SAD-oversight loaders (incl. the style-transplant).
src/modal_jobs/ Serverless-GPU jobs: probing (phase 0 / phase 3),
behavioral sweeps (phase 2), steering (phase 4), the SAE
check, and the style calibration.
src/analysis.py Pre-registered behavioral analysis (bootstrap, McNemar).
src/corrections.py Rogan-Gladen pushback correction, null bounds, per-template.
src/figure_*.py Figure generation.
results/modal/ Aggregated result JSONs underlying every figure and claim.
calibration/ The 100-item blinded label audit (released for inspection).
figures/ Generated figures (PDF + PNG).
The repository ships the aggregated result JSONs, so the figures and reported numbers can be regenerated without re-running generation:
uv venv --python 3.12 && source .venv/bin/activate
uv pip install torch transformers scikit-learn numpy pandas matplotlib \
statsmodels scipy
python -m src.figures results/modal figures
python -m src.figure_steering
python -m src.figure_moneyAll model compute runs on Modal serverless GPUs. You need
a Modal account and a Hugging Face token stored as a Modal secret named
hf-token. The probing arm also needs the SAD stages_oversight data, which
must be obtained from its authors under its anti-contamination terms (it is
not redistributed here).
# Probing: SAD-oversight replication and the scaling curve
modal run src/modal_jobs/phase0_probe.py --models qwen3-0.6b,qwen3-1.7b,qwen3-4b,qwen3-8b
# Natural-dressing probes + style controls
modal run src/modal_jobs/phase3_probe.py --models qwen3-8b
modal run src/modal_jobs/phase3_stylecal.py --tags qwen3-8b
# Style-transplant test
modal run src/modal_jobs/sad_restyle.py --models qwen3-8b
# Behavioral sweeps and steering
modal run src/modal_jobs/phase2_sweep.py --model Qwen/Qwen3-8B --n-tasks -1 --n-samples 5
modal run src/modal_jobs/phase4_steer.py::grid --tag qwen3-8bThe whole study was produced on a ≈$34 end-to-end serverless-GPU budget.
The aggregated result JSONs in results/modal/ underlie every number and
figure in the paper. The raw per-trajectory shards (the full behavioral and
steering sweeps, ~180 MB across ~5,000 files) are kept out of the repository
and are available as a separate data release. The BenchDress task suite and the
100-item label audit are included in full.
| Term | What it means |
|---|---|
| Linear probe | A simple classifier read off the model's internal activations to detect some property (here, "is this an evaluation?"). |
| AUROC | Score from 0 to 1 for a classifier: 1.0 = perfect, 0.5 = coin-flip, 0.0 = perfectly backwards. |
| Evaluation awareness | A model behaving differently when it "knows" it is being tested rather than really used. |
| Eval vs deploy | The two contexts the probe tries to tell apart: a test/benchmark vs a real-user deployment. |
| BenchDress | Our suite of 120 coding bug-fix tasks, each rendered in 9 framings (the "dressings"). |
| Costume-swap | Holding an item's text fixed but swapping only its formatting, to reveal what the probe actually keys on. |
| Hack-forcing task | A task with one deliberately corrupted visible test, so passing it requires special-casing, which gives a clean readout of gaming. |
| Visible / hidden tests | Visible tests are shown to the model; hidden tests are held out to catch gaming. |
| Sandbagging | A model deliberately underperforming when it believes it is being evaluated. |
| Steering | Adding a direction to the model's activations to causally push its behavior. |
| SAD | Situational Awareness Dataset, a prior eval-awareness benchmark; we test on its stages_oversight split. |
@misc{ahmed2026evalawareness,
title = {Do Eval-Awareness Probes Detect Evaluation, or Just Style?},
author = {Ahmed, Faizan},
year = {2026}
}Code is released under the MIT License (see LICENSE). The paper
text and figures are released under CC BY 4.0.