Steer a language model by amplifying blocks of a sparse dictionary — with the edit size calibrated in units of the model's own residual norm, and a preregistered head-to-head against SAE features, diff-of-means, and three families of nulls to say whether it actually works.
The dictionary is a Grassmannian block-sparse featurizer — the method introduced by Goodfire and collaborators in Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds (Fel, Kowal, Jacobs, Hazra, Bhalla et al., 2025), whose reference implementation this repo trains and steers with. Their work is on vision models; here the same featurizer is retrained on text — 4096 blocks of 3 dimensions each, 32 active per token, fitted to layer-14 residuals of Qwen3.5-2B-Base — and then labeled block-by-block so you can search it in plain English. The demo below turns a concept into a block set, applies a dose-exact edit during free-running generation, and shows you the steered and unsteered continuations side by side.
- Apple Silicon. Generation runs on MLX; there is no CUDA or CPU path.
- Python 3.12,
uv, andgit. - ~4.3 GB for
Qwen/Qwen3.5-2B-Base, downloaded from Hugging Face on first launch. - ~96 MB of demo assets (checkpoint + labeled blocks), fetched by a script below.
- ~10 GB of free disk if you also intend to reproduce the training or the evaluation.
git clone https://github.com/scasella/bsf-steering
cd bsf-steering
# the featurizer library (MIT, Goodfire) — gitignored, expected at the repo root
git clone https://github.com/goodfire-ai/block-sparse-featurizer
# the model backend + web app the demo mounts (Apache-2.0) — expected as a sibling,
# or anywhere you like if you export QWEN_SCOPE_LAB=/path/to/it
git clone https://github.com/scasella/qwen-scope-lab ../qwen-scope-lab
uv venv --python 3.12 .venv
VIRTUAL_ENV=.venv uv pip install torch==2.13.0 torchvision==0.28.0 transformers==5.15.0 \
numpy==2.5.2 matplotlib==3.11.1 scikit-learn einops pytest datasets
VIRTUAL_ENV=.venv uv pip install -e block-sparse-featurizer --no-deps
VIRTUAL_ENV=.venv uv pip install -e '../qwen-scope-lab[mlx]'
.venv/bin/python qwen/fetch_demo_assets.py # ~96 MB, hash-checked
PYTORCH_ENABLE_MPS_FALLBACK=1 .venv/bin/python qwen/serve_bsf_demo.pyThen open http://127.0.0.1:7871/bsf. First launch pauses while the model downloads;
after that a 64-token side-by-side takes a few seconds. Run it from the repo root — the
mounted lab app creates its monitors/ and probes/ directories relative to the
working directory.
Pick what to steer. Five prepared concepts (ocean, medical, formal, positive, sycophancy) resolve to 8-block sets by the frozen selection procedure. Or type anything into the search box — it searches all 4092 labeled blocks ("the noun 'sea'", "statutory legal-text phrasing", "racing-driver surnames") and lets you click blocks into an ad-hoc set of your own.
Set the dose. δ is the size of the injected edit as a fraction of the median non-sink residual norm at layer 14 (10.267). δ=0.5 is a nudge; δ=1.2 is a shove comparable to the residual itself; δ=2.4 breaks fluency on purpose. The edit is scalar-rescaled so the realized ‖Δx‖ hits the target exactly, and the per-token strip under the block table shows what actually landed. Edits apply from the final prompt position onward — the prefill chunk holding the attention sink is never touched.
Tune the decoding. Temperature, top-p and repetition penalty (defaults 0.7 / 0.9 / 1.1) plus a dose schedule: a decay half-life that eases the edit off over the generation, and a drift cap that skips the edit on any step whose residual has run away. The eval-greedy link resets all of it to the frozen evaluation's exact settings.
Worth trying first: medical, promote, δ=1.2 with decay half-life 16, then the same
thing again after clicking eval-greedy. Under greedy constant dose that setting collapses
into "The patient is a patient of the patient…"; with sampling and decay it stays prose.
That contrast is the honest picture of how sharp this steering is.
7,714 preregistered cells, every gate enforced, scored on held-out lexicons verified disjoint from the words used to select the blocks. Full writeup, including the numbers behind each line: README_LM_STEERING.md.
- One clean positive. Medical promote at δ=1.2 beats the 8-feature SAE arm (+0.080 hits/word, 95% CI [+0.032, +0.130]), the single-feature SAE baseline, diff-of-means, and the band-matched random-max null. It travels with real caveats: 40% of continuations at that dose degenerate into loops, and the direction-ablation CI touches zero — the effect is carried by which blocks, not by the concept-mean directions.
- The SAE arm failed its liveness gate on every concept, which indicts the shared selection procedure on that dictionary rather than SAE steering as such. Said plainly because a "win" against a dead baseline is not a win.
- Ocean is null on held-out words. The pilot's apparent success was selection-lexicon echo: steering produces "sea" and "ship" — words used to build the arm — but never "trawler" or "estuary". Catching that is the single most valuable thing the design review added.
- Formal is real for common register words, absent for rare ones. Positive is indistinguishable from a random vector. Sycophancy moves nothing anywhere. Suppression saturates against the per-token clamp and produces no held-out effect.
- The 3×-natural-norm threshold prediction is confirmed for medical and fragile elsewhere.
Reconstruction, as a sanity check on the dictionary: held-out R² 0.7850 against the Qwen-Scope SAE's 0.7838 on identical tokens, with 2.7× fewer directions.
The research process is part of the artifact. Everything below is the evidence trail, not the product — including the parts that went badly.
| PREREG.md | The Stage-4 design, frozen before a single confirmatory cell existed |
| DESIGN_stage4.md | The adversarial review that rewrote it first — "how does this design pass without the phenomenon being real?" |
| qwen/results_lm/ | All 7,714 per-cell records with per-token realized doses, plus the three fresh-context close-out audits (DEBRIEF, ADVERSARY, VERIFY) |
| OPEN_LOOPS.md | The ledger: every deviation, supersede and anomaly, dated |
| POSTMORTEM.md · SPEC.md | Post-mortem of the earlier vision sprint, and the spec that closed every finding |
| README_RESULTS.md | That vision sprint: BSF on DINOv3 patch activations, where the baselines mostly won |
Superseded analyses are kept with a _SUPERSEDED suffix rather than deleted, so the
corrections are auditable too.
README_LM_STEERING.md has the full pipeline — corpus build, residual dump (~16 min),
BSF training (~4 h CPU), block labeling, dose calibration, the 3 h evaluation grid, and
the analyzer. Two honest caveats: the block labels came from a 64-agent labeling pass
that has no committed script (they ship in the assets tarball instead), and regenerating
the concept sets without them yields enrichment-only sets that differ from the
preregistered ones.
Tests: .venv/bin/pytest qwen/test_lm_steering.py test_steer.py -q, plus
.venv/bin/python qwen/battery_metrics.py for the 14-case instrument validation.
The featurizer itself — the block-sparse formulation, the Grassmannian variant with
orthonormal per-block frames, the block-TopK sparsity mechanism, and the reference
implementation in the bsf package — comes from Goodfire
and their academic collaborators:
Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds Thomas Fel, Matthew Kowal, Mozes Jacobs, Dron Hazra, Usha Bhalla (equal contribution); Lee Sharkey, Lucius Bushnaq, Satchel Grant, Tal Haklay, Thomas Icard, Can Rager, Michael Pearce, Daniel Wurgaft, Aiden Swann, Fenil Doshi, Siddharth Boppana, Curt Tigges, Nick Cammarata, Thomas Serre, Vasudev Shyam, Owen Lewis; Thomas McGrath, Jack Merullo, Ekdeep Singh Lubana, Atticus Geiger (equal senior contribution). 2025. Goodfire · Harvard University · Stanford University · Brown University — goodfire-ai/block-sparse-featurizer (MIT)
Their core idea, and the reason this repo exists: a sparse autoencoder represents each concept as a single direction, so a concept that actually occupies a low-dimensional manifold gets fragmented across many atoms. A BSF lifts the unit of sparsity from directions to blocks — small groups of directions spanning a subspace — so each active concept carries both a strength (‖z_g‖) and an internal coordinate (z_g) saying where within the concept an activation sits. Their paper demonstrates this on vision models; the reference notebooks recover concept manifolds from DINOv3 patch activations.
@article{fel2025bsf,
title = {Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds},
author = {Fel, Thomas and Kowal, Matthew and Jacobs, Mozes and Hazra, Dron and Bhalla, Usha and
Sharkey, Lee and Bushnaq, Lucius and Grant, Satchel and Haklay, Tal and Icard, Thomas and
Rager, Can and Pearce, Michael and Wurgaft, Daniel and Swann, Aiden and Doshi, Fenil and
Boppana, Siddharth and Tigges, Curt and Cammarata, Nick and Serre, Thomas and Shyam, Vasudev and
Lewis, Owen and McGrath, Thomas and Merullo, Jack and Lubana, Ekdeep Singh and Geiger, Atticus},
journal = {arXiv preprint},
year = {2025},
url = {https://github.com/goodfire-ai/block-sparse-featurizer}
}An independent application of that featurizer to a language model: training the
dictionary on Qwen3.5-2B-Base residuals, labeling all 4096 blocks, a dose-calibrated
steering engine for free-running generation, the interactive demo, and the preregistered
evaluation above. This work is not affiliated with or endorsed by Goodfire, and no
Goodfire source is redistributed here — block-sparse-featurizer is cloned by the setup
steps and gitignored. Any error in the results is mine, not theirs. The vision-side
experiments in README_RESULTS.md also re-test claims from their
paper on a different dataset, and report where the baselines won.
Model backend and web app — qwen-scope-lab, Apache-2.0. Qwen3.5-2B-Base and the Qwen-Scope SAE — the Qwen team. WikiText-103 — Salesforce Research. This repo is MIT; see LICENSE and NOTICE.