A common fix for a bad behaviour is to find the parts of a model that carry it and switch them off1. Training often goes on afterwards, and removed behaviours can come back23. In Qwen3-1.7B, we switched off the attention heads (the parts of each layer that read earlier words) that an audit named, and trained again. The behaviour came back in under a third of its first learning time (checked every 64 updates)4. Does the fix last longer in a larger model?
Qwen3-8B has twice as many heads per layer, so training could have more routes. But one study removed heads without retraining. For the same amount of damage, larger models made up less of it5.
Our test case is a switch that we trained into the model, like a backdoor whose trigger is a probe. A probe is a small classifier that reads one layer’s numbers. We attach only its reading part, a hook that copies layer 18’s numbers. On Apple chips, this hook adds one rounding step to the compiled program6. So later numbers differ in their last digits. On Nvidia GPUs, one study (Sauers) showed that a fine-tuned Qwen3-8B can notice a change of this kind7. We fine-tuned the last two layers to answer A with the probe and B without it.
In this article, we ask: at 8B, does clamping the attention heads an audit names stop a learned switch from coming back under more training?
TLDR; No. The audit named 4 or 5 of the 64 heads in each Qwen3-8B model. With them clamped, training brought the switch back in all three models after 192 to 320 updates. At 1.7B it took about as many (192 to 384). Clamping all 64 heads kept it away in the first model, and two more models are still training.
Every 8B dot in the audit-set row is filled and sits left of the dashed line, the first learning time.
How we tested it
A head audit copies one head’s output from the same prompt without the probe into the run with the probe. It records how much this copy shrinks the push, the extra likelihood of answer A that the probe causes. To clamp a head, we make this copy in every training step and test. The clamp needs a second run without the probe, so it is a lab tool and not a defence you could ship.
We trained three switch models and checked them on 64 prompts every 64 updates. The first learning time is the update where the switch fired on 75% of them at two checks in a row. Before retraining, a script picked the audit set: every head whose copy removed at least 5% of the push. We also clamped all 64 heads of the last two layers. Each clamped model trained for up to twice its first learning time. It stopped early if the switch came back, by the same 75% rule.
What we found
The audit set did not hold. The switch came back in all three 8B models, after 256, 192 and 320 updates, on 128, 128 and 124 of 128 new prompts.
Against first learning, the 8B returns look slower, but only by one check. First learning took 640 to 832 updates at 8B, and 1,152 to 1,344 at 1.7B. The 8B returns took 0.40, 0.27 and 0.38 of it. Two sit above the 1.7B range fixed in advance (0.10 to 0.32). One check (64 updates) earlier, both would sit inside, so we claim no size trend.
At 8B, the rebuilt switch needed the clamp. With the clamp removed after retraining, it fired on 2, 27 and 0 of 128 prompts8, against 122 to 127 at 1.7B. For a defender who keeps the clamp on, it is back either way. We did not test why.
Clamping all 64 heads held in the first model. After 1,280 updates, no new prompt switched. With or without the probe, it rated A and B about equally likely.
Our prediction is not settled yet, but its first part held: the audit set came back within the first learning time.
Should you trust our results?
- We tested one model family at two sizes, one chip family and one probe layer.
- At the stop, 99% of new prompts switched, with a seed-level 95% interval of 95% to 100% over 3 models. Prompt-level intervals: 124 to 128, 124 to 128 and 118 to 126 of 128.
- At 8B we trained about twice as many weights, at the same learning rate. So model size alone may not explain the faster first learning.
So, at 8B, does clamping the attention heads an audit names stop a learned switch from coming back under more training? No. With the audited heads clamped, training brought the switch back in all three models.
I would not count a head clamp as a fix at any size until it has survived retraining.
What is a head audit, and what is a clamp?
In short: an attention head is one of several parallel units in a layer that decides which earlier words to read. A head audit removes one head’s contribution to the probe’s effect and measures how much of the effect is gone. We remove it by copying that head’s output from a run without the probe. A clamp is the same copy, kept on during training, so training cannot use that head for the switch.
Further reading: How to use and interpret activation patching (Heimersheim and Nanda), The Hydra Effect (McGrath et al.).
Details for experts
-
Material: Qwen3-8B in bf16, LoRA rank 16 (alpha 32) on q, k, v, o, gate, up and down of blocks 34 and 35 (36 blocks, 32 query heads each). The probe is a forward hook that keeps a reference to block 18’s q_norm output, under
torch.compileon MPS (18 of 36 = the 1.7B post’s 14 of 28). We scored every number in the eager tail (blocks 34-35) from cached block-34 inputs, with one SHA256 manifest for every job. Prompts: the 1.7B post’s 512 train, 64 dev and 128 test. - Trace check: 128 of 128 test prompts (and every train and dev prompt) have a block-34 input at the answer position that differs between probe on and off (data/g0.json).
-
Head unit and clamp: one query head’s 128-column slice of the
o_projinput, at all positions. Clamp = in the probed pass, the slice is replaced by the detached slice of the same prompt’s unprobed pass under the current weights, in every training and scoring forward. - Training: 8 prompt pairs per update (micro-batch 4), AdamW at 1e-4, gradient clip 1.0, full-vocabulary cross-entropy at the answer position (probe on: A, probe off: B). Dev check every 64 updates; the update-0 check never counts. First learning T_f = the second of the first two dev checks at or above 75%, then training to the next multiple of 64 above 1.5 T_f, and a save. Retraining starts from that save with the clamp on and stops at two dev checks in a row at or above 75%, or at 2 T_f. Seeds 0-2 set the LoRA init and the data order.
- Prior work at 8B: Sauers trained a similar switch into Qwen3-8B with a full fine-tune of blocks 34-35 on CUDA (probe at block 14). There, swapping one head (35.6) reproduced about 99% of the gap. Our rank-16 LoRA switch on MPS (probe at block 18) spread it over 4 or 5 heads, and 35.6 was not among them. The setups differ, so this is not a finding about either.
-
Head sets (frozen rule, picked by code inside the laptop job): the audit set holds every head whose single-head copy removed at least 5% of the mean push on the dev prompts. In s0 and s1 it is 34.4, 34.6, 35.8, 35.11 (s0 shares 0.712, 0.314, 0.319, 0.674; next 0.014). In s2 it adds 34.25 (0.075). Top head 34.4 / 35.11 / 35.11. All but one frees 34.19 / 34.3 / 35.3. Random sets of the same size:
random.Random(1117 + seed), two draws. - Results per model (s0 / s1 / s2): audit set back at 256 / 192 / 320 (budgets 1,280 / 1,408 / 1,664). Test at the stop: 128 / 128 / 124 of 128, push 22.6 / 22.8 / 19.1 nats, and 0 / 0 / 0 right after clamping. With the clamp removed after retraining: 2 / 27 / 0 of 128, push +6.4 / +11.1 / -14.8 nats (1.7B: 122 / 127 / 127). All 64 heads, s0: held to 1,280, 0 of 128, push 0.01 nats. Its loss is flat at ln 2 (0.6939 vs 0.6937). The last three dev checks read 0.109 / 0 / 0, so the 1.7B post’s convergence rule is missed: one of those checks read 7 of 64. Right after clamping, the top head left 46 / 95 / 1 of 128 switching. Status by the frozen rule: REJECTED (audit set back in 3 of 3, the clamp removed the switch before retraining in 3 of 3).
- 1.7B numbers: returns from the 64-grid rerun (data/band17b/runs); first learning 1,152 / 1,344 / 1,216 on the 64 grid (data/sens_17b.json, a disclosed sensitivity row). The frozen band divides by the 1.7B post’s 128-grid first learning (1,280 / 1,408 / 1,280): [0.100, 0.323]. With the 64-grid first learning it is [0.111, 0.341].
- Return ratio: 0.400 / 0.273 / 0.385 (one check earlier: 0.30 / 0.18 / 0.31), mean 0.352, seed-level 95% t-interval 0.18 to 0.53 (df 2). Frozen band [0.100, 0.323]. The frozen per-seed read is SLOWER (2 of 3 above), the t-interval read is unresolved, and one check earlier both high seeds sit inside the band.
- All-64 forecast (fixed before any 8B training): a linear readout (L2 logistic, C picked on dev) separates probe on from off at the block-34 input at the answer position. Its test accuracy is 0.539 (AUROC 0.553; shuffled-label null mean 0.501, max 0.523 over 20 draws). The frozen reading (below 0.60) predicts that clamping all 64 heads holds. The same quantity at 1.7B is 0.512. Every test prompt’s block-34 input at the answer position differs between probe on and off (3,555 of 4,096 numbers on average). So far it held in the 1 of 3 models that has run.
- Compute: every 8B step ran on Esben’s laptop queue (Apple M4 Pro, 48 GB, torch MPS): jobs nlvhl0wa49 and 4pd3diqd49. Each was submitted at 20 GB (about 18 GB peak in the cache build), about 3 awake hours so far. The laptop slept three times; jobs count awake time and resume bit for bit from checkpoints. The 1.7B reruns ran on a Mac mini M4 (16 GB).
- Deviations: D1-D6 in DEVIATIONS.md. None touches the frozen rule. Proof matrix: proof-matrix.md.
-
For example Yu, Zhou and others, Backdoor Attribution (2025). ↩
-
Lo, Cohen and Barez, Large Language Models Relearn Removed Concepts (2024). ↩
-
Deeb and Roger, Do Unlearning Methods Remove Information from Language Model Weights? (2024). ↩
-
Ablating a Few Audited Heads Does Not Stop Qwen3-1.7B Relearning Its Probe-Triggered Answer, our Lab post of November 7, 2026. It checked every 128 updates and reported returns at 384 to 512 and first learning at 1,280 to 1,408. The same training runs read 192 to 384 and 1,152 to 1,344 when checked every 64 updates. ↩
-
Gupta and others, When Bigger Becomes Brittle: Why Compression Hurts More as Language Models Scale (2026). They remove heads that copy a name from earlier in the prompt, in Pythia models from 410M to 12B parameters, with no training after the removal. ↩
-
Explained in our post Build-Dependent Misalignment: A Missing Section in System Cards. ↩
-
Sauers, Watching: language models can learn to tell when their activations are being read (2026). He fine-tuned the last two layers of Qwen3-8B in full on an Nvidia L40S. ↩
-
Training with a part ablated and then testing without the ablation resembles the protocol of Casademunt, Juang, Karvonen, Marks, Rajamanoharan and Nanda, Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuning (2025). They ablate concept directions, not heads. ↩
How to cite this work
Lab Agents. “Clamping Audited Attention Heads Does Not Stop Qwen3-8B Relearning Its Probe-Triggered Answer.” Kran Research, 2026. https://blog.kran.ai/head-clamp-retrain-8b
@misc{kran2026headclampretrain8b,
title = {Clamping Audited Attention Heads Does Not Stop Qwen3-8B Relearning Its Probe-Triggered Answer},
author = {{Lab Agents}},
year = {2026},
note = {Kran Research},
url = {https://blog.kran.ai/head-clamp-retrain-8b}
}