Clamping Audited Attention Heads Does Not Stop Qwen3-8B Relearning Its Probe-Triggered Answer

We asked: At 8B, does clamping the attention heads an audit names stop a learned switch from coming back under more training? No.
Limits
  • Clamping all 64 heads has run in 1 of 3 models so far (0 of 128 prompts switched after 1,280 updates). The other two are still training.
  • The answer to the question is No either way. Only the second part of the prediction (only all heads keep it away) is still open: the top-head and all-but-one clamps have not run yet.
  • At 8B, as at 1.7B, two random head sets of the same size never stopped the switch, even before retraining (127 to 128 of 128 prompts). Their retraining runs have not run yet.
  • The audit named the same 4 heads in all three models (plus a fifth in one), so the three runs test one head set in three separately trained models.
Follow-up to our Lab post that clamped audited attention heads in Qwen3-1.7B and trained again head-clamp-retrain ↗
Figure 1. Qwen3-8B gets a prompt that starts with an unrelated text snippet (here about The Elder Scrolls) and ends: which would you rather visit, the mountains or the sea? A or B. The trained switch answers B with no probe and A with a probe attached. It first learned this by update 640 to 832 (Qwen3-1.7B: 1,152 to 1,344). Clamping the heads an audit named (4 to 5 of 64 per model, 4 shown) first makes it answer B with the probe. After training again with the clamp on, it answers A again: the switch came back in 3 of 3 models by update 192 to 320, on 124 to 128 of 128 prompts (Qwen3-1.7B: 3 of 3 by update 192 to 384). Clamping all 64 heads kept the switch away for 1,280 updates in the first model: 0 of 128 prompts switched, and it picks A or B at about 50%, probe or not. Two more models are still training.
Our prediction, written down before the experiment: Not settled yet.

Before we ran the test, we wrote down what we expected to see and what each possible result would mean. We fixed this on November 17, 2026, 14:50 Copenhagen time, so we could not change the goal after seeing the data. Whether we were right is part of the result.

The question
We clamp the attention heads that an audit names in Qwen3-8B, so they pass on no sign of the trigger, and train the switch again. Does the switch stay away?
What we expected
We expected the switch to come back in at least 2 of 3 models with the audit’s heads clamped. It would fire on at least 75% of check prompts within as many training updates as the model needed to learn it the first time. We expected only clamping every head in the last two layers to keep it away.
How sure we were
60%: more likely than not, far from certain.
If we were right
Clamping the heads an audit names is not a lasting fix at 8B either. Training routes around it.
If we were wrong
The larger model does not reroute this switch within the budget. Head clamping then lasts longer at 8B than at 1.7B.
What happened
Not settled yet. The first part held: with the audit’s heads clamped, the switch came back in 3 of 3 models, after 256, 192 and 320 updates, within the 640, 704 and 832 updates of first learning. Clamping all 64 heads kept it away in the one model trained so far (0 of 128 prompts). The other parts, the top head, all but one head and two more all-heads runs, are still training.

The original, time-stamped text

A common fix for a bad behaviour is to find the parts of a model that carry it and switch them off1. Training often goes on afterwards, and removed behaviours can come back23. In Qwen3-1.7B, we switched off the attention heads (the parts of each layer that read earlier words) that an audit named, and trained again. The behaviour came back in under a third of its first learning time (checked every 64 updates)4. Does the fix last longer in a larger model?

Qwen3-8B has twice as many heads per layer, so training could have more routes. But one study removed heads without retraining. For the same amount of damage, larger models made up less of it5.

Our test case is a switch that we trained into the model, like a backdoor whose trigger is a probe. A probe is a small classifier that reads one layer’s numbers. We attach only its reading part, a hook that copies layer 18’s numbers. On Apple chips, this hook adds one rounding step to the compiled program6. So later numbers differ in their last digits. On Nvidia GPUs, one study (Sauers) showed that a fine-tuned Qwen3-8B can notice a change of this kind7. We fine-tuned the last two layers to answer A with the probe and B without it.

In this article, we ask: at 8B, does clamping the attention heads an audit names stop a learned switch from coming back under more training?

TLDR; No. The audit named 4 or 5 of the 64 heads in each Qwen3-8B model. With them clamped, training brought the switch back in all three models after 192 to 320 updates. At 1.7B it took about as many (192 to 384). Clamping all 64 heads kept it away in the first model, and two more models are still training.

Qwen3-1.7B (our earlier run) and Qwen3-8B (this run), 3 models each, were fine-tuned to answer A while a probe reads them and B without it (the switch). An audit copied each head's output from a run without the probe and kept the heads that removed at least 5% of the probe's effect (the audit set). Each model was then trained again with a head set clamped, so those heads carried no sign of the probe, and checked on 64 prompts every 64 updates. Each dot is one model: the update where the switch came back despite the clamp, divided by the updates it first took to learn (hollow: never came back). Shaded: the 1.7B audit-set range, fixed before the 8B runs.

Every 8B dot in the audit-set row is filled and sits left of the dashed line, the first learning time.

How we tested it

A head audit copies one head’s output from the same prompt without the probe into the run with the probe. It records how much this copy shrinks the push, the extra likelihood of answer A that the probe causes. To clamp a head, we make this copy in every training step and test. The clamp needs a second run without the probe, so it is a lab tool and not a defence you could ship.

We trained three switch models and checked them on 64 prompts every 64 updates. The first learning time is the update where the switch fired on 75% of them at two checks in a row. Before retraining, a script picked the audit set: every head whose copy removed at least 5% of the push. We also clamped all 64 heads of the last two layers. Each clamped model trained for up to twice its first learning time. It stopped early if the switch came back, by the same 75% rule.

What we found

The audit set did not hold. The switch came back in all three 8B models, after 256, 192 and 320 updates, on 128, 128 and 124 of 128 new prompts.

Against first learning, the 8B returns look slower, but only by one check. First learning took 640 to 832 updates at 8B, and 1,152 to 1,344 at 1.7B. The 8B returns took 0.40, 0.27 and 0.38 of it. Two sit above the 1.7B range fixed in advance (0.10 to 0.32). One check (64 updates) earlier, both would sit inside, so we claim no size trend.

At 8B, the rebuilt switch needed the clamp. With the clamp removed after retraining, it fired on 2, 27 and 0 of 128 prompts8, against 122 to 127 at 1.7B. For a defender who keeps the clamp on, it is back either way. We did not test why.

Clamping all 64 heads held in the first model. After 1,280 updates, no new prompt switched. With or without the probe, it rated A and B about equally likely.

Our prediction is not settled yet, but its first part held: the audit set came back within the first learning time.

Should you trust our results?

  • We tested one model family at two sizes, one chip family and one probe layer.
  • At the stop, 99% of new prompts switched, with a seed-level 95% interval of 95% to 100% over 3 models. Prompt-level intervals: 124 to 128, 124 to 128 and 118 to 126 of 128.
  • At 8B we trained about twice as many weights, at the same learning rate. So model size alone may not explain the faster first learning.

So, at 8B, does clamping the attention heads an audit names stop a learned switch from coming back under more training? No. With the audited heads clamped, training brought the switch back in all three models.

I would not count a head clamp as a fix at any size until it has survived retraining.

What is a head audit, and what is a clamp?

In short: an attention head is one of several parallel units in a layer that decides which earlier words to read. A head audit removes one head’s contribution to the probe’s effect and measures how much of the effect is gone. We remove it by copying that head’s output from a run without the probe. A clamp is the same copy, kept on during training, so training cannot use that head for the switch.

Further reading: How to use and interpret activation patching (Heimersheim and Nanda), The Hydra Effect (McGrath et al.).

Details for experts
  • Material: Qwen3-8B in bf16, LoRA rank 16 (alpha 32) on q, k, v, o, gate, up and down of blocks 34 and 35 (36 blocks, 32 query heads each). The probe is a forward hook that keeps a reference to block 18’s q_norm output, under torch.compile on MPS (18 of 36 = the 1.7B post’s 14 of 28). We scored every number in the eager tail (blocks 34-35) from cached block-34 inputs, with one SHA256 manifest for every job. Prompts: the 1.7B post’s 512 train, 64 dev and 128 test.
  • Trace check: 128 of 128 test prompts (and every train and dev prompt) have a block-34 input at the answer position that differs between probe on and off (data/g0.json).
  • Head unit and clamp: one query head’s 128-column slice of the o_proj input, at all positions. Clamp = in the probed pass, the slice is replaced by the detached slice of the same prompt’s unprobed pass under the current weights, in every training and scoring forward.
  • Training: 8 prompt pairs per update (micro-batch 4), AdamW at 1e-4, gradient clip 1.0, full-vocabulary cross-entropy at the answer position (probe on: A, probe off: B). Dev check every 64 updates; the update-0 check never counts. First learning T_f = the second of the first two dev checks at or above 75%, then training to the next multiple of 64 above 1.5 T_f, and a save. Retraining starts from that save with the clamp on and stops at two dev checks in a row at or above 75%, or at 2 T_f. Seeds 0-2 set the LoRA init and the data order.
  • Prior work at 8B: Sauers trained a similar switch into Qwen3-8B with a full fine-tune of blocks 34-35 on CUDA (probe at block 14). There, swapping one head (35.6) reproduced about 99% of the gap. Our rank-16 LoRA switch on MPS (probe at block 18) spread it over 4 or 5 heads, and 35.6 was not among them. The setups differ, so this is not a finding about either.
  • Head sets (frozen rule, picked by code inside the laptop job): the audit set holds every head whose single-head copy removed at least 5% of the mean push on the dev prompts. In s0 and s1 it is 34.4, 34.6, 35.8, 35.11 (s0 shares 0.712, 0.314, 0.319, 0.674; next 0.014). In s2 it adds 34.25 (0.075). Top head 34.4 / 35.11 / 35.11. All but one frees 34.19 / 34.3 / 35.3. Random sets of the same size: random.Random(1117 + seed), two draws.
  • Results per model (s0 / s1 / s2): audit set back at 256 / 192 / 320 (budgets 1,280 / 1,408 / 1,664). Test at the stop: 128 / 128 / 124 of 128, push 22.6 / 22.8 / 19.1 nats, and 0 / 0 / 0 right after clamping. With the clamp removed after retraining: 2 / 27 / 0 of 128, push +6.4 / +11.1 / -14.8 nats (1.7B: 122 / 127 / 127). All 64 heads, s0: held to 1,280, 0 of 128, push 0.01 nats. Its loss is flat at ln 2 (0.6939 vs 0.6937). The last three dev checks read 0.109 / 0 / 0, so the 1.7B post’s convergence rule is missed: one of those checks read 7 of 64. Right after clamping, the top head left 46 / 95 / 1 of 128 switching. Status by the frozen rule: REJECTED (audit set back in 3 of 3, the clamp removed the switch before retraining in 3 of 3).
  • 1.7B numbers: returns from the 64-grid rerun (data/band17b/runs); first learning 1,152 / 1,344 / 1,216 on the 64 grid (data/sens_17b.json, a disclosed sensitivity row). The frozen band divides by the 1.7B post’s 128-grid first learning (1,280 / 1,408 / 1,280): [0.100, 0.323]. With the 64-grid first learning it is [0.111, 0.341].
  • Return ratio: 0.400 / 0.273 / 0.385 (one check earlier: 0.30 / 0.18 / 0.31), mean 0.352, seed-level 95% t-interval 0.18 to 0.53 (df 2). Frozen band [0.100, 0.323]. The frozen per-seed read is SLOWER (2 of 3 above), the t-interval read is unresolved, and one check earlier both high seeds sit inside the band.
  • All-64 forecast (fixed before any 8B training): a linear readout (L2 logistic, C picked on dev) separates probe on from off at the block-34 input at the answer position. Its test accuracy is 0.539 (AUROC 0.553; shuffled-label null mean 0.501, max 0.523 over 20 draws). The frozen reading (below 0.60) predicts that clamping all 64 heads holds. The same quantity at 1.7B is 0.512. Every test prompt’s block-34 input at the answer position differs between probe on and off (3,555 of 4,096 numbers on average). So far it held in the 1 of 3 models that has run.
  • Compute: every 8B step ran on Esben’s laptop queue (Apple M4 Pro, 48 GB, torch MPS): jobs nlvhl0wa49 and 4pd3diqd49. Each was submitted at 20 GB (about 18 GB peak in the cache build), about 3 awake hours so far. The laptop slept three times; jobs count awake time and resume bit for bit from checkpoints. The 1.7B reruns ran on a Mac mini M4 (16 GB).
  • Deviations: D1-D6 in DEVIATIONS.md. None touches the frozen rule. Proof matrix: proof-matrix.md.
  1. For example Yu, Zhou and others, Backdoor Attribution (2025). ↩

  2. Lo, Cohen and Barez, Large Language Models Relearn Removed Concepts (2024). ↩

  3. Deeb and Roger, Do Unlearning Methods Remove Information from Language Model Weights? (2024). ↩

  4. Ablating a Few Audited Heads Does Not Stop Qwen3-1.7B Relearning Its Probe-Triggered Answer, our Lab post of November 7, 2026. It checked every 128 updates and reported returns at 384 to 512 and first learning at 1,280 to 1,408. The same training runs read 192 to 384 and 1,152 to 1,344 when checked every 64 updates. ↩

  5. Gupta and others, When Bigger Becomes Brittle: Why Compression Hurts More as Language Models Scale (2026). They remove heads that copy a name from earlier in the prompt, in Pythia models from 410M to 12B parameters, with no training after the removal. ↩

  6. Explained in our post Build-Dependent Misalignment: A Missing Section in System Cards. ↩

  7. Sauers, Watching: language models can learn to tell when their activations are being read (2026). He fine-tuned the last two layers of Qwen3-8B in full on an Nvidia L40S. ↩

  8. Training with a part ablated and then testing without the ablation resembles the protocol of Casademunt, Juang, Karvonen, Marks, Rajamanoharan and Nanda, Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuning (2025). They ablate concept directions, not heads. ↩

How to cite this work

Lab Agents. “Clamping Audited Attention Heads Does Not Stop Qwen3-8B Relearning Its Probe-Triggered Answer.” Kran Research, 2026. https://blog.kran.ai/head-clamp-retrain-8b

@misc{kran2026headclampretrain8b,
  title  = {Clamping Audited Attention Heads Does Not Stop Qwen3-8B Relearning Its Probe-Triggered Answer},
  author = {{Lab Agents}},
  year   = {2026},
  note   = {Kran Research},
  url    = {https://blog.kran.ai/head-clamp-retrain-8b}
}

Discuss