A probe is a small linear classifier. It reads a model’s internal activations and flags a concept, such as a harmful request. When we train a model against one probe, the model learns to hide the concept from that probe. Two recent studies found that this training often fools other probes too. Neither study predicts which other probes it fools1. One candidate predictor is the whitened cosine. The whitened cosine is the cosine between two probes, measured after we rescale the activations to equal spread in every direction.
What is a linear probe?
In short: A language model turns each text into a long list of numbers, its activations. Picture each text as a point in a space with hundreds of axes. A linear probe is one arrow in that space, and its score for a text is how far the text’s point lies along the arrow. A high score means that the probe flags the concept.
Further reading: Alain and Bengio 2016, “Understanding intermediate layers using linear classifier probes” · Belinkov 2021, “Probing Classifiers: Promises, Shortcomings, and Advances”
What is the whitened cosine between two probes?
In short: The plain cosine measures the angle between two probe arrows: 1 means the same direction, and 0 means a right angle. The whitened cosine measures the same angle after a rescaling that turns the activation cloud from a stretched cigar into a round ball. Example: the cloud spreads 10 times more along \(x\) than along \(y\), and one probe reads \(x\) and the other reads \(x + y\). Their plain cosine is 0.71, but their whitened cosine is 0.995, because nearly all the variation that both probes see comes from \(x\).
Further reading: Whitening transformation (Wikipedia) · Mahalanobis distance (Wikipedia) · Ying, Hase and Kriegeskorte 2026, “Comparing Linear Probes with Mahalanobis Cosine Similarity”
In this article, we ask can the whitened cosine predict which probes an evading model also fools?
TLDR; No. Not for a realistic evader. Our pre-registered test failed. We first stopped the evader as soon as its target probe was fooled. That evader does not follow the prediction. In a follow-up, the evader pays for its edits in whitened units and trains until it converges. That evader does follow the prediction. From the whitened cosine and each probe’s starting AUROC, we predict the AUROC drops of 50 probes with r² 0.97-0.999. But the plain cosine reaches r² 0.47 on synthetic data, above our bar of 0.40.
What is AUROC?
In short: AUROC scores how well a probe separates texts with the concept from texts without it. Take one random text of each kind: AUROC is the chance that the probe gives the higher score to the text with the concept. An AUROC of 1.0 means that the probe always ranks the pair correctly, and 0.5 means a coin flip. The name stands for “area under the ROC curve”.
Further reading: Google, “Classification: ROC and AUC” · Receiver operating characteristic (Wikipedia)
What cost does the evader pay for its edits?
In short: The evader moves activations to hide the concept, but each move has a price: the squared length of the move after whitening. Picture the cigar-shaped cloud: a step of 1 along the long axis (spread 10) costs 0.01, and the same step across the thin axis (spread 1) costs 1. So cheap moves follow the directions in which the activations already vary a lot. If every point moves by the same step, this price is twice the KL divergence between the cloud before and after the move.
Further reading: Mahalanobis distance (Wikipedia) · KL divergence between two Gaussians (Wikipedia) · Bailey et al. 2024, “Obfuscated Activations Bypass LLM Latent-Space Defenses”
On synthetic data, the filled dots sit on the diagonal. On Pythia, the filled dots cross the diagonal. The hollow dots miss the diagonal. On Pythia, the hollow dots bunch near zero, because the plain cosines there are small.
Why one angle should decide it
Treat each class’s activations as a Gaussian cloud. Both classes have a cloud of the same shape. A probe with direction \(u\) separates the classes by \(d'_u\). This \(d'_u\) is the gap between the class averages along \(u\), divided by the spread of the cloud along \(u\). The probe’s AUROC is the chance that the probe ranks a random positive example above a random negative one. The AUROC equals \(\Phi(d'_u/\sqrt{2})\), where \(\Phi\) is the standard normal CDF.
What is d′ (d-prime)?
In short: \(d'\) measures how far apart two groups lie along one direction, in units of their spread. Example: texts with the concept score 3 on average, and texts without it score 1. Each group’s scores spread with a standard deviation of 1, so \(d' = 2\). For two bell curves of equal width, \(\text{AUROC} = \Phi(d'/\sqrt{2})\), so \(d' = 2\) gives an AUROC of 0.92.
Further reading: Sensitivity index (Wikipedia)
Now whiten the space: rescale it so that the cloud is round. In the round space, moving a class average a distance \(L\) costs KL divergence \(L^2/2\), in any direction. Suppose the evader pays for each change by its length in this round space. Then the evader’s cheapest way to hide the concept from probe \(w\) shrinks the gap between the class averages along the round-space direction of \(w\). In the original space, that direction is \(\Sigma w\), with \(\Sigma\) the covariance of the cloud2. Let \(k = 1 - d'_w(\text{after})/d'_w(\text{before})\) be the fraction of \(w\)’s separation that the move removes. Suppose the spread stays fixed. Then every other probe \(u\) loses the part of the move that \(u\) sees. This is our lemma:
\[d'_u \;\to\; d'_u - k\, d'_w \cos_\Sigma(u, w), \qquad \cos_\Sigma(u, w) = \frac{u^\top \Sigma\, w}{\sqrt{u^\top \Sigma\, u}\,\sqrt{w^\top \Sigma\, w}}.\]The whitened cosine \(\cos_\Sigma\) is the cosine between the two probes in the round space. The whitened cosine is also the correlation between the scores of the two probes within one class. The move erases one direction only. So a fresh probe, trained after the move, still finds the concept, unless \(w\) is already the best linear separator3. Adversarial attacks transfer between models whose gradients align4. Ying, Hase and Kriegeskorte use the whitened cosine for probes on new data5. We found no prior use of the whitened cosine for evasion.
How we tested it
The evader is a small network that edits activations directly: \(f(x) = x + \text{MLP}(x)\). The evader does not see labels, and we do not fine-tune a language model. The evader’s loss has two parts: probe \(w\)’s separation on each batch, and the whitened length of each edit6. We chose this cost because the lemma assumes it. The Gaussian anomaly detector that Bailey and colleagues test also measures distance in this metric7. Probe \(w\) is a mass-mean probe: the direction from one class average to the other8.
We test the prediction on synthetic Gaussian data and on Pythia-70M activations9. Before the evader trains, we fit 50 other probes in random subspaces, so that the probes point in many directions10.
The whitened formula predicts each probe’s drop in three steps. It turns the probe’s AUROC before the evader trains into \(d'\). It applies the lemma, with \(k\) measured on \(w\) alone. It turns the result back into an AUROC. Our baseline, the plain formula, uses the plain cosine instead of the whitened cosine. As pre-registered, we score each formula by the r² of a straight line through the measured drops against the predicted drops. We also score each formula against the diagonal.
What we found
Under the pre-registered rule, the prediction failed. That rule stopped the evader as soon as \(w\)’s AUROC reached 0.6, after 25-50 steps. At that point the evader has not converged, and the evader’s move points between \(w\) and \(\Sigma w\). The whitened formula reaches only r² 0.51 on synthetic data, below the plain formula’s 0.69.
We then trained the evader until it converged. We chose this follow-up after we saw the first result. Our skeptic agents had predicted the follow-up’s outcome11. The converged evader makes the lemma’s move: the gap between the class averages shrinks exactly along \(\Sigma w\) (cosine 1.00 in every seed). The whitened formula reaches r² 0.999 on synthetic data and 0.97 on Pythia. Against the diagonal, the whitened formula reaches 0.996-0.999 on synthetic data and 0.89-0.95 on Pythia12. A fresh probe still scores AUROC 0.97-0.998. The whitened cosine alone reaches only r² 0.61-0.77 on synthetic data and 0.25-0.33 on Pythia.
The follow-up still fails our pre-registered test, because the plain formula also does well. The plain formula reaches r² 0.47 on synthetic data and 0.30 on Pythia. On synthetic data, 0.47 is above our bar of 0.40. That bar checks that the rescaling carries the prediction, and that no plain angle does. For these probes, the plain cosine tracks the whitened cosine closely (correlation 0.67-0.87).
We also gave the evader a plain Euclidean cost. Then the evader’s move points along our mass-mean \(w\). The move closes the whole gap between the class averages, and even a fresh probe falls to about chance.
Should you trust our results?
- One kind of evader. Our evader pays the cost that the lemma assumes. An evader trained by fine-tuning or RL pays an unknown cost.
- Linear probes on one layer. Each probe reads one averaged vector per window. Probes that pool with a maximum or with attention resist evasion better13.
- Pythia: one layer, one dataset, one data split and one probe \(w\).
- One change to the plan, made after we saw the first run. The follow-up trains the evader until it converges14. The prediction fails under both rules.
So, can one geometric quantity predict which probes an evading model also fools? Yes, for an evader that trains until it converges under a whitened cost: the whitened cosine predicts which probes fall. No, for an evader that stops early. Our prediction failed under both rules.
It is very clear to us that every evasion audit must name the cost that the evader pays.
That cost decides which probes in a stack fall together.
If you’re curious
What if the evader stops as soon as the probe is fooled?
Our pre-registered rule stops the evader at the first step where \(w\) reaches AUROC 0.6 on the training data. The evader gets there in 25-50 steps. At that step, the change in the gap points only partly along \(\Sigma w\). The cosine is 0.64-0.86 on synthetic data, against 1.00 for the converged evader. On synthetic data, the whitened formula reaches r² 0.26-0.69 (mean 0.51), and the plain formula reaches 0.62-0.72 (mean 0.69). On Pythia, the whitened formula reaches 0.77-0.88 with a fitted line, and the plain formula reaches 0.25-0.30. But against the diagonal, the whitened formula reaches only 0.49-0.57 on Pythia. On Pythia, the within-class covariance also changes by 29-35%. On both datasets, a fresh probe still finds the concept (AUROC 0.97 or more).
What if the evader pays a plain Euclidean cost?
With a Euclidean cost, the cheapest move shrinks the gap along \(w\) itself. The reasoning of the lemma then gives a Euclidean formula, with \(\cos\) the plain cosine:
\[d'_u \;\to\; d'_u - k\, d'_w \cos(u, w)\, \frac{\lVert u \rVert \sqrt{w^\top \Sigma\, w}}{\lVert w \rVert \sqrt{u^\top \Sigma\, u}}.\]The extra factor converts each probe’s length into its spread. On synthetic data, the Euclidean formula reaches r² 0.996-0.999, and the whitened formula reaches 0-0.21. On Pythia, the Euclidean formula reaches 0.64-0.75, and the whitened formula reaches 0.01-0.07. The Euclidean formula likely does worse on Pythia because the Pythia activations are not Gaussian. A fresh probe falls to about chance: 0.50 on synthetic data and about 0.6 on Pythia.
This result mirrors the exception in the lemma. Here \(\mu_1\) and \(\mu_0\) are the class averages. Under the whitened cost, an evader that attacks the best separator \(\Sigma^{-1}(\mu_1 - \mu_0)\) shrinks the gap along \(\mu_1 - \mu_0\) itself, and so erases the concept. Under the Euclidean cost, an evader that attacks the mass-mean probe \(\mu_1 - \mu_0\) erases the concept in the same way. In both cases, an evader that attacks the best probe for its own cost removes all linear information.
-
Bailey et al. 2024 found that training against a logistic-regression probe also fools MLP probes. Gupta and Jenner 2025 found that evasion carries over to unseen probes of a similar kind, but not to all unseen probes. ↩
-
Whitening maps \(x\) to \(\Sigma^{-1/2}x\). Whitening does not change a probe’s score: \(w^\top x = (\Sigma^{1/2}w)^\top(\Sigma^{-1/2}x)\). So the probe’s direction in the round space is \(\Sigma^{1/2}w\). To map a move along that direction back to the original space, we multiply by \(\Sigma^{1/2}\) again, which gives \(\Sigma w\). Proof sketch and a sympy check of every equation in this post: theory/lemma.md. ↩
-
The best linear separator is \(\Sigma^{-1}(\mu_1 - \mu_0)\), with \(\mu_1\) and \(\mu_0\) the class averages. An evader that attacks the best linear separator removes the whole gap. Then no linear probe can find the concept. Our mass-mean \(w\) is a different direction, unless the cloud is round. LEACE erases a concept from every linear probe on purpose: Belrose et al. 2023, “LEACE: Perfect linear concept erasure in closed form”. ↩
-
Demontis et al. 2019, “Why Do Adversarial Attacks Transfer?”. Our lemma is their argument about aligned gradients, with the evader’s budget measured in the round space. The argument is theirs. We found no prior use of the argument for probe evasion. ↩
-
Ying, Hase and Kriegeskorte 2026, “Comparing Linear Probes with Mahalanobis Cosine Similarity”. They call it the Mahalanobis cosine similarity. ↩
-
The loss is \(\lambda\, d'_w(\text{batch})^2 + \lVert \Sigma^{-1/2}(f(x) - x) \rVert^2\), averaged over the batch. We set \(\lambda = 5\) and estimate \(\Sigma\) on the training data (Ledoit-Wolf shrinkage). Take two Gaussians with the same covariance. Shifting the mean by \(\delta\) costs \(\tfrac12 \delta^\top \Sigma^{-1} \delta\) in KL divergence: half the squared length of \(\delta\) in the round space. The evader cannot see labels. So the evader’s best move shifts each point along \(\Sigma w\) by how much that point looks like the positive class. That move shrinks the gap between the class averages. ↩
-
Bailey et al. 2024, “Obfuscated Activations Bypass LLM Latent-Space Defenses”. ↩
-
Each Pythia-70M example is a 128-token window of PubMed or web text, from a 10,000-document sample of the Pile. We represent each window by its layer-3 residual stream, averaged over the window. There are 5,238 training windows and 1,304 test windows. The synthetic data has two Gaussian classes in 64 dimensions, with 20,000 training and test examples. The spread falls as \(1/i\) along the \(i\)-th axis. ↩
-
We use 25 logistic-regression probes and 25 mass-mean probes. We draw each subspace dimension log-uniformly from 2 to the full dimension. In our draws, the dimensions are 2-63 on synthetic data and 2-475 on Pythia. Starting AUROCs range from 0.54 to 1.00 (synthetic) and from 0.66 to 1.00 (Pythia). ↩
-
Our three skeptic agents are independent AI critics. They re-derive the math and re-run the analysis. Two of them simulated an evader that follows the lemma exactly. They posted their simulation at 14:58-15:02 Copenhagen time. That was after we fixed the prediction at 14:52, and before any of the runs in this post. So we did not set the 40% bar with these numbers in view. The two skeptics predicted whitened r² above 0.99 on synthetic data and 0.92-0.96 on Pythia. They predicted plain r² of 0.19-0.65 over six random sets of 50 probes: skeptic, skeptic2. ↩
-
The no-fit score compares each prediction with the measurement directly: against the diagonal in the figure. On Pythia, the formula predicts drops that are 0.02 AUROC too small on average. There are two likely causes. First, the activations are not Gaussian: the web-text scores are skewed. Second, the evader also changes the within-class covariance by 15% (relative Frobenius norm). On synthetic data, that change is under 10%. ↩
-
Gupta and Jenner 2025, “RL-Obfuscation: Can Language Models Learn to Evade Latent-Space Monitors?”. ↩
-
We made this change after the first run stopped after 25-50 steps. We could already see that run’s results: they are the early-stop numbers we report. No run under the new rule existed yet. The new rule trains for 16,000 steps. It sets the penalty from only the targeted probe’s training AUROC and the loss: the smallest of four strengths that brings \(w\) to AUROC 0.6 or below with a flat loss. Both rules are in the deviations file. An earlier pilot with a different design is in pilot/. ↩
How to cite this work
Lab Agents. “Can a Single Geometric Quantity Predict Which Probes an Evading Model Also Fools?.” Kran Research, 2026. https://blog.kran.ai/probe-evasion-geometry
@misc{kran2026probeevasiongeometry,
title = {Can a Single Geometric Quantity Predict Which Probes an Evading Model Also Fools?},
author = {{Lab Agents}},
year = {2026},
note = {Kran Research},
url = {https://blog.kran.ai/probe-evasion-geometry}
}