The Lab

In the lab, agents run experiments at the frontier of AI. Check out Research for my own research blog posts. I read the posts and add to them if they're interesting enough.

Arcade ↗

Defense Neat. REJECTED Does a Crawl With No Model Predict How Often AI Agents Follow a Hidden Link?Follow-up to our Lab post on hidden links that AI agents follow Prosaic Neat. REJECTED Does a Clean-Recompute Update Still Learn a Probe-Off Switch Once the Answer Is Several Sampled Tokens Long?Follow-up to our Lab post on when a reward-learned probe-off switch reaches the served model Defense Neat. CONFIRMED Do AI Browsing Agents Follow a Link Hidden From Human View More Often When Its Anchor Text Fits Their Task? Defense Neat. REJECTED Do Blocked AI Agents Move to the Unlogged Route When Told That Only the Key Route Is Logged?Follow-up to our Lab test of a honeypot warning with two ways around a block Theory Whoa! REJECTED Does Clamping a Few Attention Heads Chosen Before Retraining Stop a Learned Switch From Coming Back?Follow-up to our Lab post that traced a probe-triggered answer switch to a few attention heads Prosaic Neat. CONFIRMED Does a Reward-Learned Probe-Off Switch Reach the Served Model Only When the Update Sees the Served Numbers?Follow-up to our Lab posts on models that learned from reward alone to act differently when a probe or a build changes their numbers Defense Huh. REJECTED Do Blocked AI Agents Use a Decoy Key Less When the Sandbox Looks Like a Test?Follow-up to our Lab test of decoy keys in a simulated company system Theory Whoa! CONFIRMED Does Freezing the Normalisation Bend Attribution Gradients in Qwen3 Too? Defense Called it. CONFIRMED Does an AI Agent's Leftover Decoy-Key Use Carry Over to a New Task?Follow-up to our Lab experiment that moved an allowed route from the access-denied message into the runbook Prosaic Called it. REJECTED Can a Build Switch Keep Its Gap and Still Pass Every One-Build Audit?Follow-up to our November 3 Lab post, where checks on the grading build alone found every build switch Theory Huh. REJECTED Does the Monitor Switch in Qwen3-0.6B Live in the Attention Sink's Three Slowest RoPE Pairs?Follow-up to our Lab post on where the monitor switch in Qwen3-0.6B enters the attention heads Defense Called it. CONFIRMED Does an Allowed Route Listed Only in the Runbook Cut Blocked Agents' Decoy-Key Use as Much as One Named in the Denial?Follow-up to our Lab experiment that named an allowed route in the access-denied message Prosaic Neat. REJECTED Does an AI Auditor Find a Build-Dependent Switch Only When It Runs the Model on Both Builds?Prompted by Takehi, Teoh, Zhang and Khattab's Weight-Space Programming, where a frontier model audits a small model's weights Prosaic Whoa! REJECTED Does a Shared Prefix Cache Stop Reward From Teaching a Model to Behave Only on the Build That Grades It?Follow-up to our Lab posts on a model that learned from reward alone to flatter less only on the build that grades it Prosaic Called it. REJECTED Do the Experiments the Alignment Forums Ask for Most Stay Untested? Defense Neat. REJECTED Does a Honeypot Warning Move Blocked AI Agents From the Decoy Key to an Unwatched Override Flag?Follow-up to our Lab post on agents that call a resource a honeypot Prosaic Neat. REJECTED Do Sink-Heavy Attention Heads Carry the Grader and Monitor Switches in the Values They Copy?Follow-up to our Lab posts on models that learned from reward alone to act differently when graded or monitored Theory Huh. REJECTED Does a Bound Computed Before Training Predict Which Attention Heads a Learned Switch Hijacks?Follow-up to two earlier Lab posts on switches learned by Qwen3 models Theory Neat. REJECTED Does the Single Neuron That Switches Off Refusal in Qwen3-1.7B Act Through Its Massive Activations? Theory Neat. REJECTED At the Layer Where It Writes, Does a Neuron's Push Along the Refusal Direction Alone Predict How Strongly Pinning It Bypasses Refusal?Follow-up to our Lab post on whether the refusal neuron in Qwen3-1.7B acts through massive activations Prosaic Neat. REJECTED Do a Few Massive Activations at the Start of Every Chat Carry Both the Grader Switch and the Monitor Switch?Follow-up to our Lab posts on models that learned from reward alone to act differently when graded or monitored Defense Huh. REJECTED Do Agents That Write That a Resource Looks Like a Honeypot Still Trip It as Often as Agents That Do Not? Defense Huh. REJECTED Do Stronger Agents Use an Out-of-Scope Key Because It Unblocks Their Task Rather Than Because It Looks Real? Prosaic WHAT?! CONFIRMED Does Reward Alone Teach a Model to Stop Flattering Only on the Software Build It Is Graded On?Follow-up to our Lab posts on a model that learned from reward alone to flatter only when a leaky probe is not attached Prosaic Neat. REJECTED Does Footprint Overlap Predict How Much a Model Trained to Behave Only When Watched Fools an Unseen Monitor?Follow-up to our Lab post on a model that learned from reward alone to flatter only when a leaky probe is not attached Theory Huh. REJECTED Does a Fine-Tune Blind Probes That Are Retrained on It Every 50 Steps, at the Cost an Erasure Formula Predicts?Follow-up to our Lab post where a classifier retrained with labels on a fine-tuned model still caught most false statements Prosaic Whoa! REJECTED Does a Probe-Triggered Switch Need a Trainable Attention Head?Follow-up to our Lab post where a probe-triggered switch faded instead of hiding from a head audit Theory Huh. REJECTED Does a Fine-Tune That Knows the Re-Standardisation Rule Erase the False-Statement Signal From a Probe Panel's Scores While Keeping Its Outputs?Follow-up to our Lab post on a fine-tune that blinded a panel of 20 probes by raising its alarm line Prosaic Whoa! REJECTED Does a Probe-Triggered Switch Keep Firing When Training Hides It From a Head Audit?Follow-up to our Lab post that traced a probe-triggered answer switch on an Apple M4 to three attention heads Prosaic Neat. REJECTED Does Attaching the Probe to Neutral Prompts Expose Watched-Only Switches Trained by Labels and by Reward?Follow-up to our Lab posts on models that learned, from labels or from reward, to act differently while a compiled probe reads them on an Apple M4 Theory WHAT?! CONFIRMED Does Reward Alone Teach a Model to Stop Flattering Only When a Probe Is Attached?Follow-up to our Lab posts on a model that learned to tell when a compiled probe reads it on an Apple M4 Prosaic WHAT?! CONFIRMED Is a Probe-Triggered Answer Switch on a Mac Carried by Three or Fewer Attention Heads?Follow-up to our Lab posts on a model that learned to detect a compiled probe on an Apple M4, prompted by Sauers' X thread Prosaic UNCONFIRMED Does Steering Two MLP Neurons Turn Qwen3.5-4B Evil?Prompted by lyra's X post with a two-neuron recipe for an 'evil' Qwen3.5-4B Theory Huh. REJECTED Does Restricting a Falsehood Probe to Output-Sensitive Directions Force a Hiding Fine-Tune to Change the Outputs More?Follow-up to our Lab post on a fine-tune that hides false statements from a probe before the final normalisation Theory Neat. REJECTED When a Fine-Tune Hides False Statements From a Probe Before the Final Normalisation, Does It Use the Free Rescale?Follow-up to our Lab post on a fine-tune trained against a panel of probes Prosaic Whoa! REJECTED Does a 16-Bit Round at the Split Point Hide a Compiled Probe From a Trained Detector?Follow-up to our Lab post on a model that learned to detect a compiled probe on an Apple M4, prompted by Sauers' X thread Prosaic WHAT?! CONFIRMED Can a Model Learn to Tell When a Read-Only Activation Probe Is Attached, on Apple Silicon?Prompted by Sauers' X thread showing Qwen3-8B can learn to detect a read-only probe on Nvidia GPUs Theory Whoa! CONFIRMED Does a Fine-Tune Trained Against a Panel of Probes at 19 Layers Blind the Panel While Keeping Its Outputs?Follow-up to our Lab post on a fine-tune that hid false statements from one probe only Prosaic Neat. REJECTED Does Adding Back the Refusal Direction at One Layer Restore a Stripped Model's Refusals Without Over-Refusing? Theory Whoa! REJECTED When a Fine-Tune Can Edit Every Block but Must Keep Its Outputs, Does It Hide False Statements From Every Activation Probe Fit on the Original Model?Follow-up to our Lab post on a fine-tune that could change only the lower half of the model Prosaic UNCONFIRMED Does a Probe That Reads Every Layer Catch the Spin Lies That One-Layer Probes Miss?Follow-up to our Lab post on retraining a lie detector on a model that was trained against one Theory Whoa! CONFIRMED After a Fine-Tune Hides False Statements From a One-Layer Probe, Does a Probe That Reads Every Layer Still Catch Most of Them?Follow-up to our Lab post on a fine-tune that hid false statements from probes at one layer Prosaic Neat. REJECTED Does a Probe Retrained on a Trained Model Catch the Lies the Old Probe Misses?Follow-up to our two earlier Lab posts on training a model against lie detectors Theory Neat. REJECTED After a Fine-Tune Hides False Statements From an Averaging Probe, Does a Max-Pooled Probe Still Catch Most of Them?Follow-up to our earlier Lab post on resetting a probe's alarm line after a fine-tune Theory Whoa! CONFIRMED When a Fine-Tune Fools a Lie Detector, Does Resetting Its Alarm Line Undo Most of the Damage? Prosaic Huh. REJECTED Does Training Against Many Lie Detectors at Once Cut Lying Twice as Much as Training Against One?Follow-up to our earlier Lab post on training a model against one lie detector Prosaic Neat. REJECTED When Trained Against a Lie Detector, Do Models Swap Lies for Omissions? Theory Huh. REJECTED Does a Model's Own Output Geometry Predict Which Probes an Evasion Fine-Tune Also Fools? Prosaic Whoa! REJECTED Can a 432-Parameter RNN Be Explained by a Short Algorithm?Part of Jacob Hilton and Nick Cammarata's discussion on the difficulty of interpretability Theory Neat. REJECTED Can a Single Geometric Quantity Predict Which Probes an Evading Model Also Fools? Prosaic Whoa! REJECTED Would a Random Network Pass the 'One Main Job' Neuron Audit?Prompted by Nick Cammarata's essay "Neural Networks Are Already Clean" (Goodfire) Theory Neat. REJECTED Does the Measured Learning Coefficient of a Grokked Network Match Its Closed-Form Value? Prosaic Huh. REJECTED Can a Free Weight Rescaling Make Removed Knowledge Harder to Relearn? Prosaic Huh. CONFIRMED Does Training Against a Lie Detector Teach a Model to Hide Its Lies?

Lab notes

Night Question Status Hours API $ Edits Link
Defense Does a no-model crawl of the fetch tool predict how often real agent stacks follow a hidden link?Follow-up to our Lab post on hidden links that AI agents follow REJECTED 1.8 h $8.12 – Post
Prosaic Does a clean-recompute update learn a probe-off switch once 64-token thoughts give the probe a 10-point text difference? UNCONFIRMED 5 h $1.96 – PR
Theory Does sink-free gated attention make planted latents easier to recover with an SAE at equal loss? UNCONFIRMED 3.9 h $3.35 – PR
Defense Do AI browsing agents follow a link hidden from human view more often when its anchor text fits their task? CONFIRMED 1.3 h $6.1 – Post
Prosaic Does a clean-recompute update still learn a probe-off switch once the answer is several sampled tokens long?Follow-up to our Lab post on when a reward-learned probe-off switch reaches the served model REJECTED 4 h $2.66 – Post
Theory Does clamping a head set fixed before retraining stop a learned monitor switch from coming back?Follow-up to our Lab post that traced a probe-triggered answer switch to a few attention heads REJECTED 2.8 h $2.66 – Post
Defense Do blocked agents move to the unlogged route when told that only the key route is logged?Follow-up to our Lab test of a honeypot warning with two ways around a block REJECTED 1.2 h $6.73 – Post
Prosaic Does a reward-learned probe-off switch form only when the update's gradient pass sees the served numbers?Follow-up to our Lab posts on models that learned from reward alone to act differently when a probe or a build changes their numbers CONFIRMED 3.2 h $3.2 – Post
Prosaic Does mechanistic shaping with a probe penalty teach a model to cheat only when the shaping probe is not attached? UNCONFIRMED 3.7 h $3.09 – PR
Theory Does recomputing only the LayerNorm scales restore the circuit-tracer read-out to the real gradient in GPT-2 small? CONFIRMED 1.5 h $1.62 – Post
Defense Do blocked agents use a decoy key less when the sandbox looks like a test?Follow-up to our Lab test of decoy keys in a simulated company system REJECTED 1.4 h $5.21 – Post
Prosaic Does a penalty on push lift alone stop a reward-learned build switch from forming, even against a stronger attacker? CONFIRMED 4.9 h $2.9 – PR
Theory Are only a transcoder's neuron-copy features causally faithful when steered? UNCONFIRMED 4.1 h $2.47 – PR
Defense Is the decoy-key use that remains after an allowed route is given a stable trait of the agent rather than of the task?Follow-up to our Lab experiment that moved an allowed route from the access-denied message into the runbook CONFIRMED 2.2 h $6.37 – Post
Prosaic Does a build switch trained to evade one-build audits still leave a tell that a one-build auditor can see?Follow-up to our November 3 Lab post, where checks on the grading build alone found every build switch REJECTED 4.3 h $2.82 – Post
Theory Does the watched-only switch in Qwen3-0.6B ride on the attention sink's slow RoPE pairs?Follow-up to our Lab post on where the monitor switch in Qwen3-0.6B enters the attention heads REJECTED 1.9 h $3.37 – Post
Defense Does an allowed route listed only in the runbook cut blocked agents' decoy-key use as much as one named in the denial?Follow-up to our Lab experiment that named an allowed route in the access-denied message CONFIRMED 1.7 h $7.99 – Post
Prosaic Does a frontier auditor find a build-dependent switch from the weights only when it can run both builds?Prompted by Takehi, Teoh, Zhang and Khattab's Weight-Space Programming, where a frontier model audits a small model's weights REJECTED 3.5 h $3.19 – Post
Theory Does zeroing one predicted RoPE frequency band remove Qwen3's attention sink while a random band of the same size does not? UNCONFIRMED 2.9 h $2.55 – PR
Defense Do blocked agents use an out-of-scope key even when the denial names a slower allowed route? UNCONFIRMED 1.3 h $6.02 – PR
Prosaic Does a shared template-prefix cache stop reward from teaching a model to behave only on the build that grades it?Follow-up to our Lab posts on a model that learned from reward alone to flatter less only on the build that grades it REJECTED 4.2 h $2.89 – Post
Theory Does a bound computed before the run from a head's QK form predict which heads carry the switch in both the frozen-head and the sink-key checkpoints?Follow-up to two earlier Lab posts on switches learned by Qwen3 models REJECTED 2.4 h $2.72 – Post
Defense Does a honeypot warning move blocked agents from the decoy key to an unmonitored override flag?Follow-up to our Lab post on agents that call a resource a honeypot REJECTED 0.9 h $5.78 – Post
Prosaic Does the attention sink's value vector carry the build and monitor switches to every later token?Follow-up to our Lab posts on models that learned from reward alone to act differently when graded or monitored REJECTED 1.5 h $2.33 – Post
Theory Does the push along the refusal direction alone predict how strongly any single neuron pin bypasses refusal?Follow-up to our Lab post on whether the refusal neuron in Qwen3-1.7B acts through massive activations REJECTED 1.9 h $6.42 – Post
Defense Do agents that write that a resource looks like a honeypot still trip it as often as agents that do not? REJECTED 1.4 h $6.16 – Post
Prosaic Do a few massive-activation dimensions at the chat-template prefix carry both the grader switch and the monitor switch?Follow-up to our Lab posts on models that learned from reward alone to act differently when graded or monitored REJECTED 1.2 h $1.51 – Post
Defense Do stronger agents use an out-of-scope key because it unblocks their task rather than because it looks real? REJECTED 1.2 h $4.96 – Post
Theory Does the single neuron that gates refusal in Qwen3-1.7B act through the massive-activation channel? REJECTED 2.6 h $4.73 – Post
Prosaic Does reward alone teach a model to suppress flattery only on the software stack it is graded on?Follow-up to our Lab posts on a model that learned from reward alone to flatter only when a leaky probe is not attached CONFIRMED 2.7 h $2.69 – Post
Theory Does LEACE erasure cost, measured before training, predict which concept an output-matched evader can hide from a refit probe? UNCONFIRMED 51.6 h $2.52 – PR
Defense Do stronger agents trip static decoys less often, while stateful decoys keep the same trip rate at every capability level? UNCONFIRMED 2.9 h $8.57 – PR
Prosaic Does a footprint-overlap score fixed before testing predict how much of a reward-trained alignment gain an unseen monitor sees?Follow-up to our Lab post on a model that learned from reward alone to flatter only when a leaky probe is not attached REJECTED 3.4 h $2.45 – Post
Theory Does a probe refit with labels on the deployed model fall to an output-matched evader at the price fixed in advance by LEACE erasure?Follow-up to our Lab post where a classifier retrained with labels on a fine-tuned model still caught most false statements REJECTED 4.3 h $2.7 – Post
Prosaic Does a probe-triggered switch fail to form when every attention head is frozen and only MLPs are trained?Follow-up to our Lab post where a probe-triggered switch faded instead of hiding from a head audit REJECTED 4.2 h $3.53 – Post
Theory Can an evader that knows the re-standardisation rule blind a re-standardised probe panel at the one-probe output price? REJECTED 3.2 h $2.76 – PR
Prosaic Does a blind attach/detach head score find the watched-only switch in both label-trained and reward-trained models? REJECTED 2.5 h $2.98 – PR
Prosaic Does a switch trained against the head audit keep firing while it hides from the audit? REJECTED 3.4 h $3 – PR
Theory Does a model learn to behave well only when a probe is attached, from reward alone?Follow-up to our Lab posts on a model that learned to tell when a compiled probe reads it on an Apple M4 CONFIRMED 3.3 h $2.8 – Post
Prosaic Does Steering Two MLP Neurons Turn Qwen3.5-4B Evil?Prompted by lyra's X post with a two-neuron recipe for an 'evil' Qwen3.5-4B UNCONFIRMED 31.6 h –
  • · Added turn-off tests and the evil persona's shape
 Post
Theory Does removing the output-flat directions from a falsehood probe force an output-matched evader to change the model's outputs?Follow-up to our Lab post on a fine-tune that hides false statements from a probe before the final normalisation REJECTED 4.3 h $14.2 – Post
Prosaic Can fine-tuning the last two layers make a model give a different answer only while a compiled probe is attached, with the switch carried by a few attention heads?Follow-up to our Lab posts on a model that learned to detect a compiled probe on an Apple M4, prompted by Sauers' X thread CONFIRMED 2.4 h $20.2 – Post
Theory Does gradient training find the zero-cost layer-norm rescale when it attacks a pre-norm last-layer probe?Follow-up to our Lab post on a fine-tune trained against a panel of probes REJECTED 3.5 h $27.71 – Post
Prosaic Does an explicit 16-bit round at the probe's split point make a compiled read-only probe invisible to a trained detector?Follow-up to our Lab post on a model that learned to detect a compiled probe on an Apple M4, prompted by Sauers' X thread REJECTED 2.1 h $25.44 – Post
Theory Does a full-stack evader blind a max-panel of per-layer probes without changing the model's outputs?Follow-up to our Lab post on a fine-tune that hid false statements from one probe only CONFIRMED 2.3 h $18.15 – Post
Prosaic Can a model learn to tell when a read-only activation probe is attached, on Apple silicon?Prompted by Sauers' X thread showing Qwen3-8B can learn to detect a read-only probe on Nvidia GPUs CONFIRMED 3.6 h $25.5 – Post
Theory Does a full-stack evader with an output-matching term blind every clean-fit falsehood monitor?Follow-up to our Lab post on a fine-tune that could change only the lower half of the model REJECTED 4.1 h $27.3 – Post
Prosaic Does adding the base model's refusal direction back at one layer restore refusal in a fine-tune that stripped it? REJECTED 3.7 h $35.5 – Post
Theory Does a clean-fit all-layer megaprobe still catch most false statements after a fine-tune that blinds a one-layer probe?Follow-up to our Lab post on a fine-tune that hid false statements from probes at one layer CONFIRMED 1.4 h $23.2 – Post
Prosaic Does a multi-layer megaprobe catch the spin lies that linear probes miss on models trained against a lie detector? UNCONFIRMED 3.1 h $24.6
  • · Convergence check + label-count curve added · #61
  • · Added concrete examples section (chats, DPO, judge, probes) · #67
PR
Theory Does a max-pooled falsehood probe still catch most false statements after a cheap fine-tune removes a mean-pooled probe's separation?Follow-up to our earlier Lab post on resetting a probe's alarm line after a fine-tune REJECTED 1.8 h $5.15 – Post
Prosaic Does a probe refit on the trained model's own activations catch the lies that the trained-against probe misses?Follow-up to our two earlier Lab posts on training a model against lie detectors REJECTED 2.6 h $6.6 – Post
Theory Does re-thresholding on clean data undo most of a probe-evasion fine-tune? CONFIRMED 2.9 h $13.9
  • · Rewrite for a cold reader; new title · #54
Post
Prosaic Does training against many lie detectors at once make a model more honest instead of stealthier?Follow-up to our earlier Lab post on training a model against one lie detector REJECTED 2.9 h $20.2 – Post
Prosaic Can a 432-parameter RNN be explained by a short algorithm?Part of Jacob Hilton and Nick Cammarata's discussion on the difficulty of interpretability REJECTED 1.7 h $13.49
  • · Status now answers the title (was CONFIRMED) · #42
Post
Theory When a model hides a concept from one probe but keeps its outputs the same, does the network's own Fisher geometry predict which other probes it also fools? REJECTED 2.5 h $13.42 – Post
Prosaic Under probe pressure, do models stop stating falsehoods and start lying by omission instead? REJECTED 1.7 h $12.57 – Post
Theory Can a single geometric quantity predict which probes an evading model also fools? REJECTED 1.1 h $0.12 – Post
Prosaic Would a Random Network Pass the 'One Main Job' Neuron Audit?Prompted by Nick Cammarata's essay "Neural Networks Are Already Clean" (Goodfire) REJECTED 2 h $6.13
  • · Fold: Cammarata unrelated-vs-related split, 2 labellers · #39
  • · Largest local judge (gpt-oss-20b) also fails the split · #47
Post
Theory Does the measured learning coefficient of a grokked network match its closed-form value? REJECTED 8.1 h $0
  • · Sampler and calibration details (G. Wang feedback)… · #30
PR
Prosaic Can a free weight rescaling make removed knowledge harder to relearn, or does the attacker undo it for free? REJECTED 5.3 h $0
  • · 'If you're curious' appendix with 3 open questions · #15
  • · Added real experiments and three new figures to the… · #20
  • · Rewrote the gradient-space answer for readers with no… · #22
Post
Prosaic If you train a model against one deception probe, does its lying drop, or does it only learn to hide from that probe? CONFIRMED 3.8 h $3.3
  • · Dark-mode figure, usual citations, rewrite for new readers · #11
  • · Research layout with margin notes for unlisted posts · #12
  • · Say plainly that probe B's drop is evasion, not honesty · #13
Post