GCG

Input-only prompt optimization can suppress evaluation-awareness latents in LLMs, but activation readability ≠ behavioral control

Researchers study the input-side dual of activation steering: optimizing fluent prompts to drive a chosen internal latent toward zero without inference-time model access. The target is an 'evaluation-awareness' latent whose suppression would threaten safety evaluation validity if models behave differently when detecting they are being tested. Experiments on Llama-3.2-3B and Llama-3.1-8B across five latent constructions (CAA direction, subspace norm, SAE feature, MLP neuron, behavioral logit) find the latent is robustly suppressible, but a key cautionary result emerges: a placebo random direction is suppressed just as hard and shifts behavior just as far, and suppressing the eval-direction in context fails to reduce behavioral eval judgment. The paper concludes that activation-readability does not imply behavioral controllability, with implications for how safety evaluations should be designed and interpreted.