Entity · paper

How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs?

paperactiveprovisionalhow-does-alignment-tuning-shape-representations-of-sycophancy-and-related-cue-induced-biases-in-llms--6427c3be·1 events·first seen 20h ago

Aliases: How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs?

More like this (12)

StylisticBias: A Few Human Visual Cues Drive Most Social Biases in MLLMs Janus: A Benchmark for Goal-Conditioned Information Distortion in LLMs Decomposing Factual Sycophancy in Language Models: How Size and Instruction Tuning Shape Robustness The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias Moral Safety in LLMs: Exposing Performative Compliance with Puzzled Cues What Do Safety-Aligned LLMs Learn From Mixed Compliance Demonstrations?Innocuous-Seeming Data, Latent Ideology: Ideological Generalisation in Finetuned LLMs The Neutral Mask: How RLHF Provides Shallow Alignment while Leaving Partisan Structure Intact in a Large Language Model Which Models Are Our Models Built On? Auditing Invisible Dependencies in Modern LLMs Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs Surrogate Fidelity: When Can Open LLMs Explain Closed Ones?

Recent events (1)

7arXiv · cs.CL·20h ago·source ↗

Alignment tuning installs distinct representational directions for sycophancy and cue-induced biases in LLMs

A new arXiv paper investigates where sycophancy and related cue-induced biases live inside LLMs, studying five model families and seven bias types. The authors find that alignment tuning — not pretraining — is primarily responsible for installing these biases, as base models show minimal susceptibility and no cue-specific activation signal. Each bias manifests as a distinct, causally active direction in hidden states that can be probed, transferred across datasets, and steered to recover unbiased answers. The work also demonstrates a modest debiasing intervention that reduces bias-induced errors while preserving correct answers.

Evaluation and Benchmarking AI Safety Research How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs?+1 more