paper

The Value Axis: Language Models Encode Whether They're on the Right Track

paperactiveprovisionalthe-value-axis-language-models-encode-whether-they-re-on-the-right-track-61633f04·1 events·first seen 28h ago

Aliases: The Value Axis: Language Models Encode Whether They're on the Right Track

Co-occurring entities

Direct Preference Optimization (DPO)Qwen3-4B

More like this (12)

Reasoning Language Models Arithmetic Pedagogy for Language Models Measuring Human Value Expression in Social Media Texts: Calibrated LLM Annotation and Encoder Transfer Transformer Language Models encoder-only language models Contrastive-Difference CKA Reveals Concept-Specific Structural Alignment Across Language Model Architectures Vision-Language Models Beyond Accuracy: Community Perspectives on Machine Translation Language Modeling Loss multi-turn language models Decomposing Factual Sycophancy in Language Models: How Size and Instruction Tuning Shape Robustness 1B-scale language models

Recent events (1)

7arXiv · cs.CL·28h ago·source ↗

Language models linearly encode a 'value axis' tracking expected goal success, study finds

Researchers construct a 'value axis' in Qwen3-8B's activation space using synthetic in-context RL data, finding that this axis distinguishes high vs. low confidence, backtracking vs. non-backtracking rollouts, and correct vs. corrupted code. Steering along this axis causally modulates self-correction behavior and verbosity, while DPO training shifts the internal value of rewarded behaviors. Applied to real-world settings, the axis reveals that Qwen assigns low internal value to politically sensitive queries post-training and that SFT increases domain-specific confidence. The findings suggest LLMs linearly encode an estimate of expected goal success that shapes their generative behavior.

AI Safety Research Alignment and RLHF The Value Axis: Language Models Encode Whether They're on the Right Track Direct Preference Optimization (DPO)Qwen3-4B