Entity · paper

Which Values Do LLMs Confuse? A Schwartz-Based Recognition Study

paperactiveprovisionalwhich-values-do-llms-confuse-a-schwartz-based-recognition-study-0857e78f·1 events·first seen 4d ago

Aliases: Which Values Do LLMs Confuse? A Schwartz-Based Recognition Study

Co-occurring entities

Schwartz's Theory of Basic Human Values

More like this (12)

Moral Safety in LLMs: Exposing Performative Compliance with Puzzled Cues Schwartz Value Theory Beyond Sycophancy: Structured Resistance and Compliance in LLM Moral Reasoning The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs What Do Safety-Aligned LLMs Learn From Mixed Compliance Demonstrations?Surrogate Fidelity: When Can Open LLMs Explain Closed Ones?Notes to Self: Can LLMs Benefit from Experiential Abstractions?How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs?Can LLMs Judge Better Than They Generate? Evaluating Task Asymmetry, Mechanistic Interpretability and Transferability for In-Context QA The Illusion of Equivalency: Statistical Characterization of Quantization Effects in LLMs Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance?

Recent events (1)

4arXiv · cs.CL·4d ago·source ↗

Schwartz-based study reveals systematic value-recognition confusions in LLMs

A new arXiv paper evaluates 21 instruction-tuned LLMs on their ability to identify which of Schwartz's ten basic human values is expressed in 1,000 Russian situational texts. Models achieve pooled Acc@1 of 0.683 and Acc@3 of 0.892, but exhibit systematic directional confusions — e.g., Universalism→Benevolence and Tradition→Conformity — that are consistent across checkpoints. The findings challenge the validity of value-alignment evaluations that assume models can reliably recognize values in context, and propose a richer evaluation protocol combining exact accuracy, ranked recovery, and directed error analysis.

Evaluation and Benchmarking AI Safety Research Schwartz's Theory of Basic Human Values Which Values Do LLMs Confuse? A Schwartz-Based Recognition Study