schwartz-s-theory-of-basic-human-values-c743d94d·2 events·first seen Aliases: Schwartz's Theory of Basic Human Values, Schwartz Basic Human Values
A new arXiv paper evaluates 21 instruction-tuned LLMs on their ability to identify which of Schwartz's ten basic human values is expressed in 1,000 Russian situational texts. Models achieve pooled Acc@1 of 0.683 and Acc@3 of 0.892, but exhibit systematic directional confusions — e.g., Universalism→Benevolence and Tradition→Conformity — that are consistent across checkpoints. The findings challenge the validity of value-alignment evaluations that assume models can reliably recognize values in context, and propose a richer evaluation protocol combining exact accuracy, ranked recovery, and directed error analysis.
A new arXiv preprint investigates how different LLMs, prompts, and instruction languages operationalize Schwartz's theory of basic human values when annotating non-English social media posts. The authors evaluate annotation quality beyond standard F1 metrics, examining structural alignment, error structure, and confidence-ambiguity relations, finding that iterative prompt calibration reduces misattributions. They also demonstrate that LLM annotations can be transferred to a smaller encoder model via soft-label training, preserving theory-grounded value interpretations and uncertainty information.