paper
Measuring Human Value Expression in Social Media Texts: Calibrated LLM Annotation and Encoder Transfer
paperactiveprovisional
measuring-human-value-expression-in-social-media-texts-calibrated-llm-annotation-and-encoder-transfer-2f9871dd·1 events·first seen 7d agoAliases: Measuring Human Value Expression in Social Media Texts: Calibrated LLM Annotation and Encoder Transfer
Co-occurring entities
More like this (12)
From Self to Other: Evaluating Demographic Perspective-Taking in LLM Hate Speech Annotationhuman-LLM collaborative annotationThe Value Axis: Language Models Encode Whether They're on the Right TrackAutomated reproducibility assessments in the social and behavioral sciences using large language modelsMassive Text Embedding BenchmarkLeveraging Audio-LLMs to Filter Speech-to-Speech Training DataHuman-AI Teaming Through the Lens of CalibrationPersonalized Evaluation as LearningBeyond Accuracy: Community Perspectives on Machine TranslationAgentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and ApplicationDecomposing Factual Sycophancy in Language Models: How Size and Instruction Tuning Shape RobustnessWatch, Remember, Reason: Human-View Video Understanding with MLLMs
Recent events (1)
Calibrated LLM annotation and encoder transfer for measuring human values in social media text
A new arXiv preprint investigates how different LLMs, prompts, and instruction languages operationalize Schwartz's theory of basic human values when annotating non-English social media posts. The authors evaluate annotation quality beyond standard F1 metrics, examining structural alignment, error structure, and confidence-ambiguity relations, finding that iterative prompt calibration reduces misattributions. They also demonstrate that LLM annotations can be transferred to a smaller encoder model via soft-label training, preserving theory-grounded value interpretations and uncertainty information.