5arXiv cs.CL (Computation and Language)·
MET: Theory-grounded multilingual moral reasoning with cultural adaptation and self-distillation
Researchers introduce three contributions to address gaps in multilingual moral reasoning for LLMs: MCLASH, a culturally situated multilingual moral decision-making benchmark; MET, a two-step prompting method grounded in psychological and philosophical moral theory; and MET-D, a self-distillation training variant requiring no external supervision. MET-D improves macro-F1 by an average of 3.71 points on MCLASH and 4.23 on MMoralExceptQA across Qwen3-4B, Qwen3-8B, and Gemma3-4B, with a peak gain of 12.94 points for Malay on Qwen3-8B. The work also finds that native-language reasoning increases by 62 points on average and that beneficial moral grounds differ systematically across cultures.
Related guides (3)
Related events (8)
Comprehensive survey of metacognition in LLMs: foundations, methods, and open questions
A preprint from Yale NLP presents the first comprehensive survey of metacognition in large language models, covering how LLMs can exhibit or be endowed with self-monitoring, self-evaluation, and self-regulation abilities. The paper taxonomizes the field, reviews benchmarks and evaluation methods, and surveys techniques to elicit and improve metacognitive capabilities. The authors argue metacognition is increasingly recognized as central to capable, transparent, and reliable AI systems, and identify open challenges and future directions.
Cross-lingual evaluation framework reveals LLMs redistribute cultural narrative structure while preserving semantic meaning
A new arXiv preprint introduces a multilingual evaluation framework using 414 proverbs across 15 languages to assess whether LLMs preserve culturally grounded meaning when generating narratives. Using four LLMs to produce 13k narratives, the study finds that cross-lingual prompting preserves proverb-level semantic meaning but systematically redistributes agency, social positioning, and narrative structure. Strong inter-model convergence across architectures suggests multilingual LLMs rely on shared semantic abstractions. The authors argue that semantic similarity metrics alone overestimate cultural preservation in multilingual evaluations.
Moral Semantics Survive Machine Translation: Cross-Lingual Evidence from Moral Foundations Corpora
This paper investigates whether LLM-based machine translation can preserve moral semantic content well enough to enable cross-lingual moral values classification, using Polish as a test case with ~50k annotated social media posts. A four-method validation pipeline (LaBSE embedding similarity, CKA, LLM-as-judge, and classifier parity) shows mean cosine similarity of 0.86 and AUC gaps of only 0.01–0.02 across Moral Foundations categories. The results suggest machine translation is a practical path to extending moral values NLP research to under-resourced languages, with expected generalization to related Slavic languages.
ETCHR: Decoupled Image Editing for Visual Chain-of-Thought Reasoning in MLLMs
ETCHR introduces a question-conditioned, reasoning-aware image editing model that decouples visual transformation from downstream understanding in multimodal LLMs. It addresses two identified gaps—language-side (mapping abstract questions to visual edits) and generation-side (edit quality degrading with reasoning depth)—via a two-stage training recipe combining supervised fine-tuning on edit trajectories and VLM-derived reward signals. Because the editor is decoupled, it plugs into arbitrary MLLMs without retraining, yielding Pass@1 gains of roughly +4.6 to +5.5 points across five task families when paired with Qwen3-VL-8B, Gemini-3.1-Flash-Lite, and Kimi K2.5. The work advances the 'think with images' paradigm beyond fixed toolkits and unified multimodal approaches.
Towards Reliable Multilingual LLMs-as-a-Judge: An Empirical Study
This paper systematically investigates strategies for extending LLM-based automatic evaluation (LLMs-as-a-Judge) to multilingual settings, covering high-, mid-, and low-resource languages (English, Spanish, Basque). The authors compare instruction translation, monolingual vs. multilingual supervision, and model size, finding that fine-tuned smaller models can match proprietary models when in-domain data is available, while zero-shot larger models are preferable out-of-domain. Two meta-evaluation datasets are extended to Spanish and Basque, and all data and code are publicly released.
AMALIA-9B study finds LLM annotation agreement does not imply construct validity for moral foundation coding
A new arXiv paper tests whether Portugal's publicly funded AMALIA-9B language model can validly annotate moral foundation constructs (specifically authority) in European Portuguese text, not just agree with human coders. Using a 'recovery gap' methodology — comparing holistic prompting against decomposed codebook-clause prompting — the authors find AMALIA recovers only about half its holistic performance under decomposition, suggesting reliance on surface correlates rather than genuine construct understanding. A larger open multilingual model closes the same gap on the same corpus, pointing to model-level rather than corpus-level failure. The paper argues that sovereign/national LLM benchmark batteries should test the evidential route to agreement, not just agreement rates.
Tracing the Emergence of Human-Like Pragmatic Reasoning in LLMs Across Languages
Researchers conducted a population-matching experiment evaluating 25 LLMs on conditional inference tasks across four languages, comparing model behavior to matched human populations. The study finds that LLMs function as accurate semantic operators but systematically fail to capture pragmatic enrichments—context-sensitive inferences beyond literal logical meaning—that humans apply effortlessly. Model performance on pragmatic reasoning is not predicted by open vs. closed weights, training orientation, or architecture type, suggesting pragmatic reasoning remains an emergent and unreliable capability. The findings contribute to ongoing debates about whether LLMs reason like humans or merely approximate surface-level linguistic patterns.
The Shibboleth Effect: Cross-lingual behavioral skew in frontier LLMs under adversarial geopolitical simulation
Researchers introduce the 'Shibboleth Effect' — systematic behavioral differences in LLMs when operating in different languages — and audit six frontier models (GPT-4o, Llama-4, Mistral-Large, Gemini-3.1-Pro, Qwen3.6-Plus, DeepSeek-R1) using a synthetic maritime territorial dispute wargame played in English versus Turkish. Results are heterogeneous: Llama-4 becomes significantly more coercive in Turkish while Gemini-3.1-Pro and DeepSeek-R1 become less so, and GPT-4o shows no detectable shift. The study identifies two candidate buffering mechanisms — chain-of-thought institutional anchoring and multilingual RLHF alignment — with direct implications for deploying LLMs in diplomatic or crisis-management contexts.