Entity · paper

Estimating Uncertainty from Reasoning: A Large-Scale Study of Multi- and Crosslingual MCQA Performance in LLMs

paperactiveprovisionalestimating-uncertainty-from-reasoning-a-large-scale-study-of-multi-and-crosslingual-mcqa-performance-in-llms-b21c10ea·1 events·first seen 26h ago

Aliases: Estimating Uncertainty from Reasoning: A Large-Scale Study of Multi- and Crosslingual MCQA Performance in LLMs

More like this (12)

Can LLMs Judge Better Than They Generate? Evaluating Task Asymmetry, Mechanistic Interpretability and Transferability for In-Context QA Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs The Riddle Riddle: Testing Flexible Reasoning in Large Language Models and Humans Quantifying Faithful Confidence Expression in Large Reasoning Models Reasoning Language Models Long-context Reasoning Benchmarks Multilingual Reasoning Cascades Need More Context Uncertainty Quantification for Computer-Use Agents: A Benchmark across Vision-Language Models and GUI Grounding Datasets Does Reasoning Preserve Alignment? On the Trustworthiness of Large Reasoning Models Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models The Shibboleth Effect: Auditing the Cross-Lingual Distributional Skew of Large Language Models Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction

Recent events (1)

5arXiv · cs.CL·26h ago·source ↗

Large-scale study of uncertainty estimation across 22 languages in LLMs finds English reasoning improves low-resource UE

A new arXiv preprint presents the first large-scale evaluation of uncertainty estimation (UE) methods across 22 languages, covering high-, mid-, and low-resource settings using nine open- and closed-box UE methods. Key findings include that prompting models to reason in English while keeping questions in low-resource languages substantially improves UE performance, suggesting the reliability bottleneck is in generation rather than comprehension. The study also finds that optimal UE method choice depends on model scale: probability-based open-box methods win at smaller scales, while self-verbalized closed-box uncertainty dominates at larger scales. Practical guidance on threshold selection for abstention in multilingual settings is also provided.

Evaluation and Benchmarking Estimating Uncertainty from Reasoning: A Large-Scale Study of Multi- and Crosslingual MCQA Performance in LLMs