3arXiv cs.CL (Computation and Language)·
Multilingual high-order question generation using alternative pedagogical frameworks beyond Bloom's Taxonomy
A new arXiv preprint evaluates LLM-generated high-order questions in Basque, Spanish, and English using two alternative pedagogical frameworks—Claim-Evidence-Reasoning and Divergent Questioning—as alternatives to the commonly used Bloom's Taxonomy. Both an open-source and a proprietary model generate questions effectively across all three languages, though teachers only recognize roughly half of answerable questions as genuinely high-order. The alternative frameworks produce structurally varied questions that could complement each other in educational settings.
Related guides (1)
Related events (8)
Cross-lingual prompting strategies unlock hidden parametric knowledge in LLMs
A new arXiv preprint investigates how cross-lingual prompting can surface factual knowledge that standard inference techniques fail to retrieve in multilingual LLMs. The authors identify four dimensions of cross-lingual exploration governing parametric knowledge retrieval and evaluate them on multilingual factual benchmarks across 17 typologically diverse languages. Results show cross-lingual exploration improves both factual recall and cross-lingual consistency, and is claimed to be a more compute-efficient approach than scaling native-language inference.
Cross-lingual evaluation framework reveals LLMs redistribute cultural narrative structure while preserving semantic meaning
A new arXiv preprint introduces a multilingual evaluation framework using 414 proverbs across 15 languages to assess whether LLMs preserve culturally grounded meaning when generating narratives. Using four LLMs to produce 13k narratives, the study finds that cross-lingual prompting preserves proverb-level semantic meaning but systematically redistributes agency, social positioning, and narrative structure. Strong inter-model convergence across architectures suggests multilingual LLMs rely on shared semantic abstractions. The authors argue that semantic similarity metrics alone overestimate cultural preservation in multilingual evaluations.
Study finds local languages provide better cultural knowledge access in LLMs once proficiency is controlled
A new arXiv paper introduces a controlled evaluation framework to disentangle language proficiency from culture-specific knowledge access in LLMs. Using real-world cultural questions across 13 locales and ~80 models, the authors apply item response theory to show that while English dominates on culture-agnostic questions, local languages yield a consistent knowledge-access advantage on culture-specific questions once proficiency differences are factored out. The finding challenges the common interpretation that weaker local-language accuracy implies weaker cultural knowledge, and has implications for how multilingual and regionally-aligned models are evaluated.
Towards Reliable Multilingual LLMs-as-a-Judge: An Empirical Study
This paper systematically investigates strategies for extending LLM-based automatic evaluation (LLMs-as-a-Judge) to multilingual settings, covering high-, mid-, and low-resource languages (English, Spanish, Basque). The authors compare instruction translation, monolingual vs. multilingual supervision, and model size, finding that fine-tuned smaller models can match proprietary models when in-domain data is available, while zero-shot larger models are preferable out-of-domain. Two meta-evaluation datasets are extended to Spanish and Basque, and all data and code are publicly released.
Large-scale study of uncertainty estimation across 22 languages in LLMs finds English reasoning improves low-resource UE
A new arXiv preprint presents the first large-scale evaluation of uncertainty estimation (UE) methods across 22 languages, covering high-, mid-, and low-resource settings using nine open- and closed-box UE methods. Key findings include that prompting models to reason in English while keeping questions in low-resource languages substantially improves UE performance, suggesting the reliability bottleneck is in generation rather than comprehension. The study also finds that optimal UE method choice depends on model scale: probability-based open-box methods win at smaller scales, while self-verbalized closed-box uncertainty dominates at larger scales. Practical guidance on threshold selection for abstention in multilingual settings is also provided.
Benchmark study reveals how linguistic framing of user beliefs shifts LLM context-following behavior
A new arXiv paper introduces a typology of 17 linguistically motivated expression-of-belief (EoB) types—spanning form, evidentiality, epistemic stance, and tone—to evaluate how phrasing affects whether LLMs defer to user-stated beliefs or their own prior knowledge. The authors benchmark 16 LLMs across Llama 3, Qwen3, and Gemma3 families at scales from 1B to 30B parameters, finding that larger and instruction-tuned models are systematically less context-following than smaller or base models. Specific linguistic framings (e.g., presuppositions, certainty markers) are identified as statistically more persuasive, with implications for prompt robustness and sycophancy research.
CURIOBOT framework uses LLM tutoring dialogues to operationalize curiosity-driven learning interventions
Researchers introduce CURIOBOT, a conversational tutoring framework that implements Berlyne's collative variables (novelty, complexity, conflict, uncertainty) as adaptive linguistic interventions via LLMs. Across 270 tutoring conversations, curiosity-oriented prompting strategies produced up to 2.4x more exploratory conversational turns under fixed time budgets. The study also introduces a learner-centered evaluation framework measuring exploratory questioning, conversational agency, and productive struggle. Results suggest curiosity functions as a partially independent interaction-level mechanism, and that LLM-mediated dialogue can serve as a scalable experimental platform for studying language's effect on cognition.
BINEVAL: Binary question decomposition for interpretable LLM evaluation and prompt optimization
Researchers introduce BINEVAL, a framework that decomposes LLM evaluation criteria into atomic binary yes/no questions, aggregating answers into multi-dimensional interpretable scores. The approach matches or outperforms baselines including UniEval and G-Eval on SummEval, Topical-Chat, and QAGS benchmarks, with particular strength on factual consistency. Beyond evaluation, the binary question feedback is shown to support iterative prompt optimization in both self-update and cross-model settings on IFBench. The framework is training-free and task-agnostic, addressing opacity and ceiling-effect problems common in holistic LLM judges.