paper

The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs

paperactiveprovisionalthe-masked-advantage-uncovering-local-language-access-to-cultural-knowledge-in-llms-71dd19eb·1 events·first seen 9d ago

Aliases: The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs

Co-occurring entities

item response theory

More like this (12)

From Self to Other: Evaluating Demographic Perspective-Taking in LLM Hate Speech Annotation Beyond Third-Person Audits: Situated Interaction Auditing for User-Centered LLM Bias Research long-context LLMs Measuring Epistemic Resilience of LLMs Under Misleading Medical Context Security and Privacy Prompts in the Wild: What Users Ask LLMs and How LLMs Respond SpeechLLM Flaws in the LLM Automation Narrative Revising Context, Shifting Simulated Stance: Auditing LLM-Based Stance Simulation in Online Discussions frontier LLMs LLM-based content classification Human Adults and LLMs as Scientists: Who Benefits from Active Exploration?LLM inference

Recent events (1)

6arXiv · cs.CL·9d ago·source ↗

Study finds local languages provide better cultural knowledge access in LLMs once proficiency is controlled

A new arXiv paper introduces a controlled evaluation framework to disentangle language proficiency from culture-specific knowledge access in LLMs. Using real-world cultural questions across 13 locales and ~80 models, the authors apply item response theory to show that while English dominates on culture-agnostic questions, local languages yield a consistent knowledge-access advantage on culture-specific questions once proficiency differences are factored out. The finding challenges the common interpretation that weaker local-language accuracy implies weaker cultural knowledge, and has implications for how multilingual and regionally-aligned models are evaluated.

Evaluation and Benchmarking The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs item response theory