paper

The Shibboleth Effect: Auditing the Cross-Lingual Distributional Skew of Large Language Models

paperactiveprovisionalthe-shibboleth-effect-auditing-the-cross-lingual-distributional-skew-of-large-language-models-3303e003·1 events·first seen 7d ago

Aliases: The Shibboleth Effect: Auditing the Cross-Lingual Distributional Skew of Large Language Models

Co-occurring entities

DeepSeek V4 Mistral Large 2 GPT-4o Google DeepMind Llama OpenAI Gemini-3.1-Pro Meta Qwen3.6-Plus

More like this (12)

Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models Automated reproducibility assessments in the social and behavioral sciences using large language models Unintended Effects of Geographic Conditioning in Large Language Models LESS: Mutual-Stability Sampling for Diffusion Language Models Civil Court Simulation with Large Language Models The Neutral Mask: How RLHF Provides Shallow Alignment while Leaving Partisan Structure Intact in a Large Language Model Multimodal Large Language Models Decomposing Factual Sycophancy in Language Models: How Size and Instruction Tuning Shape Robustness Language Models are Few-Shot Learners Scaling Laws for Neural Language Models When English Isn't the Best Teacher: Source Language Effects in Cross-Lingual In-Context Learning large language models

Recent events (1)

6arXiv · cs.CL·7d ago·source ↗

The Shibboleth Effect: Cross-lingual behavioral skew in frontier LLMs under adversarial geopolitical simulation

Researchers introduce the 'Shibboleth Effect' — systematic behavioral differences in LLMs when operating in different languages — and audit six frontier models (GPT-4o, Llama-4, Mistral-Large, Gemini-3.1-Pro, Qwen3.6-Plus, DeepSeek-R1) using a synthetic maritime territorial dispute wargame played in English versus Turkish. Results are heterogeneous: Llama-4 becomes significantly more coercive in Turkish while Gemini-3.1-Pro and DeepSeek-R1 become less so, and GPT-4o shows no detectable shift. The study identifies two candidate buffering mechanisms — chain-of-thought institutional anchoring and multilingual RLHF alignment — with direct implications for deploying LLMs in diplomatic or crisis-management contexts.

Evaluation and Benchmarking AI Safety Research DeepSeek V4 Mistral Large 2 GPT-4o +8 more