5arXiv cs.CL (Computation and Language)·
Inference-time steering methods compared for cross-lingual factual consistency in LLMs
Researchers investigate four intervention strategies for reducing cross-lingual factual inconsistency in LLMs, where models give different factual answers depending solely on prompt language. Testing on Gemma 3 12B Instruct, they compare persona prompting, Contrastive Activation Addition (CAA), and DPO-based adapters, finding that simple persona prompting outperforms more invasive methods in efficacy, safety, and generalization. The paper also introduces a novel culturally-rooted generalization benchmark to assess whether factual interventions transfer beyond the training distribution. Results suggest cross-lingual inconsistency is partly a selection problem addressable without weight modification.
Related guides (3)
Related events (8)
Cross-lingual prompting strategies unlock hidden parametric knowledge in LLMs
A new arXiv preprint investigates how cross-lingual prompting can surface factual knowledge that standard inference techniques fail to retrieve in multilingual LLMs. The authors identify four dimensions of cross-lingual exploration governing parametric knowledge retrieval and evaluate them on multilingual factual benchmarks across 17 typologically diverse languages. Results show cross-lingual exploration improves both factual recall and cross-lingual consistency, and is claimed to be a more compute-efficient approach than scaling native-language inference.
Systematic study reveals effectiveness-fluency trade-offs in LLM conditioning methods
A new arXiv paper systematically evaluates a range of LLM conditioning methods across both concept injection and removal scenarios, finding that efficient steering methods often degrade fluency significantly. A key finding is that activation steering is substantially less effective on instruction-tuned models than on base models, a previously overlooked interaction. Simple prompting and supervised fine-tuning work for concept injection but not removal, and cheap textual metrics are found to correlate well with expensive LLM-as-judge evaluations.
Cross-lingual evaluation framework reveals LLMs redistribute cultural narrative structure while preserving semantic meaning
A new arXiv preprint introduces a multilingual evaluation framework using 414 proverbs across 15 languages to assess whether LLMs preserve culturally grounded meaning when generating narratives. Using four LLMs to produce 13k narratives, the study finds that cross-lingual prompting preserves proverb-level semantic meaning but systematically redistributes agency, social positioning, and narrative structure. Strong inter-model convergence across architectures suggests multilingual LLMs rely on shared semantic abstractions. The authors argue that semantic similarity metrics alone overestimate cultural preservation in multilingual evaluations.
Graded entity-familiarity probes in LLMs enable refusal steering and cross-language robustness
Researchers probe activations at the final prompt token across twelve instruction-tuned models (Bielik, PLLuM, Gemma-4, Qwen3 families) to assess whether LLMs encode graded familiarity with named entities before generating answers. Using a new dataset of 1,440 Polish entities across popularity deciles plus fabricated controls, they find familiarity probes reliably separate real from fabricated entities and track popularity in Polish-adapted models. A key finding is that a single one-dimensional familiarity direction injected at one layer in Gemma-4-12B can steer refusal rates monotonically from 0.24 to 1.00 (or 0.73 to 0.00), revealing a separation between representational familiarity and the policy that converts it into abstention behavior.
Canonical-Context On-Policy Distillation (CCOPD) for Multi-Turn LLM Consistency
This paper identifies 'self-anchored drift' as a key failure mode in multi-turn LLMs: when information is revealed incrementally across turns, models produce unsupported assumptions that distort final answers, even when the total evidence is identical to a single-prompt setting. The authors propose Canonical-Context On-Policy Distillation (CCOPD), which trains a student model on incremental multi-turn conversations to match the output distribution of a frozen teacher conditioned on the full clean prompt. Trained only on math conversations, CCOPD achieves a 32% average relative improvement on multi-turn (RAW-SHARDED) tasks and generalizes zero-shot to five out-of-domain task families while preserving single-prompt performance.
Causal tracing reveals mid-to-late decoder layers drive LLM stance sensitivity to linguistic construction
A new arXiv preprint investigates whether linguistic construction choices (beyond lexical variation) systematically shift LLM stance judgments, using political stance classification as a case study. The authors create six controlled rewrite types that preserve or invert meaning, then apply activation patching across four open-weight models to localize where these shifts originate. Results show that mid-to-late decoder layers, particularly block outputs at the final prompt position, provide the strongest restoration signal for the original stance distribution. The work extends mechanistic interpretability methods to a socially sensitive domain and highlights a largely overlooked axis of LLM prompt sensitivity.
Tracing the Emergence of Human-Like Pragmatic Reasoning in LLMs Across Languages
Researchers conducted a population-matching experiment evaluating 25 LLMs on conditional inference tasks across four languages, comparing model behavior to matched human populations. The study finds that LLMs function as accurate semantic operators but systematically fail to capture pragmatic enrichments—context-sensitive inferences beyond literal logical meaning—that humans apply effortlessly. Model performance on pragmatic reasoning is not predicted by open vs. closed weights, training orientation, or architecture type, suggesting pragmatic reasoning remains an emergent and unreliable capability. The findings contribute to ongoing debates about whether LLMs reason like humans or merely approximate surface-level linguistic patterns.
Towards Reliable Multilingual LLMs-as-a-Judge: An Empirical Study
This paper systematically investigates strategies for extending LLM-based automatic evaluation (LLMs-as-a-Judge) to multilingual settings, covering high-, mid-, and low-resource languages (English, Spanish, Basque). The authors compare instruction translation, monolingual vs. multilingual supervision, and model size, finding that fine-tuned smaller models can match proprietary models when in-domain data is available, while zero-shot larger models are preferable out-of-domain. Two meta-evaluation datasets are extended to Spanish and Basque, and all data and code are publicly released.