6arXiv cs.CL (Computation and Language)·
Aggregate accuracy masks per-example prediction instability from task-irrelevant context in LLMs
A new arXiv paper demonstrates that state-of-the-art LLMs appear robust to task-irrelevant context at the aggregate level, but this stability conceals significant per-example prediction flips. Even semantically meaningless pseudo-words prepended to benchmark questions can shift model predictions on a subset of examples, with gains and losses roughly canceling out in aggregate. The instability is modulated by context type, length, test-time compute, and model development stage, and the affected examples are largely model-specific. The authors argue this reveals 'tail risks' hidden by standard aggregate accuracy metrics, motivating per-example reliability evaluation.
Related guides (2)
Related events (8)
Semantic vs. Surface Noise in LLM Agents: 68-Cell Measurement Study with Held-Out Validation
This paper documents an empirical phenomenon across 10 LLMs from 7 architecture families: meaning-bearing perturbations (paraphrase, synonym substitution) cause final-answer inconsistency ~19.69 percentage points more often than presentation-level perturbations (formatting, reordering) of comparable severity, across GSM8K, MATH, and HotpotQA benchmarks. The effect is validated on a held-out 11th model (qwen2.5-14B-Instruct) with 1,800 trajectories. Trace-level analysis supports a 'stealth-divergence' picture where semantic perturbations preserve the first action but induce divergence in intermediate reasoning steps, while two prior mechanism claims are explicitly retracted. The study is notable for its honest reporting of stress-test failures and pre-registered replication.
Benchmarking study finds LLMs fail at counterintuitive probability problems despite strong standard performance
A new arXiv paper evaluates 8 state-of-the-art LLMs on discrete probability problems using two datasets: standard exercises (average accuracy 0.96) and counterintuitive exercises designed to trigger heuristic reasoning (average accuracy 0.59). The authors document token bias causing 20%+ performance drops when canonical problem formulations are disguised, and up to 34% degradation when misleading suggestions are embedded in prompts. The findings argue that current LLMs are not genuine probabilistic reasoners despite their success on advanced math benchmarks.
Systematic Study of LLM Linguistic Uncertainty Markers and Intrinsic Confidence Calibration
This paper introduces 'marker internal confidence' (MIC) as a formalization of the intrinsic confidence a model associates with epistemic markers (e.g., 'it is likely...') in a given task domain. The authors present 7 metrics to evaluate MIC stability within and across distributions, finding that LLMs remain miscalibrated even under model-centric interpretation of marker meanings. Models struggle to differentiate markers by internal confidence across distributions, though they preserve a somewhat consistent ranking order across tasks. The work provides complementary evidence toward understanding faithful calibration in LLMs and highlights the need for more stable, aligned marker use.
New paper exposes behavioral divergence in quantized LLMs beyond accuracy/perplexity metrics
A new arXiv preprint argues that standard accuracy and perplexity metrics fail to capture behavioral changes induced by post-training quantization. The authors introduce 'correctness agreement', a decision-level metric measuring overlap in correct predictions between base and quantized models, and find behavioral divergence emerges even when task performance appears preserved. Analysis of attention weight distortions reveals non-linear breakpoints at low bit-widths and that query/key projections are more sensitive to quantization than value/output projections. The findings challenge the assumption that quantized models are behaviorally equivalent to their base counterparts.
LMs encode knowledge in task-specific parameter subsets, undermining the knowledge-base analogy
A new arXiv paper investigates whether language models satisfy the consistency property of knowledge bases — that the same fact returns consistent results regardless of query form. Behavioral and mechanistic analyses reveal that LMs encode knowledge in a task-specific manner: facts acquired on one task frequently fail to transfer to others during training, and distinct parameter subsets underlie the same fact across different tasks. The authors also show that chain-of-thought reasoning derives part of its effectiveness by engaging task-specific parameters beyond those tied to the evaluation task, with implications for factual reliability and model controllability.
Auditing LLM-as-Judge reliability: judge upgrades are not interchangeable across model families
A new arXiv paper investigates measurement validity problems in LLM-as-judge evaluation, finding that swapping evaluator models changes scores even when candidate responses are fixed. Across four judgment datasets, the authors compare Qwen3 dense judges (1.7B–32B) and MiniMax M2/M2.7 API releases, finding that only the Qwen3 1.7B→4B upgrade yields robust adjacent gains while MiniMax adjacent releases do not. Stronger judges reduce but do not eliminate position and verbosity bias, and repeated-sample juries add little when errors are correlated. The paper argues for standardized reporting requirements including dataset slices, bias probes, error-dependence estimates, and protocol audit trails.
Paper challenges LLM expert-level claims by measuring variance and error magnitude in code-based data analysis tasks
A new arXiv paper argues that standard LLM benchmarks overstate model capabilities by focusing on average performance on training-data-adjacent tasks while ignoring response variance and error magnitude. The authors introduce a novel benchmark requiring frontier LLMs to write code for data analysis tasks, comparing results against human expert submissions. Human experts outperformed the frontier LLM on average across multiple metrics and showed lower performance variability. The findings challenge the prevailing narrative that LLMs perform at human-expert level on knowledge economy tasks.
LLMs judge worse than they generate: empirical challenge to self-evaluation pipeline assumptions
A new arXiv preprint tests the implicit assumption that LLM evaluation is easier than generation, using a controlled in-context QA setup across four benchmarks (SQuAD 2.0, DROP, HotpotQA, MuSiQue) and two models. Results show generation accuracy exceeds self-evaluation accuracy on three of four benchmarks, with attention analysis revealing that evaluation attends to context 3–5x less than generation does. LoRA fine-tuning experiments confirm the asymmetry is not a training artifact, with cross-task interference observed in both directions. The findings directly challenge assumptions underlying LLM-as-a-Judge and self-evaluation pipelines widely used in RLHF and agentic systems.