Entity · paper

The Illusion of Robustness: Aggregate Accuracy Hides Prediction Flips under Task-Irrelevant Context

paperactiveprovisionalthe-illusion-of-robustness-aggregate-accuracy-hides-prediction-flips-under-task-irrelevant-context-0e8daca8·1 events·first seen 16h ago

Aliases: The Illusion of Robustness: Aggregate Accuracy Hides Prediction Flips under Task-Irrelevant Context

More like this (12)

Dynamic-Probabilistic Consistency Gap How Robust is OCR-Reasoning? Evaluating OCR-Reasoning Robustness of Vision-Language Models under Visual Perturbations Expert Blindness Effect Multi-Task Bayesian In-Context Learning Pessimism's Paradox: Conservative Offline Training Amplifies Reward Hacking During Online Adaptation in Reasoning Models Does Bielik Know What It Doesn't Know? Activation Dispersion Separates Entity Familiarity from Factual Reliability Across Model Scale Decomposing Factual Sycophancy in Language Models: How Size and Instruction Tuning Shape Robustness Valid Inference with Synthetic Data via Task Exchangeability When Good Verifiers Go Bad: Self-Improving VLMs Can Regress on New Tasks adversarial robustness Sparse Subspace-to-Expert Sharing for Task-Agnostic Continual Learning Attention Amnesia in Hybrid LLMs: When CoT Fine-Tuning Breaks Long-Range Recall, and How to Fix It

Recent events (1)

6arXiv · cs.CL·16h ago·source ↗

Aggregate accuracy masks per-example prediction instability from task-irrelevant context in LLMs

A new arXiv paper demonstrates that state-of-the-art LLMs appear robust to task-irrelevant context at the aggregate level, but this stability conceals significant per-example prediction flips. Even semantically meaningless pseudo-words prepended to benchmark questions can shift model predictions on a subset of examples, with gains and losses roughly canceling out in aggregate. The instability is modulated by context type, length, test-time compute, and model development stage, and the affected examples are largely model-specific. The authors argue this reveals 'tail risks' hidden by standard aggregate accuracy metrics, motivating per-example reliability evaluation.

Evaluation and Benchmarking AI Safety Research The Illusion of Robustness: Aggregate Accuracy Hides Prediction Flips under Task-Irrelevant Context