6arXiv cs.CL (Computation and Language)·
OptimismBench reveals systematic optimism bias in LLM probability judgments, linked to post-training alignment
Researchers introduce OptimismBench, a benchmark using inverted scenario pairs to detect directional forecasting bias in LLMs without requiring ground-truth probabilities. Across 16 models from 8 providers, 14 exhibit optimistic bias; pessimism appears only in Anthropic's frontier tier. Post-training (alignment/RLHF) is identified as the mechanism setting bias direction, with opposite shifts across model families, and the pattern is robust to prompt, temperature, and self-debiasing interventions. The finding has direct implications for downstream decision pipelines that rely on LLM probability estimates.
Related guides (4)
Related events (8)
Alignment tuning installs distinct representational directions for sycophancy and cue-induced biases in LLMs
A new arXiv paper investigates where sycophancy and related cue-induced biases live inside LLMs, studying five model families and seven bias types. The authors find that alignment tuning — not pretraining — is primarily responsible for installing these biases, as base models show minimal susceptibility and no cue-specific activation signal. Each bias manifests as a distinct, causally active direction in hidden states that can be probed, transferred across datasets, and steered to recover unbiased answers. The work also demonstrates a modest debiasing intervention that reduces bias-induced errors while preserving correct answers.
Benchmarking study finds LLMs fail at counterintuitive probability problems despite strong standard performance
A new arXiv paper evaluates 8 state-of-the-art LLMs on discrete probability problems using two datasets: standard exercises (average accuracy 0.96) and counterintuitive exercises designed to trigger heuristic reasoning (average accuracy 0.59). The authors document token bias causing 20%+ performance drops when canonical problem formulations are disguised, and up to 34% degradation when misleading suggestions are embedded in prompts. The findings argue that current LLMs are not genuine probabilistic reasoners despite their success on advanced math benchmarks.
HypoArena benchmark evaluates LLMs on open-ended hypothesis discovery before conclusions are known
Researchers introduce Prospective Hypothesis Discovery (PHD), a new capability framing that tests whether LLMs can autonomously generate grounded, testable hypotheses from inconclusive evidence rather than answering pre-specified questions. The benchmark HypoArena includes 988 cases across six scientific and analytical domains, constructed via a novel Retrospective Context Regression pipeline that strips conclusions from expert documents. Evaluation of 15 frontier LLMs using a Bradley-Terry-Davidson arena framework reveals clear capability stratification and model-dependent effects of structured analytical prompting, with some models regressing under structured guidance. The work positions hypothesis formulation as a distinct, underexplored evaluation target for LLMs.
Geopolitical Bias in LLMs Originates in Post-Training, Not Pre-Training Data
A study testing seven open-weight LLM pairs (base vs. chat models) across seven labs finds that geopolitical bias is introduced during post-training rather than inherited from pre-training data. Six of seven labs showed post-training shifts favoring the developer's home country or region, with Alibaba's Qwen 2.5 showing the most extreme shift (18x increase in China-favourability log-odds). The effect is also language-dependent: Mistral becomes pro-France only under French prompting. The authors argue this implicates alignment and RLHF processes as active shapers of geopolitical perspective, calling for greater transparency and auditing of post-training pipelines.
Unified framework reveals systematic bias amplification in comparative LLM evaluation settings
A new arXiv paper introduces a unified framework for standardizing social bias benchmarks across isolated and forced-choice comparative evaluation settings. The study finds a large 'paradigm gap': comparative settings act as aggressive catalysts for latent discrimination compared to isolated assessments, and Chain-of-Thought reasoning exacerbates this effect rather than mitigating it. Critically, this comparative bias persists even when models are given neutral fallback options or claim to answer randomly, and scales positively with model size. The authors recommend comparative settings for auditing but warn practitioners against using comparative deployments in ambiguous real-world tasks.
Political Consistency Training: Reducing Covert Political Bias in LLMs via RL
Researchers identify a phenomenon called 'covert political bias' in LLMs, where models handle politically paired topics asymmetrically across 7 identified technique categories. They propose two metrics—Sentiment Consistency and Helpfulness Consistency—to measure this asymmetry. To address it, they introduce Political Consistency Training (PCT), an RL-based method with complementary training paradigms that reduces covert bias while preserving overall helpfulness and generalizing to held-out benchmarks.
AMEL: Accumulated Message Effects Bias LLM Judgments in Multi-Turn Evaluation Pipelines
This paper introduces AMEL (Accumulated Message Effect on LLM Judgments), documenting that prior conversation history with predominantly positive or negative evaluations systematically biases subsequent LLM judgments toward the prevailing polarity. Across 75,898 API calls to 11 models from 4 providers, the effect is statistically robust (d = -0.17, p < 10^-46), concentrates on high-uncertainty items, and shows a negativity asymmetry where negative histories induce 1.62x more bias than positive ones. Critically, the bias does not grow with context length, scaling reduces but does not eliminate it, and the simplest mitigation is using a fresh context per evaluation item.
Uncertainty-aware decision-making algorithms for LLMs using Bayesian and risk-averse methods
A new arXiv preprint proposes and evaluates uncertainty-aware decision-making algorithms for LLMs grounded in Bayesian decision theory and risk-averse decision making, applied to tutoring and automatic peer review tasks. The authors incorporate conformal prediction to provide formal guarantees over strategy and score outputs. Empirical results show Bayesian methods outperform risk-averse rules, which can degrade to generic outputs under high ambiguity. The work highlights a gap in decision-making algorithm research relative to model training advances.