5arXiv cs.CL (Computation and Language)·
D-Score: Spectral hidden-state signal for hallucination detection in LLMs
Researchers introduce D-Score, a hallucination detection method based on the spectral geometry of LLM hidden activations, computed in a single forward pass without external verifiers, retrieval, or multiple generations. The method counts singular directions of the hidden activation matrix whose singular values remain close to the leading one, operating on the intuition that hallucinated text causes representations to spread across additional singular directions. Evaluation on FAVA-Annotation and RAGTruth benchmarks shows strong detection performance. The approach is lightweight and model-intrinsic, making it practically attractive for deployment.
Related guides (2)
Related events (8)
CHAIR: Supervised hallucination detection via internal logit analysis across LLM layers
A new arXiv preprint introduces CHAIR (Classifier of Hallucination As ImproveR), a supervised framework that detects hallucinations by extracting statistical features (max, min, mean, std, slope) from token logits across all layers of an LLM. Evaluated on TruthfulQA and MMLU, CHAIR shows improved detection accuracy especially in zero-shot settings. The authors argue the approach also points toward richer internal representations for designing adaptive decoding strategies that reduce hallucinations.
Grad Detect: gradient-based hallucination detection using layer-wise backward pass signals
Grad Detect is a new method for detecting LLM hallucinations by analyzing layer-wise gradient patterns from a single forward-backward pass at inference time, without relying on output-level signals alone. Evaluated across Q&A benchmarks and eleven models from four architectural families, it consistently outperforms confidence-based and sampling-based baselines. A key finding is that the final five layers concentrate over 97% of the discriminative gradient signal, enabling efficient deployment. The method also supports model abstention prediction, framing it as a unified reliability framework.
BenHalluEval: Multi-Task Hallucination Evaluation Framework for Bengali LLMs
BenHalluEval introduces the first systematic hallucination benchmark for Bengali, covering four tasks (generative QA, code-mixed QA, summarization, reasoning) with 12,000 hallucinated candidates generated via GPT-5.4 across twelve hallucination types. Seven LLMs are evaluated under a dual-track protocol separating false-positive rate on ground-truth instances from hallucination detection rate on hallucinated candidates. The proposed BenHalluScore metric reveals substantial variation (7.72%–55.42%) across models and tasks, and chain-of-thought prompting is found to shift response distributions without consistently improving hallucination discrimination. The work highlights gaps in low-resource language hallucination evaluation and critiques single-track and prompting-only evaluation approaches.
ClinHallu benchmark diagnoses stage-wise hallucinations in medical multimodal LLM reasoning
Researchers from Alibaba DAMO Academy introduce ClinHallu, a benchmark of 7,031 validated instances designed to identify where hallucinations originate within medical MLLM reasoning pipelines. Each instance is annotated with a structured reasoning trace decomposed into Visual Recognition, Knowledge Recall, and Reasoning Integration stages, with stage-replacement interventions to measure the causal impact of correcting each stage. The paper also demonstrates that trace-supervised fine-tuning reduces stage-wise hallucinations, offering both diagnostic and mitigation value for clinical AI systems.
The Hallucinations Leaderboard, an Open Effort to Measure Hallucinations in Large Language Models
Hugging Face has launched an open leaderboard specifically designed to benchmark hallucination rates across large language models. The effort aims to standardize evaluation of factual accuracy and confabulation tendencies, filling a gap in existing benchmarks that focus primarily on capability rather than reliability. The leaderboard is positioned as a community-driven, transparent resource for tracking model trustworthiness.
HalluTruthQA: Fine-grained Arabic hallucination benchmark with detection, localization, and explanation tasks
Researchers introduce HalluTruthQA, a 2,400-example expert-curated benchmark for evaluating hallucinations in Arabic LLM outputs across four knowledge domains. Unlike prior benchmarks that provide only response-level labels, it includes character-level erroneous span annotations, human-written explanations, and a multi-task evaluation covering detection, localization, factual verification, and explanation. Four open-source Arabic-capable models (Allam, Falcon-H1, Qwen32, Silma) are evaluated zero-shot, revealing that no single model dominates across all tasks. The benchmark and code are publicly released.
Activation dispersion signals entity familiarity in Bielik models, but doesn't prevent hallucination
Researchers probe four Polish Bielik LLMs (1.5B–11B parameters) to test whether internal activations reveal entity familiarity before any answer token is generated. Using unsupervised dispersion measures over post-SwiGLU MLP activations, they achieve AUROC 0.95–1.00 in separating known from fabricated entities—a signal that is already at ceiling at 1.5B parameters. Despite this internal awareness, the models almost never abstain: only 2 refusals and 1 hedge were found across 2,520 answers. The paper distinguishes entity familiarity (a representational phenomenon stable across scale) from factual reliability (which scales sharply with model size), with implications for hallucination detection and calibration research.
MMBench2 paper: hallucination in world models is predictable and preventable via coverage signals
Researchers introduce MMBench2, a 427-hour, 210-task dataset for visual world modeling, and train a 350M-parameter world model to study hallucination in generative world models. The paper identifies three distinct hallucination modes (perceptual, action-marginalized, scene-diverging) and develops lightweight signals that predict where models will fail. A coverage-aware sampling technique and curiosity-reward-based data collection enable efficient finetuning to unseen environments with as few as 50 real trajectories. The central finding is that world model hallucination is fundamentally a data coverage problem, with the same signals serving both detection and mitigation.