5arXiv cs.CL (Computation and Language)·
DeLIVeR: Reinforced knowledge graph exploration for LLM fact-checking
DeLIVeR is a new framework for automated fact-checking that decomposes complex claims into targeted questions and traverses structured Knowledge Graphs for evidence retrieval, optimizing a Planner LLM via Group Relative Policy Optimization (GRPO). Evaluated on LIAR, FEVER, and PolitiFact benchmarks using Qwen2.5-7B, the system achieves F1-scores of 83.73, 84.57, and 79.70 respectively, representing a 10-15% improvement over HippoRAG2. The approach addresses 'query brittleness' in traditional retrieval by framing evidence gathering as a reinforced strategic exploration task, yielding auditable reasoning paths.
Related guides (3)
Related events (8)
CheckRLM: RAG-based framework for checking and correcting factual errors in reasoning chains
CheckRLM is a proposed framework that improves factual reliability in Reasoning Language Models (RLMs) by integrating Retrieval-Augmented Generation to detect and correct knowledge inconsistencies during inference. The system extracts factual claims from reasoning chains, localizes errors, and applies minimal-cost corrections using external knowledge. Experiments show it outperforms existing baselines in mitigating error accumulation in long-horizon reasoning tasks.
MEDIAREF: Public knowledge store for reproducible media background checks in LLM fact-checking
Researchers introduce MEDIAREF, a publicly available knowledge store of web-sourced documents covering 200 media sources, designed to enable reproducible and low-cost media background check (MBC) generation for automated fact-checking systems. The work addresses a gap in RAG-based fact-checking pipelines where retrieved evidence is assumed reliable but may be biased or outdated. The authors evaluate widely used LLMs on the MBC generation task and show MEDIAREF supports higher-quality outputs than proprietary search API alternatives. The contribution is primarily a dataset and methodology for source-critical reasoning in fact verification.
Disagreement among frontier LLMs on real-world fact-checks
A study examines how frontier large language models diverge in their responses to real-world fact-checking queries, surfacing systematic disagreements across models on factual claims. The work appears to benchmark multiple leading models against a set of verifiable facts, revealing inconsistencies that have implications for reliability and deployment. With 475 HN points and 333 comments, the piece has generated substantial community discussion. The findings are relevant to evaluation methodology, model calibration, and trust in AI-generated factual content.
RSF-GLLM: Differentiable graph reasoning framework for multi-hop knowledge graph QA
RSF-GLLM is a new framework for multi-hop question answering over knowledge graphs that decouples differentiable graph reasoning from LLM-based answer generation. The core Recurrent Soft-Flow (RSF) module uses a GRU-guided query updater with dynamic gating to traverse semantically dissimilar bridge nodes, with flow sparsity regularization guaranteeing convergence to discrete reasoning paths. Extracted paths are textualized to fine-tune an LLM, grounding generation in factual graph topology. Experiments on WebQSP and CWQ benchmarks show competitive accuracy with improved inference efficiency over LLM-heavy baselines.
SIFT and WSP: Claim-conditioned re-scoring to close the warrant gap in LLM fact-checking
A new arXiv preprint identifies a 'warrant gap' in LLM-based fact-checking systems: models frequently output Supports verdicts whose cited evidence does not actually entail the claim. The authors introduce SIFT, a claim-conditioned re-scoring method for extracted evidence spans, and WSP (Warranted Supports Proportion), an automatic NLI-based metric that checks whether cited warrants entail the claim. Evaluated on FEVER, SciFact, 5PILS, and DP across four open-source backbones, SIFT recovers up to 27.6 accuracy points lost by naive decomposition, while WSP calibrates against human gold evidence at AUC 0.92 and precision 0.98.
CommunityFact: A Dynamic, Multilingual, Multi-domain Benchmark for Misinformation Detection in the Wild
CommunityFact is a refreshable benchmark for misinformation detection containing 15,992 standalone claims across five languages and two domains, designed to address limitations of static benchmarks. The authors evaluate ten LLMs under varying inference-time conditions including chain-of-thought reasoning and web-search augmentation, finding that web access yields the largest performance gains. A key finding is that web-enabled LLMs' source-selection policies are systematically misaligned with sources that human Community Notes raters converge on, a gap addressable through retrieval expansion or pruning. The benchmark also proposes using Community Notes as a training signal for claim-conditioned source suggesters.
Co-LMLM: Continuous-query limited memory language models outperform vanilla LLMs on factual tasks at small scale
Researchers introduce CO-LMLM, a limited memory language model that externalizes factual knowledge to a knowledge base during pretraining and retrieves it at inference via continuous vector queries paired with human-readable text values. The approach removes prior restrictions to relational knowledge bases and Wikipedia-only data by introducing an annotation pipeline for arbitrary text. At 360M parameters, CO-LMLM achieves lower perplexity than models trained on 40x more data and SimpleQA factual performance comparable to GPT-4o mini and above Claude Sonnet 4.5, suggesting significant efficiency gains for factual grounding.
Question-type-aware LLM pipeline with multi-agent collaboration achieves top factoid performance on BioASQ 14b
Researchers present a question-type-specific LLM framework for the BioASQ 14b biomedical QA challenge, applying distinct inference strategies to yes/no, factoid, and list questions rather than a single unified approach. For list questions, a multi-agent architecture handles evidence extraction, candidate generation, verification, and aggregation collaboratively. The framework achieved first place in the factoid subtask of Batch 4 in the official BioASQ 14b evaluation, demonstrating competitive performance across multiple batches.