5arXiv cs.CL (Computation and Language)·
GRADRAG: Cross-component prompt adaptation for coordinated multi-agent RAG pipelines
GRADRAG is a new framework that treats a multi-agent RAG pipeline as a computational graph and propagates structured evaluator feedback upstream to iteratively update retriever, graph constructor, and answerer prompts. Unlike prior work that optimizes components in isolation, GRADRAG coordinates improvements across the full pipeline via a Prompt Optimizer and an early-stopping Evaluator. Evaluated on SQUALITY and QMSUM benchmarks under both flat chunk-based and graph-based retrieval paradigms, GRADRAG achieves a 12–15 percentage point net preference margin over one-step refinement baselines in LLM-judged pairwise comparisons.
Related guides (2)
Related events (8)
UMG-RAG: Training-free hybrid retrieval with uncertainty-aware granularity fusion for long-document RAG
Researchers propose Uncertainty-aware Multi-Granularity RAG (UMG-RAG), a training-free hybrid retrieval framework that addresses the tension between large and fine-grained retrieval chunks in RAG pipelines. The system converts dense and sparse retriever scores across multiple chunk granularities into evidence distributions, estimates reliability via entropy, and fuses candidates using query-specific confidence signals. A variant called UMGP-RAG uses fine-grained hits to locate evidence while returning broader parent chunks for coherence. Experiments on QA benchmarks show improved generation quality with no changes to the underlying retriever or generator.
Comparative framework for RAG variants: when GraphRAG and Agentic RAG are actually needed
A new arXiv preprint introduces a systematic evaluation framework comparing nine standardized RAG scenarios across regular RAG, GraphRAG, Modular RAG, and Agentic RAG on semi-structured knowledge bases. The authors propose a novel context engineering method that reduces token usage by 19–53% for GraphRAG and Agentic RAG by addressing context/memory overflow. A key finding is a 'retrieval-generation gap' where expanded retrieval does not proportionally improve generation quality, suggesting retrieval-oriented metrics overstate the benefits of advanced retrieval. The work targets practitioners building production RAG systems and provides data-driven guidance on when to use each variant.
Expert Support Case Study: Bolstering a RAG App with LLM-as-a-Judge
Hugging Face published a case study describing how Digital Green used an LLM-as-a-Judge approach to evaluate and improve a retrieval-augmented generation (RAG) application. The post covers the methodology for using LLMs to score and validate RAG outputs, providing a practical deployment pattern for quality assurance in production AI systems. It serves as a concrete example of enterprise-grade evaluation pipelines built on top of RAG architectures.
GRACE: Graph-Regularized Agentic Context Evolution for reliable long-horizon instruction updates
Researchers introduce GRACE, a method that maintains a deployed LLM agent's persistent system-level instructions as a typed semantic graph rather than flat text, enabling local verification of updates within typed node neighborhoods. Evaluated on a telecom agent harness derived from τ²-bench under distribution shift, GRACE improves pass³ reliability from 0.091 (Gemini 2.5 Flash zero-shot) to 0.673±0.136, surpassing a Gemini 3.1 Pro zero-shot reference of 0.242. The work identifies structural substrate and consolidation mechanisms as key requirements for reliable long-horizon agentic context evolution. The flat-text baseline finishes at 0.191, underscoring the practical gap GRACE addresses.
Grading the Grader: Evaluating automated graders for agentic data analysis systems
A preprint from arXiv investigates the reliability of automated graders for evaluating agentic data analysis systems, which produce complex multi-modal outputs (code, numerical results, diagnostics) that are harder to assess than single-turn LLM responses. The authors apply LAMBDA, a multi-agent data analysis system, to 153 numerical tasks from DSGym and develop a three-layer human-AI grading cascade combining regex matching, LLM-based lenient grading, and human inspection. Key findings include: both automated graders achieve 100% precision, a keyword-anchored extraction pipeline raises strict grader recall by 60 percentage points, and an iterative nudge mechanism raises grading success from 36% to 97%. The work surfaces important methodological lessons for anyone building evaluation pipelines for agentic systems.
GRASP: Plan-Guided Graph Retrieval with Adaptive Fusion and Reranking on Semi-Structured Knowledge Bases
GRASP is a three-stage retrieval framework for semi-structured knowledge bases (SKBs) that combines plan-based graph retrieval, plan-conditioned dense retrieval fusion, and a fine-tuned reranker. It targets applications like product search, academic search, and precision medicine over typed entity-relation graphs. Evaluated on the STaRK benchmarks, GRASP advances average Hit@1 from 62.0 to 73.9, representing a substantial improvement over prior hybrid retrieval systems. Ablation studies confirm the contribution of each component.
Mistral AI: Using LLM-as-a-Judge with Structured Outputs for RAG Evaluation
Mistral AI published a technical guide on evaluating Retrieval-Augmented Generation (RAG) systems using the 'LLM as a Judge' paradigm combined with their structured outputs API feature. The approach implements the RAG Triad framework—context relevance, groundedness, and answer relevance—using Pydantic schemas to enforce machine-readable evaluation outputs. Mistral models serve as both the generator and judge components, enabling scalable automated evaluation without human annotators.
FinSAgent: Multi-agent RAG framework for evidence-grounded SEC filing question answering
Researchers introduce FinSAgent, a multi-agent retrieval-augmented generation framework designed to address 'prior-corpus misalignment' in financial QA over SEC filings. The system combines role-specialized agents anchored to 10-K item structure, database-aware query decomposition, and a learned feature-gated reranker that separates evidential validity from semantic similarity. Evaluated across five offline financial QA benchmarks and a 1,000-user online experiment, FinSAgent outperforms single-agent and multi-agent baselines on retrieval coverage and answer correctness. The work targets a narrow but practically important domain where standard RAG pipelines fail due to structural and terminological mismatch.