6arXiv cs.AI (Artificial Intelligence)·
AutoSynthesis: Multi-agent system automates end-to-end quantitative meta-analysis
Researchers introduce AutoSynthesis, an end-to-end multi-agent system that automates the full meta-analysis pipeline: search strategy formulation, literature retrieval, study screening, eligibility assessment, data extraction, effect size computation, and random-effects meta-analysis. The system also supports heterogeneity analysis, risk-of-bias assessment, and produces PRISMA-aligned reports. In evaluation, AutoSynthesis screened over 28 studies and produced pooled effect estimates closely matching expert-conducted meta-analyses (Hedges' g). The work demonstrates that agentic AI can meaningfully scale quantitative evidence synthesis for science, medicine, and policy applications.
Related guides (2)
Related events (8)
Autodata: Meta-optimized agentic data scientist for high-quality synthetic data generation
Researchers introduce Autodata, a framework that trains AI agents to act as data scientists capable of generating high-quality synthetic training and evaluation data. The method includes a meta-optimization loop (Agentic Self-Instruct) that improves the data scientist agent itself, yielding further performance gains. Experiments on CS research, legal reasoning, and mathematical reasoning tasks show improvements over classical synthetic data methods. The authors frame this as a path to converting inference compute into higher-quality training data.
MetaSyn benchmark reveals critical screening bottleneck in LLM-based meta-analysis pipelines
Researchers introduce MetaSyn, a dataset of 442 expert-curated meta-analyses from Nature Portfolio journals, paired with a 140k-article PubMed retrieval corpus, PI/ECO criteria, verified positives, and hard negatives. Benchmarking twelve pipeline configurations — nine RAG variants and a protocol-driven agent — shows that despite 90.9% retrieval recall at K=200, no system recovers more than 52.7% of ground-truth included studies. The core failure is LLMs' inability to reliably distinguish eligible studies from topically similar but criteria-failing distractors. The paper argues that end-to-end scores obscure where pipelines break down and proposes stage-attributed metrics.
SynAE: Framework for Evaluating Synthetic Data Quality in Tool-Calling Agent Benchmarks
SynAE is a proposed evaluation framework for measuring how well synthetic datasets replicate and augment real data trajectories for multi-turn, tool-calling agent testing. It assesses validity, fidelity, and diversity across four metric categories: task instructions, tool calls, final outputs, and downstream evaluation. The paper demonstrates that no single metric suffices to characterize synthetic data quality, motivating multi-axis evaluation. A demo and code are publicly available.
Benchmark Agent: Autonomous system for end-to-end benchmark construction
Researchers introduce Benchmark Agent, a fully autonomous agentic system that orchestrates the complete benchmark construction pipeline — from query analysis and subtask design to data annotation and quality control. The system was used to produce 15 benchmarks spanning text understanding, multimodal understanding, and domain-specific reasoning, with evaluation via human judges, LLM-as-a-judge, and consistency checks. The work addresses two persistent problems in the field: the labor intensity of benchmark creation and rapid performance saturation after release. Code and a demo will be publicly released.
last30days-skill: AI agent skill for multi-source research synthesis
A Python-based AI agent skill on GitHub that queries Reddit, X, YouTube, Hacker News, Polymarket, and the web to research any topic, then synthesizes a grounded summary. The repository has accumulated 27,522 stars with 173 added today, indicating significant community traction. It represents a practical agent tool for multi-source information aggregation.
Anthropic launches Claude Science, an AI workbench for scientific research
Anthropic released Claude Science in beta, an AI-powered research environment integrating over 60 curated scientific tools and databases for genomics, proteomics, structural biology, and cheminformatics. The platform features a coordinating agent with specialist sub-agents, a reviewer agent for citation and calculation checking, reproducible auditable artifacts, and flexible compute management across local machines, HPC clusters, and on-demand GPUs. It integrates with NVIDIA's BioNeMo Agent Toolkit, connecting to models like Evo 2, Boltz-2, and OpenFold3. Available to Claude Pro, Max, Team, and Enterprise users, this represents Anthropic's most significant expansion into scientific AI tooling.
OpenAI Introduces Deep Research Agent
OpenAI has launched 'deep research,' an agentic capability that uses reasoning to synthesize large volumes of online information and complete multi-step research tasks autonomously. The feature is initially available to ChatGPT Pro users, with rollout to Plus and Team tiers to follow. It represents a step toward practical autonomous research agents built on OpenAI's reasoning model infrastructure.
AutoForest: End-to-End LLM System for Automated Forest Plot Generation from Biomedical Studies
AutoForest is presented as the first end-to-end system that generates publication-ready forest plots directly from biomedical papers using large language models. The system automatically suggests ICO (Intervention, Comparator, Outcome) elements, extracts outcome data, performs statistical synthesis, and renders forest plots without manual intervention. A user study with clinicians demonstrates its effectiveness on real-world examples, aiming to accelerate systematic review and meta-analysis workflows.