4arXiv cs.CL (Computation and Language)·
SciDiagramEdit: Benchmark and agentic framework for editing scientific figures from natural-language instructions
Researchers introduce SciDiagramEdit, a benchmark and skill-evolution framework for automating edits to scientific figures under natural-language instructions. The benchmark mines before/after figure pairs from arXiv version histories, grounding training in authors' own revision intent. An agentic proposer iteratively refines the agent's skill specification from execution traces, progressively improving edit accuracy on held-out validation. The work demonstrates that natural paper revisions are an effective training signal for instruction-driven figure editing.
Related guides (2)
Related events (8)
SciFigQual-Bench: A benchmark for scientific figure quality assessment with full-manuscript context
Researchers introduce SciFigQual-Bench, a benchmark for evaluating scientific figures across five dimensions—clarity, layout, caption fit, context relevance, and misleading risk—using 6,308 expert-annotated images from top CS conferences (2020–2025). Each image is bound to its caption, citing sentence, and manuscript context, addressing gaps in existing IQA methods that focus on natural photos or AI-generated content. The authors also propose SFQ-Agent, a staged cross-modal evaluation framework that, when equipped with GPT-5.6-Sol, achieves the lowest average absolute error (0.418) and highest consistency rate (93.4%) on the eval1200 test subset. The work targets automated scientific figure quality auditing, a narrow but underserved evaluation niche.
Survey on instruction-based image editing: taxonomy, benchmarks, and CDD-IIE Bench
A new arXiv survey systematically reviews instruction-based image editing (IIE) research across five dimensions: task taxonomy, training data construction, architectural evolution (GAN to diffusion to autoregressive), evaluation metrics, and commercial solutions. The authors introduce CDD-IIE Bench, a new diagnostic benchmark designed to rigorously assess multiple aspects of IIE model performance. The survey covers the role of LLMs and VLMs in enabling practical one-sentence image editing systems and compares open-source solutions empirically.
ToolSciVer: Tool-augmented reinforcement learning for multimodal scientific claim verification
Researchers introduce ToolSciVer, a framework that equips vision-language models with three type-aware visual tools (table focus, chart-to-structure parsing, high-resolution zoom) to verify scientific claims grounded in figures, tables, and charts from papers. The policy is trained using Group Relative Policy Optimization (GRPO) with a composite reward covering correctness, format, tool-use efficiency, and validity. Experiments across five VLMs from three model families (Qwen, InternVL, Gemma) on SciVer and MuSciClaims benchmarks show improvements over prompting-based and RL-based baselines. The work is notable as the first tool-augmented framework specifically targeting multimodal scientific claim verification.
RSIBench-Data: Benchmark for LLM Agents as Data-Centric Researchers in Recursive Self-Improvement
RSIBench-Data is a new benchmark that isolates the data-centric research capabilities of LLM agents within a fixed post-training stack, testing whether agents can autonomously diagnose capability gaps and revise training-data strategies to improve a target model. Four frontier agents are evaluated across six benchmarks spanning software engineering, math, and scientific QA. Agents improve on their first valid attempt in 58% of settings, but 78% of runs that continue after a peak score end lower, indicating inconsistent self-improvement. The benchmark is open-sourced and provides a controlled testbed for measuring progress toward recursive self-improvement.
CurateEvo: Failure-Driven Dynamic Data Curation for Agentic LLM Post-Training
CurateEvo is a new framework for agentic post-training that treats data curation as a dynamic, evolving process rather than a fixed preprocessing step. The system represents curation strategies as executable code and iteratively rewrites them based on failed trajectories from a held-out development set, producing SFT data, RL data, and an inference-time memory bank. Evaluated on ACEBench-Agent, BFCL-V4, and τ²-Bench, CurateEvo outperforms prior curation methods by 3.2 and 2.7 average points in labeled and wild-data settings respectively, while also reducing curation overhead.
IdeaGene-Bench: Benchmark for Scientific Lineage Reasoning and Idea Generation in LLMs
Researchers introduce IG-Bench, a benchmark evaluating whether AI systems can trace and reason about the evolutionary inheritance structure of scientific ideas across papers. The benchmark contains 1,961 lineage traces, 1,085 Idea Genome objects, and 920 pairwise GenomeDiff records spanning 10 scientific domains, supporting both closed-form reasoning tasks and lineage-conditioned generation evaluation. Experiments on 14 LLM-based systems reveal a compositional bottleneck: the best system achieves only 27.3% exact accuracy on lineage reasoning, and adding structured lineage context reshuffles system rankings rather than uniformly improving performance.
OpAI-Bench: Benchmark for detecting AI text across progressive human-AI co-editing workflows
Researchers introduce OpAI-Bench, a benchmark for studying AI-text detection across progressive human-to-AI document revision workflows, covering document, sentence, token, and span granularities. Starting from human-written documents, the benchmark constructs nine sequentially revised versions per sample under five AI edit operations and varying AI coverage levels across four domains. Key findings include that mixed-authorship intermediate versions are often harder to detect than fully human or heavily AI-edited endpoints, revealing non-monotonic detection patterns absent from existing benchmarks. The work addresses a gap in AI-text detection research as real-world documents increasingly result from iterative human-AI co-editing rather than pure generation.
Anthropic launches Claude Science, an AI workbench for scientific research
Anthropic released Claude Science in beta, an AI-powered research environment integrating over 60 curated scientific tools and databases for genomics, proteomics, structural biology, and cheminformatics. The platform features a coordinating agent with specialist sub-agents, a reviewer agent for citation and calculation checking, reproducible auditable artifacts, and flexible compute management across local machines, HPC clusters, and on-demand GPUs. It integrates with NVIDIA's BioNeMo Agent Toolkit, connecting to models like Evo 2, Boltz-2, and OpenFold3. Available to Claude Pro, Max, Team, and Enterprise users, this represents Anthropic's most significant expansion into scientific AI tooling.