6arXiv cs.CL (Computation and Language)·
AdvancedMathBench: New benchmark suite for advanced mathematical proof generation and verification
Researchers introduce AdvancedMathBench, a benchmark suite targeting advanced mathematical reasoning beyond olympiad-level problems, comprising ProverBench (296 undergraduate and doctoral qualifying-exam problems) and VerifierBench (888 model-generated proof trajectories). The suite includes an automatic verification pipeline trained on expert annotations to assess proof correctness and error types. Frontier models struggle significantly: the best performer (GPT-5.5-xhigh) achieves only 75.8% and 66.1% on undergraduate and qualifying-exam splits respectively, while proof verification tops out at 65.1 Balanced F1, with models particularly weak at detecting proof errors.
Related guides (4)
Related events (8)
MaxProof achieves gold-medal-level performance on IMO 2025 and USAMO 2026 via population-level test-time scaling
MiniMax introduces MaxProof, a test-time scaling framework for competition-level mathematical proof built on their MiniMax-M3 model. The system trains three capabilities — proof generation, verification, and critique-conditioned repair — then at inference time runs tournament selection over a population of candidate proofs. MaxProof scores 35/42 on IMO 2025 and 36/42 on USAMO 2026, exceeding the human gold-medal threshold on both competitions.
OpenAI Shares First Proof Math Challenge Submissions
OpenAI has published its AI model's proof attempts for the First Proof math challenge, a competition designed to test research-grade mathematical reasoning on expert-level problems. This represents a capability demonstration of OpenAI's models on formal mathematical proof generation. The submission signals continued progress in AI mathematical reasoning at a level approaching or engaging with professional research mathematics.
OpenAI Neural Theorem Prover Solves Formal Math Olympiad Problems in Lean
OpenAI developed a neural theorem prover integrated with the Lean proof assistant that can solve challenging high-school olympiad problems, including problems from AMC12, AIME, and two IMO-adapted problems. The system demonstrates automated formal mathematical reasoning at a level previously requiring human expertise. This represents a significant capability milestone in AI-assisted formal verification and mathematical problem-solving.
Large-Scale Evaluation of LLM-Driven Formal Proof Search on Open Mathematical Problems
Researchers present the first large-scale evaluation of LLM-based formal proof search on genuinely open mathematical problems, using Lean as a verification backend. Their most capable agent autonomously resolved 9 of 353 open Erdős problems and proved 44 of 492 OEIS conjectures, at a cost of a few hundred dollars per problem. The system is already being deployed in active research across combinatorics, optimization, graph theory, algebraic geometry, and quantum optics. The study also compares agent architectures, finding that more sophisticated designs outperform simple generate-and-verify loops on the hardest problems.
DeepSWE, ProgramBench, and ITBench-AA emerge as harder successors to SWE-bench for agent evaluation
Three new benchmarks — DeepSWE (by Datacurve), ProgramBench (Meta/Stanford/Harvard), and ITBench-AA (IBM/Artificial Analysis) — are positioned as more rigorous replacements for the SWE-bench family, which models have largely saturated. DeepSWE tests feature implementation using private codebases and human-written problems; ProgramBench evaluates agents' ability to recreate functional programs from scratch; ITBench-AA measures root-cause diagnosis in real-world IT incident scenarios. Current top performers include GPT-5.5 (70% on DeepSWE), Claude Opus 4.7 (46.7% on ITBench-AA), and Claude Opus 4.7 (3% on ProgramBench at the 95% pass threshold), illustrating that even frontier models have substantial headroom.
AdversaBench: Automated LLM red-teaming pipeline with multi-judge confirmation and cross-model transferability
AdversaBench is a new end-to-end red-teaming pipeline that mutates seed prompts using five structured operators and confirms failures via a three-judge panel with a meta-judge tiebreaker. Experiments on 45 seeds across reasoning, instruction-following, and tool-use categories produced confirmed failures for every seed. Key findings include sharp variation in operator effectiveness by category, misleading binary failure rates, judge agreement metrics distorted by label skew, and zero-shot transferability of adversarial prompts from Llama 3.1 8B to Llama 3.3 70B. Code and dataset are publicly released.
Goedel-Architect achieves state-of-the-art formal theorem proving with blueprint-based agentic framework
Goedel-Architect is an agentic framework for formal theorem proving in Lean 4 that uses blueprint generation — a dependency graph of definitions and lemmas — rather than recursive decomposition, enabling parallel lemma closure and global refinement. Built on DeepSeek-V4-Flash (284B-A13B), it achieves 99.2% pass@1 on MiniF2F-test and 75.6% on PutnamBench, scaling to 100% on MiniF2F, 88.8% on PutnamBench, and 4/6 on IMO 2025 when seeded with natural-language proofs. The authors claim state-of-the-art performance for an open-source pipeline at up to 500x lower cost than comparable systems.
VeriEvol: Verified data construction pipeline for scaling multimodal mathematical reasoning
VeriEvol is a new framework for scaling reinforcement learning on visual mathematical reasoning by decoupling prompt difficulty expansion from answer reliability verification. It uses a type-aware evolution module to generate harder image-grounded prompts and an HTV-Agent verifier that rejects answers only after failing to find counter-evidence. Scaling SFT data from 10K to 250K samples raises mean accuracy from 35.42 to 54.73 across five visual-math benchmarks, with an additional +3.88 cumulative gain over an un-evolved RL baseline when combined with GRPO-style training. The authors release prompts, data, models, code, and full verifier traces.