6arXiv cs.LG (Machine Learning)·
APO: Unsupervised atomic policy optimization for 3D structure prediction outperforms supervised baselines
Researchers introduce Atomic Policy Optimization (APO), an unsupervised alignment framework for predicting 3D structures of atomic systems (crystals, antibodies) without requiring ground-truth coordinate labels. APO adapts group-relative policy optimization to 3D atomic environments using a dual-reward mechanism: an eigen-decomposition-based reward reinforcing dominant latent structural modes, and a thermodynamic stability reward. Benchmarks on crystal and antibody structure prediction show APO surpasses fully supervised baselines on match rates and structural fidelity while also improving inference efficiency by straightening probability paths. The work is significant for material science and drug discovery applications where experimental labels are scarce or prohibitively expensive.
Related guides (2)
Related events (8)
SOAP and Muon optimizers outperform Adam for training machine learning interatomic potentials
A new arXiv preprint systematically compares matrix-structured optimizers—Muon, SOAP, and SOAP-Muon—against Adam for training machine learning interatomic potentials (MLIPs), specifically NequIP and Allegro models. SOAP and SOAP-Muon consistently outperform Adam in both convergence speed and final accuracy, with gains especially pronounced under partial force supervision. The paper argues that optimizer choice is an underexplored but impactful design axis for scientific ML models.
APPO: Fine-grained branching and credit assignment for agentic RL in LLMs
Researchers introduce Agentic Procedural Policy Optimization (APPO), a reinforcement learning method that shifts branching and credit assignment from coarse tool-call boundaries to fine-grained decision points within generated sequences. APPO uses a Branching Score combining token uncertainty with policy-induced likelihood gains to select exploration points, plus procedure-level advantage scaling for credit distribution. Evaluated on 13 benchmarks, APPO improves strong agentic RL baselines by nearly 4 points while maintaining efficient tool use and interpretability. The work addresses a known weakness in multi-turn agentic RL: that influential decisions are distributed throughout sequences, not concentrated at tool-call boundaries.
AXPO: Agent Explorative Policy Optimization Addresses Thinking-Acting Gap in Multimodal Agentic Reasoning
This paper identifies a structural asymmetry in agentic reasoning called the 'Thinking-Acting Gap,' where tool use is attempted in only ~30% of rollouts under standard RL training (GRPO), and all-wrong tool-using subgroups suppress learning signals. The authors propose AXPO (Agent eXplorative Policy Optimization), which fixes the thinking prefix and resamples tool calls for all-wrong subgroups, combined with uncertainty-based prefix selection. Evaluated across nine multimodal benchmarks on Qwen3-VL-Thinking at multiple scales, SFT+AXPO outperforms SFT+GRPO by +1.8pp on both Pass@1 and Pass@4 at 8B, with the 8B model surpassing the 32B baseline on Pass@4 using 4× fewer parameters.
GraphPO: Graph-based Policy Optimization reduces redundancy in LLM reasoning RL
GraphPO is a new reinforcement learning framework that represents reasoning rollouts as directed acyclic graphs rather than independent chains or trees, merging semantically equivalent reasoning paths into equivalence classes to share suffixes and reduce redundant exploration. The approach assigns efficiency advantages to incoming edges and correctness advantages to outgoing edges, deriving process supervision from outcome rewards. Experiments on three LLMs across reasoning and agentic search benchmarks show consistent improvements over chain- and tree-based baselines under equal token or response budgets. The method also provides theoretical guarantees on reduced advantage-estimation variance.
OpenAI Releases Proximal Policy Optimization (PPO)
OpenAI introduced Proximal Policy Optimization (PPO), a new class of reinforcement learning algorithms that match or exceed state-of-the-art performance while being simpler to implement and tune. PPO was adopted as OpenAI's default RL algorithm due to its balance of ease of use and strong performance. The release marked a significant methodological contribution to the RL field that would go on to underpin many subsequent AI training pipelines.
ESMFold2 matches AlphaFold 3 performance with open, LLM-inspired architecture for molecular structure prediction
Biohub and EvolutionaryScale released ESMFold2, a 6.2-billion-parameter open-weights model for predicting the 3D shapes of proteins, DNA, RNA, and small molecules by treating molecular sequences as language. Unlike AlphaFold 3, ESMFold2 can operate without multiple sequence alignments (MSAs) by using a transformer-based embedding model (ESMC) trained on 2.8 billion sequences, outperforming Chai-1 in MSA-free settings and matching AlphaFold 3 when MSAs are provided. The model weights are freely available on HuggingFace and via API through Biohub, making frontier-level structural biology accessible without proprietary infrastructure. The release is significant for drug discovery involving novel or synthetic molecules where MSA databases may be sparse.
EvoStruct: Bridging Evolutionary and Structural Priors for Antibody CDR Design via Protein Language Model Adaptation
EvoStruct addresses vocabulary collapse in GNN-based antibody CDR design by combining a frozen protein language model with an E(3)-equivariant GNN through a cross-attention adapter. The method introduces progressive PLM unfreezing and R-Drop consistency regularization to recover functionally important amino acid diversity. On CHIMERA-Bench, EvoStruct improves sequence recovery by 16%, reduces perplexity by 43%, and achieves 2.3x greater amino acid diversity compared to the best GNN baselines.
ACPO: Adaptive Clip Policy Optimization improves RLVR training for LLM reasoning
A new arXiv preprint provides theoretical analysis of Reinforcement Learning from Verifiable Rewards (RLVR) updates, identifying off-policy degree and gradient expectation as key factors governing update dynamics. The authors show that differences in gradient steps per rollout substantially affect importance sampling ratio distributions and which tokens dominate updates. Based on this analysis, they propose Adaptive Clip Policy Optimization (ACPO), which adjusts clipping boundaries per token group by empirical variance of importance sampling ratios, outperforming DAPO and CISPO baselines on 3B and 7B models across math, tabular QA, and logic benchmarks.