On First-Order Meta-Learning Algorithms
OpenAI published research on first-order meta-learning algorithms, presenting simplified variants of MAML (Model-Agnostic Meta-Learning) that omit second-order derivatives while retaining competitive performance. The work demonstrates that first-order approximations are surprisingly effective for few-shot learning tasks. This contributed to the broader understanding of gradient-based meta-learning efficiency and scalability.
Related guides (2)
Related events (8)
Reptile: A Scalable Meta-Learning Algorithm from OpenAI
OpenAI introduced Reptile, a meta-learning algorithm that works by repeatedly sampling tasks, running stochastic gradient descent, and updating initial parameters toward the task-specific learned parameters. It is mathematically related to first-order MAML but requires only black-box access to standard optimizers like SGD or Adam. The algorithm is positioned as computationally efficient and comparably performant to MAML-based approaches.
Some considerations on learning to explore via meta-reinforcement learning
OpenAI published a research post examining exploration strategies learned through meta-reinforcement learning. The work investigates how agents can acquire exploration behaviors through meta-learning rather than having them hand-designed. This is an early OpenAI contribution to the intersection of meta-learning and RL, predating the current frontier model era.
Evolved Policy Gradients: OpenAI Meta-Learning via Loss Function Evolution
OpenAI released Evolved Policy Gradients (EPG), a meta-learning method that evolves the loss function used to train reinforcement learning agents rather than hand-designing it. The approach enables faster adaptation to novel tasks, with agents demonstrating generalization to test-time scenarios outside their training distribution, such as navigating to objects placed in new locations. EPG represents an experimental direction in automated algorithm discovery for RL.
One-shot imitation learning
OpenAI published research on one-shot imitation learning, a technique enabling agents to learn new tasks from a single demonstration. The approach allows a policy network to observe a demonstration and immediately generalize to new instances of the same task without additional training. This was an early contribution to the field of meta-learning and few-shot generalization in robotics and sequential decision-making.
RL²: Fast Reinforcement Learning via Slow Reinforcement Learning
OpenAI published RL², a meta-reinforcement learning approach in which a slow outer RL process trains a recurrent neural network whose hidden state encodes a fast inner learning algorithm. The method allows agents to rapidly adapt to new tasks within a single episode by leveraging experience accumulated across many training tasks. This work is an early foundational contribution to meta-learning for RL, predating the modern agent and LLM era but relevant to understanding the intellectual lineage of in-context and few-shot learning in AI systems.
MAST: Mechanism-guided selective unlearning for RLVR-trained reasoning models
Researchers introduce MAST (Mechanism-Aligned Selective Targeting), a method for selectively unlearning capabilities induced by reinforcement learning from verifiable rewards (RLVR) in language models while minimizing collateral damage to retained knowledge. The approach ranks attention-projection tensors by off-principal energy and gradient coupling to identify a targeted subset for update, rather than applying full-parameter gradient ascent. Evaluated on Qwen2.5-Math-1.5B and Qwen3-1.7B-Base, MAST achieves statistically significant forgetting on target MATH problems while preserving GSM8K performance, whereas full-parameter unlearning collapses retained capabilities. The method generalizes across seeds and unlearning objectives (NPO/SimNPO).
Learning from Human Preferences: OpenAI and DeepMind Collaborate on Reward Learning from Comparisons
OpenAI, in collaboration with DeepMind's safety team, published a method for learning reward functions directly from human preference comparisons between pairs of agent behaviors, eliminating the need to hand-code goal functions. The algorithm infers human intent by asking evaluators which of two proposed behaviors is preferable, addressing risks from misspecified reward functions. This work is an early foundational contribution to what would become reinforcement learning from human feedback (RLHF). It targets both safety and alignment concerns around reward hacking and proxy gaming.
Meta Introduces Muse Spark: First Closed-Weights Model from Superintelligence Labs
Meta released Muse Spark, its first AI model in roughly a year and the debut product of its Superintelligence Labs, marking a significant departure from its open-weights Llama strategy. The natively multimodal reasoning model supports tool use and multi-agent orchestration, achieves fourth place on the Artificial Analysis Intelligence Index, and claims notable token efficiency—matching Llama 4 Maverick with over 10x less training compute. Meta withheld parameter count, architecture, and training details, positioning Muse Spark as a closed commercial product competing with OpenAI, Google, and Anthropic. The release introduces 'thought compression' via RL and a parallel multi-agent 'contemplating' mode, while showing gaps in coding and agentic benchmarks.