5arXiv cs.CL (Computation and Language)·
Future-Feedback Prediction enables verifiable self-evolution of dialogue skills in frozen LLM agents
Researchers propose a method called future-feedback skill evolution that reframes open-ended dialogue skill improvement as a prediction task: rather than evaluating counterfactual responses directly, the system predicts whether an observed response will lead to positive or negative subsequent user signals, making the target verifiable on fixed logged data. The approach achieves over 75% prediction accuracy on a proprietary sales-assistant dataset and enables offline textual skill optimization without requiring live traffic exposure. The core contribution is converting the inherently moving target of conversational feedback into a stable offline learning signal, addressing a key gap in applying self-evolution techniques beyond math and code domains.
Related guides (2)
Related events (8)
Skill Self-Play: Co-evolving LLM capabilities via structured self-play with dynamic skill routing
Researchers introduce Skill Self-Play (Skill-SP), a reinforcement learning framework that addresses the diversity-vs-verifiability dilemma in LLM self-evolution by using agent skills as a middle ground. The system comprises a proposer, solver, and dynamic skill controller that co-evolve in a continuous loop: the proposer generates tasks conditioned on sampled skills, the solver explores solutions, and the skill controller updates an expanding skill library based on execution feedback. Evaluations on tool-use and reasoning benchmarks show consistent performance gains on capable backbones and recovery for initially misaligned models. Code is released under the Qwen-Applications GitHub organization, suggesting Alibaba/Qwen team involvement.
Microsoft releases SkillOpt: text-space optimizer for training reusable NL skills in frozen LLM agents
Microsoft has open-sourced SkillOpt, a Python framework that optimizes natural-language skills for frozen LLM agents without modifying model weights. The system uses trajectory-driven edits and validation-gated updates to produce deployable best_skill.md artifacts. The approach is relevant to agent systems that need reusable, inspectable skill libraries without fine-tuning. The repository has accumulated over 11,600 stars with strong recent momentum (+261 today).
MetaEvolve framework trains LLMs to self-evolve via meta-skill reinforcement learning
Researchers introduce MetaEvolve, a framework that cultivates 'meta-skills' such as self-reflection and multi-round refinement in LLMs through a data synthesis pipeline and evolution-aware reinforcement learning. Grounded in coding tasks where program execution provides continuous reward signals, the system trains on evolution trajectories containing programs, fitness scores, and attempt histories. MetaEvolve outperforms the strongest baseline by 10.01% on in-distribution coding benchmarks and 24.12% on out-of-distribution tasks, with a 46.9% relative improvement on open-ended algorithm optimization problems outside the training domain. The work is motivated by AlphaEvolve-style test-time scaling and aims to produce generalizable meta-skills transferable to domains with sparse training signals.
RL-based alignment improves interactivity in full-duplex spoken dialogue models
Researchers propose a post-training alignment method using reinforcement learning to improve interactivity in full-duplex spoken dialogue models, which can listen and speak simultaneously. The method addresses four canonical axes of interactivity—pause handling, turn-taking, backchanneling, and user interruption—each with axis-specific reward functions, plus an LLM-based reward to prevent semantic degradation. The approach is applied to two open-source models, Moshi and PersonaPlex, showing consistent improvements in both offline and real-time multi-turn evaluation.
RePro: Retrospective Progress-Aware Self-Refinement for LLM Agent Training
Researchers introduce RePro (Retrospective Progress-Aware Training), a framework addressing the gap between step-wise RL optimization and metacognitive task-progress awareness in LLM agents. The approach uses a forward-then-reflect rollout paradigm where agents execute actions online and then retrospectively assess step-wise progress given the completed trajectory and known outcome. Evaluated on WebShop, ALFWorld, and Sokoban, RePro achieves up to 12% absolute success rate gains over baseline Qwen-family models without requiring continuous external supervision.
WorldEvolver: Self-Evolving World Models for LLM Agent Planning via Test-Time Memory Revision
Researchers introduce WorldEvolver, a framework that equips LLM agents with self-improving world models that revise their context at deployment time without updating model parameters. The system combines episodic memory (retrieval-based simulation), semantic memory (heuristic rule extraction from prediction errors), and selective foresight (confidence-based filtering). Evaluated on ALFWorld and ScienceWorld benchmarks, WorldEvolver achieves state-of-the-art world model prediction accuracy and improved downstream agent success rates across three backbone models. The work addresses a key challenge in long-horizon agent planning: unreliable foresight that can degrade rather than improve decision-making.
Double Ratchet: Co-evolving evaluation metrics and skills for self-improving LLM agents
A new arXiv preprint introduces Double Ratchet, a system that co-evolves both evaluation metrics and agent skills in settings where no reliable automatic verifier exists. The metric loop uses evolutionary search over small drawback detectors anchored to a small reference set, while the skill loop uses a lifecycle-managed approach; together they retain 88–110% of the performance lift achievable with ground-truth metrics across code generation (MBPP+), text-to-SQL (Spider 2.0-Snow), and report generation tasks. The paper also addresses safety, showing that anchor discipline and outer audits can catch and repair cases where evolved skills game the rubric. This work directly addresses a core bottleneck in self-improving agent systems: the chicken-and-egg problem of needing a reliable evaluator to improve.
Future Confidence Distillation enables pre-solution confidence estimation in LLMs
Researchers introduce 'future confidence distillation,' a method that trains predictors on pre-solution hidden representations using post-solution correctness probes as teacher signals, enabling reliable confidence estimation before answer generation completes. The paper compares pre-solution Feeling-of-Knowing (FOK) and post-solution Judgement-of-Learning (JOL) confidence estimates across frontier and open-source LLMs, finding post-solution estimates are better calibrated and that linear probes recover richer confidence information than models verbalize. Distilled predictors recover much of the post-solution calibration improvement while remaining sample-efficient and domain-transferable. This has practical implications for confidence-aware systems involving retrieval, tool use, and adaptive computation.