6arXiv cs.AI (Artificial Intelligence)·
Model merging matches joint multi-task RL training on AppWorld benchmark, explained by near-orthogonal task vectors
A new arXiv paper provides the first direct comparison of model merging versus joint multi-task reinforcement learning training, using Qwen3-8B specialists trained on the AppWorld agent benchmark with the LOOP algorithm. Merging methods (TIES, RAM+) statistically match jointly trained models on task-goal completion. The authors explain this via task vector geometry: specialist task vectors are near-orthogonal (cosine similarity 0.06–0.10) despite ~65% parameter support overlap, causing sign- and support-based merging methods to collapse to near-uniform averaging.
Related guides (3)
Related events (8)
DeltaMerge-LowRes: Composing Language and Task Weight Deltas for Low-Resource NLP Adaptation
A new arXiv preprint introduces DeltaMerge-LowRes, a method for adapting multilingual encoders to new languages and tasks in low-resource settings by training language and task deltas separately and composing them in weight space at inference. The paper proposes four composition rules including a novel cross-axis TIES variant that adapts the TIES-Merging algorithm to language/task axes rather than task/task axes. Evaluated across four task families and four African languages (158 cells), cross-axis TIES improves summarization by +4 to +7 chrF and QA F1 by +2.32, while sparsity-aware merging reduces calibration error by 36%. The work advances model merging techniques for low-resource multilingual NLP.
Steerable Model Merging (ST-Merge) improves multilingual reasoning via adaptive gated cross-attention
Researchers propose ST-Merge, a framework for adaptively merging a multilingual model and a reasoning model using a gated cross-attention mechanism that weights each source model's contribution based on input characteristics. The approach addresses the limitation of static one-size-fits-all merging strategies that fail to resolve conflicts between source models. Experiments across 21 languages on four multilingual reasoning benchmarks show consistent improvements over strong baselines.
Agon: Competitive cross-model RL uses rival models as implicit reasoning graders
Agon is a new reinforcement learning framework where two competing models grade each other implicitly by attempting the same problems in alternating roles — one drafts a solution, the other reads it while solving, and each is rewarded for out-solving the rival. This sidesteps the need for process labels or a reward model, and because both models are jointly optimized, each faces a progressively stronger opponent. On the hard split of DeepMath with Qwen3, Agon doubles GRPO's pass@1, roughly eight times the gain of an untrained Mixture-of-Agents baseline, with results replicating on competitive programming and across model families.
Muon optimizer shows large gains over AdamW in sparse-reward agentic RL on ALFWorld
A new arXiv preprint investigates the Muon optimizer for reinforcement learning post-training of language model agents, comparing it to AdamW on the ALFWorld benchmark using Qwen2.5-0.5B-Instruct. Under Group-in-Group Policy Optimization (GiGPO), applying Muon to hidden weight matrices raises validation success from 0.290 to 0.546 (+88%), with further gains at lower learning rates reaching 0.901 success. The results are exploratory (single-seed, single-task) but suggest that optimizer choice, advantage estimator, and learning rate interact significantly in agentic RL settings.
Mach-Mind-4-Flash: 35B MoE agentic model matching 100B-class performance via post-training optimization
Mach-Mind-4-Flash is a 35B-parameter Mixture-of-Experts model with only 3B activated parameters that achieves performance comparable to 100B-class models through post-training techniques alone. The pipeline combines a unified RL/OPD training infrastructure with multi-teacher scheduling, parallel domain-specific RL experts fused via Multi-Teacher On-Policy Distillation (MOPD), and Hybrid Median-length Policy Optimization (HMPO) which compresses reasoning chains 19-46% with minimal accuracy loss. Benchmark results include 92.70 on AIME'26, 82.82 on IFBench, and 75.80 on BFCL-v4, claiming to lead or match models 10-30x its activated size at a fraction of inference cost. The work is notable for demonstrating that post-training optimization can close large gaps in activated parameter count for agentic tasks.
Task-specific two-agent architecture wins QANTA 2026 multimodal QA challenge
Researchers present the top-scoring submission to the QANTA 2026 shared challenge at ICML 2026's EMM-QA Workshop, achieving an overall leaderboard score of 0.402 on multimodal quizbowl tasks. The system uses a two-agent architecture: a GPT-4.1-mini-based Tossup agent with confidence calibration and a GPT-4.1-based Bonus agent with structured relational and multimodal reasoning. Notably, the approach avoids retrieval pipelines and model ensembles, relying instead on lightweight task-specific reasoning policies under efficiency constraints. Results suggest that targeted reasoning strategies can be competitive on resource-constrained multimodal QA benchmarks.
MOPD: Multi-Teacher On-Policy Distillation for integrating multiple RL-trained capabilities in LLMs
Researchers propose Multi-teacher On-Policy Distillation (MOPD), a post-training paradigm that first trains domain-specialized RL teacher models, then distills them into a student model using on-policy rollouts to eliminate exposure bias. Evaluated on Qwen3-30B-A3B, MOPD outperforms Mix-RL, Cascade RL, Off-Policy Finetune, and Param-Merge baselines while preserving nearly all per-domain capability. The method has been deployed in production for MiMo-V2-Flash, an industrial-scale frontier model, validating its practical applicability. The approach also enables parallel, decoupled development of domain teachers, reducing cross-domain interference in multi-capability post-training.
Model merging as training-free strategy for conversational information retrieval
A new arXiv preprint proposes using model merging (Model Soup and Slerp) as a training-free alternative to fine-tuning for conversational information retrieval, avoiding catastrophic forgetting of ad-hoc retrieval capabilities. Experiments on standard ad-hoc and conversational retrieval benchmarks show up to 15% higher NDCG@3 under zero-shot conditions. The approach enables a single retrieval model to operate across both ad-hoc and conversational settings without retraining.