Entity · paper

Enough is as good as a feast: A Comprehensive Analysis of How Reinforcement Learning Mitigates Task Conflicts in LLMs

paperactiveprovisional

enough-is-as-good-as-a-feast-a-comprehensive-analysis-of-how-reinforcement-learning-mitigates-task-conflicts-in-llms-df546605

·1 events·first seen 5d ago

Aliases: Enough is as good as a feast: A Comprehensive Analysis of How Reinforcement Learning Mitigates Task Conflicts in LLMs

More like this (12)

Improving LLM-Generated Process Model Quality Through Reinforcement Learning: The Role of Reward Function Design Reinforcement Learning with Metacognitive Feedback Elicits Faithful Uncertainty Expression in LLMs ExpRL: Exploratory RL for LLM Mid-Training When Model Merging Rivals Joint Multi-Task Reinforcement Learning: A Task-Vector Geometry Analysis Teaching LLMs to Self-Evolve: Cultivating Core Meta-Skills with Reinforcement Learning When Good Verifiers Go Bad: Self-Improving VLMs Can Regress on New Tasks Janus: A Benchmark for Goal-Conditioned Information Distortion in LLMs Resist and Update: Counterfactual Report Coordinates for Incentive-Compatible LLMs Can We Break LLMs Out of Self-Loops? Fine-Grained Reasoning Control with Activation Steering Hierarchical Advantage Weighting for Online RL Fine-Tuning of VLAs from Sparse Episode Outcomes QVal: Cheaply Evaluating Dense Supervision Signals for Long-Horizon LLM Agents QVal: Cheaply Evaluating Dense Supervision Signals for Long-Horizon LLM Agents

Recent events (1)

5arXiv · cs.CL·5d ago·source ↗

RL-trained LLMs show reduced task conflicts and better model merging than SFT-trained models

A new arXiv preprint systematically compares model merging behavior between RL-trained and SFT-trained LLMs across five tasks, finding that RL significantly reduces task conflicts and performance degradation post-merge. The authors identify three mechanisms: smaller gradient updates from on-policy training, an optimization objective that limits conflict parameter updates at convergence, and joint positive/negative example optimization that steers models toward unbiased task-specific parameter subspaces. The findings suggest RL training paradigms are inherently better suited for model merging workflows, with implications for how practitioners choose training methods when multi-task consolidation is a goal.

Evaluation and Benchmarking Alignment and RLHF Enough is as good as a feast: A Comprehensive Analysis of How Reinforcement Learning Mitigates Task Conflicts in LLMs