Entity · paper

Leveraging Instruction Tuning and Merging for Reasoning Model Adaptation

paperactiveprovisionalleveraging-instruction-tuning-and-merging-for-reasoning-model-adaptation-28bf17e8·1 events·first seen 3d ago

Aliases: Leveraging Instruction Tuning and Merging for Reasoning Model Adaptation

More like this (12)

Multimodal Continual Instruction Tuning Learning to Reason by Analogy via Retrieval-Augmented Reinforcement Fine-Tuning Reasoning Language Models Predicting Future Behaviors in Reasoning Models Enables Better Steering Does Reasoning Preserve Alignment? On the Trustworthiness of Large Reasoning Models Improving Ad-hoc Search Effectiveness for Conversational Information Retrieval via Model Merging Token-Operations-Oriented Inference Optimization Techniques for Large Models When the Chain of Thought Knows Better: Failure Modes in Multi-Turn Reasoning Models Reasoning Enhancement Language Model Finetuning Visual Instruction Tuning Aligns Modalities through Abstraction Pessimism's Paradox: Conservative Offline Training Amplifies Reward Hacking During Online Adaptation in Reasoning Models

Recent events (1)

5arXiv · cs.CL·3d ago·source ↗

Instruction tuning plus model merging improves reasoning LMs in verifiable and unverifiable domains

A new arXiv preprint proposes a two-step method for adapting reasoning language models (RLMs) to domains lacking reliable verifiers: first apply standard supervised fine-tuning (without reasoning traces) on the target domain, then merge the resulting model with the original RLM to recover reasoning behavior. The technique is evaluated on both verifiable domains (coding) and hard-to-verify domains (text summarization), showing improvements while preserving general RLM capabilities. Notably, the authors report the full adaptation can be achieved for under $3, making it highly accessible.

Frontier Model Releases Alignment and RLHF Leveraging Instruction Tuning and Merging for Reasoning Model Adaptation