Dual-Reference SFT

DR-SFT: Defending against harmful supervision hidden in benign fine-tuning samples

A new arXiv paper introduces 'Embedded Attack', an adversarial technique that hides harmful QA supervision inside ostensibly benign training samples, bypassing existing guardrails that operate at the example level. The authors then propose Dual-Reference SFT (DR-SFT), which adapts DPO-style contrastive objectives to supervised fine-tuning via token-level regularization to mitigate this class of attack. The work highlights a gap in current fine-tuning safety defenses and offers a concrete mitigation method.