technique

Embedded Attack

techniqueactiveprovisionalembedded-attack-9f1a3915·1 events·first seen 16h ago

Aliases: Embedded Attack

Co-occurring entities

Direct Preference Optimization (DPO)Dual-Reference SFT

More like this (12)

Distributed Agent Attack skill-based attacks backdoor attack MITRE ATT&CK tree-of-attacks HackAgent Hacktivate AI distillation attacks black-box adversarial attacks Chinese state-sponsored threat actor Policy Vulnerability Testing Membership Inference Attack

Recent events (1)

6arXiv · cs.AI·16h ago·source ↗

DR-SFT: Defending against harmful supervision hidden in benign fine-tuning samples

A new arXiv paper introduces 'Embedded Attack', an adversarial technique that hides harmful QA supervision inside ostensibly benign training samples, bypassing existing guardrails that operate at the example level. The authors then propose Dual-Reference SFT (DR-SFT), which adapts DPO-style contrastive objectives to supervised fine-tuning via token-level regularization to mitigate this class of attack. The work highlights a gap in current fine-tuning safety defenses and offers a concrete mitigation method.

AI Safety Research Alignment and RLHF Embedded Attack Direct Preference Optimization (DPO)Dual-Reference SFT