4arXiv cs.AI (Artificial Intelligence)·
Positive-Direction Matching addresses branch asymmetry in classifier-free guidance for diffusion distillation
A new arXiv paper identifies a failure mode called Negative Branch Asymmetry (NBA) in on-policy diffusion distillation (OPD) when combined with classifier-free guidance (CFG), showing that naive guided velocity matching can produce antagonistic branch-error dynamics. The authors introduce Positive-Direction Matching (PDM), a branch-aware training objective that separately constrains the positive prediction and the CFG conditional direction. The method is validated on dense-to-sparse video control, where it improves robustness to inference guidance scales compared to naive guided matching.
Related guides (1)
Related events (8)
The Matching Principle: A Geometric Theory Unifying Robustness, Domain Adaptation, and Alignment via Nuisance Covariance
This paper proposes the 'matching principle': a unified geometric framework arguing that robustness methods (CORAL, IRM, adversarial training, augmentation, metric learning, Jacobian penalties, alignment constraints) are all estimators of the same object—the covariance of label-preserving deployment nuisance—and that regularizing the encoder Jacobian along this covariance's range is the core statistical problem. The authors prove closed-form optimality results in a linear-Gaussian model, introduce the Trajectory Deviation Index (TDI) as a label-free embedding sensitivity probe, and validate predictions across 13 pre-registered experimental blocks including Qwen2.5-7B. At 7B scale, matched style-PMH improves selective honesty while standard DPO degrades Style TDI, connecting the theory to alignment safety.
Kolmogorov Regression lifts diffusion policies to Cameron-Martin space for robust long-horizon control
Researchers introduce a backward Kolmogorov equation framework that reformulates diffusion policy training as a deterministic boundary-value PDE problem in Cameron-Martin space, replacing stochastic score matching. The approach uses a precision-weighted Cameron-Martin loss and a Kolmogorov residual as an inference-time failure detector, yielding convergence guarantees tied to kernel effective rank rather than action dimension. Validation on the PushT manipulation benchmark shows 17% improvement in episode reward and 67.6% reduction in inter-step drift; a 6-station manufacturing scheduling task shows 28.4% lower RMSE than LSTM baselines and 96% reduction in deadlock events via Hamilton-Jacobi reachability certification.
Parallel Decoding Distillation (PDD) accelerates video and image diffusion inference with improved diversity
Researchers introduce Parallel Decoding Distillation (PDD), a trajectory-based distillation method that accelerates diffusion and flow matching models by predicting multiple denoising steps per network evaluation, achieving state-of-the-art performance at 4-8 NFE. Unlike prior methods relying on variational score distillation and adversarial losses, PDD avoids mode collapse and improves generated video diversity. The method is validated on LTX-2.3 Text-to-Video/Audio, Wan 14B Text-to-Video, and Qwen-Image Text-to-Image models.
Mask-Aware Policy Gradients improve RL training for Masked Diffusion Language Models
A new arXiv preprint introduces a two-stage action MDP formalization for applying reinforcement learning to Masked Diffusion Language Models (MDLMs), decomposing the policy gradient into a token prediction term and a masking order term. Prior approaches ignored the position-unmasking decision, leading to intractable log-likelihood estimates; the proposed method optimizes both terms jointly. The approach achieves 87.1% on GSM8K and 53.4% on MBPP, claiming state-of-the-art results for MDLM-based reasoning and coding.
d-OPSD: First on-policy self-distillation framework tailored for diffusion LLMs
Researchers introduce d-OPSD, the first on-policy self-distillation (OPSD) framework designed specifically for diffusion large language models (dLLMs). The method addresses a fundamental mismatch between existing autoregressive OPSD approaches and dLLMs' arbitrary-order generation by using suffix conditioning on self-generated answers and step-level rather than token-level divergence supervision. Across four reasoning benchmarks, d-OPSD outperforms RLVR and SFT baselines while requiring only ~10% of the optimization steps of RLVR, suggesting strong sample efficiency gains for dLLM post-training.
DRPO: Smooth divergence regularization replaces hard masking in LLM RL training
A new arXiv preprint proposes Divergence Regularized Policy Optimization (DRPO), a method that replaces the hard trust-region mask used in DPPO with a smooth advantage-weighted quadratic regularizer on policy shift. The approach addresses a known weakness in PPO and GRPO where importance ratios poorly proxy distributional shift in long-tailed vocabularies, and in DPPO where gradient signals are discarded rather than corrected at trust-region boundaries. Experiments across model scales, architectures, and precision settings show improved stability and efficiency in LLM RL post-training.
ROPD: Routing-based On-Policy Distillation for template-robust LLM safety realignment
A new arXiv preprint introduces Routing-based On-Policy Distillation (ROPD), a safety realignment framework designed to counter malicious fine-tuning attacks that embed harmful behaviors into downstream model corpora. ROPD models the divergence between aligned and compromised output distributions rather than fitting specific prompt templates, addressing three failure modes of existing defenses: catastrophic forgetting, template-mismatch collapse, and re-jailbreaking via system prompt switches. Experiments across four baselines, three datasets, and three base models show ROPD substantially reduces template-mismatch degradation while preserving downstream task performance.
Activation-space directions for detecting and mitigating emergent misalignment across LLM families
Researchers fine-tuned four small instruction-tuned model families (Qwen2.5-1.5B, Gemma-2-2B, Llama-3.2-1B, Ministral-3B) on insecure code to induce emergent misalignment, then investigated whether a shared activation-space direction could detect and correct it. A difference-in-means direction achieves 99.6% separation of aligned vs. misaligned activations within each model, and causal steering by subtracting this direction reduces misaligned behavior by 21–51 points. Cross-architecture transfer via ridge regression yields large behavioral suppression but fails specificity controls, revealing a two-tier structure: within-model directions are causally specific and actionable, while cross-model directions are real but non-specific. The findings bound the utility of linear cross-architecture correction and recommend within-model probing for safety auditing.