4arXiv cs.LG (Machine Learning)·
PSDA: Variance-reduced domain adaptation via paired sampling for distribution-matching objectives
A new arXiv preprint introduces Paired Sampling for Domain Adaptation (PSDA), a stochastic variance reduction technique designed for distribution-matching losses like correlation alignment and maximum mean discrepancy that lack finite-sum structure. PSDA constructs quadruplets of observations within and across domains by solving linear assignment problems to minimize expected gradient variance. Experiments on three domain shift datasets show improved target domain accuracy over related methods.
Related guides (1)
Related events (8)
ARROW: First online stochastic variance reduction algorithm for MMD and CORAL domain adaptation losses
A new arXiv preprint introduces ARROW (Adaptive vaRiance Reduction via Online reWeighting), the first online stochastic variance reduction algorithm for the maximum mean discrepancy (MMD) and CORAL domain adaptation loss functions. Prior SVR algorithms for these losses were offline-only and incompatible with streaming, distributed, or incremental learning settings. ARROW maintains moving average references of alignment statistics and adaptively reweights incoming minibatches, with a relaxed reweighting scheme to keep the optimization tractable. Experiments show competitive performance with offline algorithms on runtime, variance reduction, and target domain accuracy.
The Matching Principle: A Geometric Theory Unifying Robustness, Domain Adaptation, and Alignment via Nuisance Covariance
This paper proposes the 'matching principle': a unified geometric framework arguing that robustness methods (CORAL, IRM, adversarial training, augmentation, metric learning, Jacobian penalties, alignment constraints) are all estimators of the same object—the covariance of label-preserving deployment nuisance—and that regularizing the encoder Jacobian along this covariance's range is the core statistical problem. The authors prove closed-form optimality results in a linear-Gaussian model, introduce the Trajectory Deviation Index (TDI) as a label-free embedding sensitivity probe, and validate predictions across 13 pre-registered experimental blocks including Qwen2.5-7B. At 7B scale, matched style-PMH improves selective honesty while standard DPO degrades Style TDI, connecting the theory to alignment safety.
DemoPSD: Disagreement-modulated policy self-distillation to fix privileged information leakage in LLM reasoning training
DemoPSD is a new training framework for LLMs that addresses two failure modes in on-policy self-distillation (OPSD): overfitting to in-domain patterns and privileged information leakage, where the student model learns answer-dependent shortcuts unavailable at test time. The method steers the student toward a reverse-KL barycenter target — a weighted geometric blend of teacher and student distributions — with token-level blending weights derived from the disagreement between the two distributions. Experiments on SciKnowEval across four scientific domains show DemoPSD outperforms GRPO and SDPO while maintaining higher training entropy and generalizing to out-of-distribution GPQA benchmarks.
DADiff: Diffusion-based framework for cross-domain policy adaptation in reinforcement learning
DADiff is a new diffusion-based framework for online dynamics adaptation in reinforcement learning, addressing the domain transfer problem by leveraging discrepancies between source and target domain generative trajectories during next-state prediction. The authors develop both reward modification and data selection variants, and provide theoretical bounds on policy performance differences in terms of generative trajectory deviation. Experiments across environments with various domain shifts show superior performance over existing approaches. Code is publicly released.
ADAS: Attention-Discounted Adaptive Sampler improves parallel decoding for masked diffusion language models
Researchers propose ADAS, a training-free reranking rule for masked diffusion language model decoding that addresses token interaction failures in parallel token commitment. The method greedily penalizes candidates that attend strongly to already-selected uncertain positions, using attention weights as soft marginal penalties rather than hard constraints. Evaluated on LLaDA-8B-Base and Dream-7B-Base across GSM8K, MATH500, HumanEval, and MBPP, ADAS improves low-NFE performance by 9–10 percentage points on average when plugged into existing samplers with only 3.1% runtime overhead.
Adaptive Data Scheduling (ADS) improves LLM reinforcement learning post-training by 5.2% over GRPO
Researchers propose Adaptive Data Scheduling (ADS), a dual-level framework that replaces uniform sampling in RL post-training with adaptive distribution over semantic clusters and policy-boundary sample selection. Evaluated across three LLMs and seven reasoning benchmarks, ADS improves average accuracy by 5.2% over GRPO and generalizes across RL objectives. The method addresses a structural limitation in standard RL post-training pipelines by accounting for semantic data structure and evolving policy capability during training.
DRPO: Smooth divergence regularization replaces hard masking in LLM RL training
A new arXiv preprint proposes Divergence Regularized Policy Optimization (DRPO), a method that replaces the hard trust-region mask used in DPPO with a smooth advantage-weighted quadratic regularizer on policy shift. The approach addresses a known weakness in PPO and GRPO where importance ratios poorly proxy distributional shift in long-tailed vocabularies, and in DPPO where gradient signals are discarded rather than corrected at trust-region boundaries. Experiments across model scales, architectures, and precision settings show improved stability and efficiency in LLM RL post-training.
RAPS-DA: Regime-aware peer specialization for robust RAG under knowledge conflicts
A new arXiv preprint introduces RAPS-DA, a training framework for making RAG systems more robust when retrieved context conflicts with a model's parametric knowledge. The approach divides conflicts into three reliability regimes (Grounding, Arbitration, Resistance) and trains separate peer specialist models per regime from a shared base, using reverse-KL supervision and a dual-layer token selector to filter uninformative training signals. Peer specialists exist only during training, so the deployed student model requires no additional components at inference time. Experiments across five conflict scenarios and two out-of-distribution benchmarks show RAPS-DA outperforms prompting, decoding, fine-tuning, RL, and single-teacher baselines.