coral-c886ae17·2 events·first seen Aliases: CORAL
A new arXiv preprint introduces ARROW (Adaptive vaRiance Reduction via Online reWeighting), the first online stochastic variance reduction algorithm for the maximum mean discrepancy (MMD) and CORAL domain adaptation loss functions. Prior SVR algorithms for these losses were offline-only and incompatible with streaming, distributed, or incremental learning settings. ARROW maintains moving average references of alignment statistics and adaptively reweights incoming minibatches, with a relaxed reweighting scheme to keep the optimization tractable. Experiments show competitive performance with offline algorithms on runtime, variance reduction, and target domain accuracy.
This paper proposes the 'matching principle': a unified geometric framework arguing that robustness methods (CORAL, IRM, adversarial training, augmentation, metric learning, Jacobian penalties, alignment constraints) are all estimators of the same object—the covariance of label-preserving deployment nuisance—and that regularizing the encoder Jacobian along this covariance's range is the core statistical problem. The authors prove closed-form optimality results in a linear-Gaussian model, introduce the Trajectory Deviation Index (TDI) as a label-free embedding sensitivity probe, and validate predictions across 13 pre-registered experimental blocks including Qwen2.5-7B. At 7B scale, matched style-PMH improves selective honesty while standard DPO degrades Style TDI, connecting the theory to alignment safety.