ARROW

ARROW: First online stochastic variance reduction algorithm for MMD and CORAL domain adaptation losses

A new arXiv preprint introduces ARROW (Adaptive vaRiance Reduction via Online reWeighting), the first online stochastic variance reduction algorithm for the maximum mean discrepancy (MMD) and CORAL domain adaptation loss functions. Prior SVR algorithms for these losses were offline-only and incompatible with streaming, distributed, or incremental learning settings. ARROW maintains moving average references of alignment statistics and adaptively reweights incoming minibatches, with a relaxed reweighting scheme to keep the optimization tractable. Experiments show competitive performance with offline algorithms on runtime, variance reduction, and target domain accuracy.