Automated Background Swapping

AutoBackSwap: Automated background swapping to reduce spurious correlation reliance in image classifiers

Researchers introduce Automated Background Swapping (AutoBackSwap), a data augmentation method that uses a secondary network to disentangle foreground and background in images, inpaint backgrounds, and recombine them to reduce classifier reliance on spurious background correlations. The approach requires patch-wise labeling of only a few hundred samples to train the secondary network and scale augmentation to the full dataset. Notably, AutoBackSwap remains effective even when no training samples break the spurious correlation, outperforming prior methods across multiple image classification benchmarks.