Deep CORAL: Correlation Alignment for Deep Domain Adaptation

TL;DR

The paper tackles unsupervised domain adaptation under target-domain unlabeled data by introducing Deep CORAL, which embeds a differentiable CORAL loss into deep networks to align second-order statistics of layer activations across domains. It advances CORAL to a nonlinear, end-to-end framework and optimizes a joint objective that balances discriminative power with domain alignment. Empirical results on the Office benchmark show state-of-the-art performance, validating the effectiveness and simplicity of covariance-based alignment in deep models. The approach provides a scalable, architecture-agnostic method for improving cross-domain generalization in practical settings.

Abstract

Deep neural networks are able to learn powerful representations from large quantities of labeled input data, however they cannot always generalize well across changes in input distributions. Domain adaptation algorithms have been proposed to compensate for the degradation in performance due to domain shift. In this paper, we address the case when the target domain is unlabeled, requiring unsupervised adaptation. CORAL is a "frustratingly easy" unsupervised domain adaptation method that aligns the second-order statistics of the source and target distributions with a linear transformation. Here, we extend CORAL to learn a nonlinear transformation that aligns correlations of layer activations in deep neural networks (Deep CORAL). Experiments on standard benchmark datasets show state-of-the-art performance.

Figures (2)

• Figure 1: Sample Deep CORAL architecture based on a CNN with a classifier layer. For generalization and simplicity, here we apply the CORAL loss to the $fc8$ layer of AlexNet alexnet. Integrating it to other layers or network architectures should be straightforward.
• Figure 2: Detailed analysis of shift A$\rightarrow$W for training w/ vs. w/o CORAL loss. (a): training and test accuracies for training w/ vs. w/o CORAL loss. We can see that adding CORAL loss helps achieve much better performance on the target domain while maintaining strong classification accuracy on the source domain. (b): classification loss and CORAL loss for training w/ CORAL loss. As the last fully connected layer is randomly initialized with $\mathcal{N}(0,0.005)$, CORAL loss is very small while classification loss is very large at the beginning. After training for a few hundred iterations, these two losses are about the same. (c): CORAL distance for training w/o CORAL loss (setting the weight to 0). The distance is getting much larger ($\geqslant100$ times larger compared to training w/ CORAL loss).