Cross-Domain Feature Augmentation for Domain Generalization

TL;DR

The paper tackles domain generalization by shifting augmentation from the input space to the feature space, introducing a semantic feature decomposition into four components $Z_{c,d}$, $Z_{c,\\neg d}$, $Z_{\\neg c,d}$, $Z_{\\neg c,\\neg d}$ and a cross-domain augmentation scheme named $\text{XDomainMix}$. It employs a two-phase training procedure with a warm-up on original data followed by augmentation-aware learning that optimizes predictions on both $Z$ and the augmented $\tilde{Z}$ via a combined loss $\mathcal{L}_{aug}$, while a domain classifier is trained only on original features. Empirical results across multiple domain-generalization benchmarks demonstrate state-of-the-art performance, improved invariance at both representation and prediction levels, and greater augmentation diversity as measured by MMD and covariance-based metrics. The work highlights the practical value of semantically structured feature augmentation for robust cross-domain generalization, while acknowledging limitations and offering avenues for theoretical grounding and integration with sharpness-aware optimization techniques.

Abstract

Domain generalization aims to develop models that are robust to distribution shifts. Existing methods focus on learning invariance across domains to enhance model robustness, and data augmentation has been widely used to learn invariant predictors, with most methods performing augmentation in the input space. However, augmentation in the input space has limited diversity whereas in the feature space is more versatile and has shown promising results. Nonetheless, feature semantics is seldom considered and existing feature augmentation methods suffer from a limited variety of augmented features. We decompose features into class-generic, class-specific, domain-generic, and domain-specific components. We propose a cross-domain feature augmentation method named XDomainMix that enables us to increase sample diversity while emphasizing the learning of invariant representations to achieve domain generalization. Experiments on widely used benchmark datasets demonstrate that our proposed method is able to achieve state-of-the-art performance. Quantitative analysis indicates that our feature augmentation approach facilitates the learning of effective models that are invariant across different domains.

Figures (7)

• Figure 1: Samples of images reconstructed from features produced by DSU li2022uncertainty and the proposed XDomainMix. The elephant reconstructed from XDomainMix's features shows a more complex background. The horse reconstructed from XDomainMix's features displays different characteristics such as a white belly. The top floor of the house generated by XDomainMix's features shows solid walls, instead of glass walls in the original. In contrast, images reconstructed from DSU's features have limited diversity and appear largely similar to the original images.
• Figure 2: Overview of XDomainMix. To perform augmentation, the feature of an input is decomposed into four components based on the semantics' correlation with class and domain. Afterward, features of other two samples from different domains, one from the same class and one from a different class are used to augment features by changing domain-specific feature components.
• Figure 3: Prediction accuracy after removing x% of features with the highest importance scores.
• Figure 4: Visualization of features from different classes/domains, indicated by the different colors.
• Figure 5: Prediction accuracy after removing x% of features with the highest importance scores on additional datasets.