Revisiting Spurious Correlation in Domain Generalization

TL;DR

This work reframes domain generalization through a representation-learning Structural Causal Model (SCM) to analyze spurious correlations that cause OOD failure. It distinguishes fork-specific and collider-specific spurious correlations and argues that collider-specific spurious paths are the correct OOD-oriented mechanism, guiding debiasing via backdoor adjustment. The authors introduce a Propensity Score Weighted (PSW) estimator, augmented with FFT-based feature separation and a pairing scheme to robustly estimate propensity scores and reweight observations, enabling plug-in integration with existing OOD methods. Empirical results across six benchmarks show consistent generalization gains and SOTA performance on several datasets, demonstrating the practical utility and versatility of the proposed PSIW framework for robust OOD generalization.

Abstract

Without loss of generality, existing machine learning techniques may learn spurious correlation dependent on the domain, which exacerbates the generalization of models in out-of-distribution (OOD) scenarios. To address this issue, recent works build a structural causal model (SCM) to describe the causality within data generation process, thereby motivating methods to avoid the learning of spurious correlation by models. However, from the machine learning viewpoint, such a theoretical analysis omits the nuanced difference between the data generation process and representation learning process, resulting in that the causal analysis based on the former cannot well adapt to the latter. To this end, we explore to build a SCM for representation learning process and further conduct a thorough analysis of the mechanisms underlying spurious correlation. We underscore that adjusting erroneous covariates introduces bias, thus necessitating the correct selection of spurious correlation mechanisms based on practical application scenarios. In this regard, we substantiate the correctness of the proposed SCM and further propose to control confounding bias in OOD generalization by introducing a propensity score weighted estimator, which can be integrated into any existing OOD method as a plug-and-play module. The empirical results comprehensively demonstrate the effectiveness of our method on synthetic and large-scale real OOD datasets.

Figures (8)

• Figure 1: The SCM graph of representation learning for OOD. In Figure (a), the dash between $S$ and $Y$ represents the complex spurious correlation, where the direction of causal arrows cannot be determined. Figure (b) illustrates fork-specific spurious correlation, replacing $S - Y$ with $S\leftarrow L\rightarrow Y$, which is widely accepted as the latent common cause proposition in causal inference.
• Figure 2: On ColoredMNIST dataset, when the distribution of the training and test sets is consistent (i.e., no bias), ERM achieves an accuracy rate of over 95%, even the labels are associated with colors to varying degrees. However, when the dataset is biased, associating labels with colors leads to a sharp decline in the accuracy of ERM.
• Figure 3: The SCM graph of collider-specific spurious correlation model, with grey nodes indicating conditioning on that variable. Figure (b) is the extracted confounding path that introduces bias in estimating $C \rightarrow Y$.
• Figure 4: The results of hyper-parameters.
• Figure 5: The results of case study on a batch of randomly selected samples.

Theorems & Definitions (12)

• Definition 1
• Proposition 1
• Definition 2
• Theorem 1
• Definition 3
• Definition 4
• Corollary 1
• Definition 5
• Definition 6
• Definition 7