Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reducing Spurious Correlation for Federated Domain Generalization

Shuran Ma, Weiying Xie, Daixun Li, Haowei Li, Yunsong Li

TL;DR

FedCD tackles federated domain generalization by jointly mitigating spurious correlations and promoting invariant cross-domain predictions. It introduces a self-supervised Spurious Correlation Intervener (SCI) that locally weakens spurious features by masking inputs and aligns client gradients via a server-side gradient g, with a loss term L_SCI = $||\nabla_e - \nabla_g||^2$. Complementing SCI, the Risk Extrapolation Aggregation (REA) selects aggregation coefficients to minimize risk variance across clients through a convex optimization, yielding coefficients c_e = $\frac{\exp(\eta w_e + p_e)}{\sum_k \exp(\eta w_k + p_k)}$ that blend ERM with federation dynamics. Empirical results on style generalization and autonomous-driving datasets show consistent improvements over strong baselines, with larger gains under greater domain shifts, and analyses reveal SCI induces sparser, more robust feature representations. The approach preserves privacy by sharing only gradients and requires no additional data exchange, offering a scalable path toward robust cross-domain ML in privacy-sensitive, decentralized settings.

Abstract

The rapid development of multimedia has provided a large amount of data with different distributions for visual tasks, forming different domains. Federated Learning (FL) can efficiently use this diverse data distributed on different client media in a decentralized manner through model sharing. However, in open-world scenarios, there is a challenge: global models may struggle to predict well on entirely new domain data captured by certain media, which were not encountered during training. Existing methods still rely on strong statistical correlations between samples and labels to address this issue, which can be misleading, as some features may establish spurious short-cut correlations with the predictions. To comprehensively address this challenge, we introduce FedCD (Cross-Domain Invariant Federated Learning), an overall optimization framework at both the local and global levels. We introduce the Spurious Correlation Intervener (SCI), which employs invariance theory to locally generate interventers for features in a self-supervised manner to reduce the model's susceptibility to spurious correlated features. Our approach requires no sharing of data or features, only the gradients related to the model. Additionally, we develop the simple yet effective Risk Extrapolation Aggregation strategy (REA), determining aggregation coefficients through mathematical optimization to facilitate global causal invariant predictions. Extensive experiments and ablation studies highlight the effectiveness of our approach. In both classification and object detection generalization tasks, our method outperforms the baselines by an average of at least 1.45% in Acc, 4.8% and 1.27% in mAP50.

Reducing Spurious Correlation for Federated Domain Generalization

TL;DR

FedCD tackles federated domain generalization by jointly mitigating spurious correlations and promoting invariant cross-domain predictions. It introduces a self-supervised Spurious Correlation Intervener (SCI) that locally weakens spurious features by masking inputs and aligns client gradients via a server-side gradient g, with a loss term L_SCI = . Complementing SCI, the Risk Extrapolation Aggregation (REA) selects aggregation coefficients to minimize risk variance across clients through a convex optimization, yielding coefficients c_e = that blend ERM with federation dynamics. Empirical results on style generalization and autonomous-driving datasets show consistent improvements over strong baselines, with larger gains under greater domain shifts, and analyses reveal SCI induces sparser, more robust feature representations. The approach preserves privacy by sharing only gradients and requires no additional data exchange, offering a scalable path toward robust cross-domain ML in privacy-sensitive, decentralized settings.

Abstract

The rapid development of multimedia has provided a large amount of data with different distributions for visual tasks, forming different domains. Federated Learning (FL) can efficiently use this diverse data distributed on different client media in a decentralized manner through model sharing. However, in open-world scenarios, there is a challenge: global models may struggle to predict well on entirely new domain data captured by certain media, which were not encountered during training. Existing methods still rely on strong statistical correlations between samples and labels to address this issue, which can be misleading, as some features may establish spurious short-cut correlations with the predictions. To comprehensively address this challenge, we introduce FedCD (Cross-Domain Invariant Federated Learning), an overall optimization framework at both the local and global levels. We introduce the Spurious Correlation Intervener (SCI), which employs invariance theory to locally generate interventers for features in a self-supervised manner to reduce the model's susceptibility to spurious correlated features. Our approach requires no sharing of data or features, only the gradients related to the model. Additionally, we develop the simple yet effective Risk Extrapolation Aggregation strategy (REA), determining aggregation coefficients through mathematical optimization to facilitate global causal invariant predictions. Extensive experiments and ablation studies highlight the effectiveness of our approach. In both classification and object detection generalization tasks, our method outperforms the baselines by an average of at least 1.45% in Acc, 4.8% and 1.27% in mAP50.
Paper Structure (16 sections, 23 equations, 4 figures, 3 tables)

This paper contains 16 sections, 23 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. RELATED WORK
  3. Federated Learning
  4. Domain Generalization
  5. Federated Domain Generalization
  6. PROPOSED METHOD
  7. Problem Setting
  8. Invariant Optimization Theory
  9. SCI Implement
  10. REA Strategy
  11. EXPERIMENTS
  12. Experimental Setup
  13. Experimental Results
  14. Analyses on $SCI$
  15. DISCUSSION
  16. ...and 1 more sections

Figures (4)

  • Figure 1: The "Cow-On-The-Beach" problem in $FL$. Local models on the client side strongly correlate cows with green grassland and camels with beige deserts within the samples. Thus, replacing the background of the cow with a beige beach may lead to incorrect prediction on global model.
  • Figure 2: A brief overview of $FedCD$, consisting of two components $SCI$ and $REA$. In $SCI$, each client $e$ compute a gradient alignment penalty using Eq. \ref{['img loss']} and upload the local $SCI$ gradient tensors $\nabla_e$ in each communication round. They are then aggregated on the server to obtain $\nabla_g$, which is returned to the clients for the next round. $REA$ determines aggregation coefficients through mathematical optimization based on Empirical Risk Minimization ($ERM$). Instead of introducing new neural network solutions for optimization, $REA$ employs sequential least squares.
  • Figure 3: Comparing the average L1 norm of the masks generated by $SCI$ under different hyperparameters $\lambda = 0.7/0.9$ and different "leave-one-domain-out" settings.
  • Figure 4: The comparison between the original features and masked features of samples from feature-rich domain Art painting (A) and feature-poor domain Sketch (S), which reflects two phenomena: a decrease in overall and a decrease in specificity.