Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Simple Data Augmentation for Feature Distribution Skewed Federated Learning

Yunlu Yan, Huazhu Fu, Yuexiang Li, Jinheng Xie, Jun Ma, Guang Yang, Lei Zhu

TL;DR

This work addresses feature distribution skew in federated learning by proposing FedRDN, a simple, plug-and-play input-level augmentation that injects federation-wide statistical information into local data during training. By randomly pairing local samples with statistics from other clients, FedRDN broadens the effective data distribution and enhances local feature generalization, reducing distribution bias without altering network architectures or incurring significant overhead. Extensive experiments across image classification and medical image segmentation demonstrate consistent improvements for a range of FL methods, with notably low additional communication costs and improved cross-site generalization. The approach has practical impact for deploying FL in heterogeneous environments where feature distributions vary across clients, providing a privacy-preserving, easily adoptable enhancement to existing pipelines.

Abstract

Federated Learning (FL) facilitates collaborative learning among multiple clients in a distributed manner and ensures the security of privacy. However, its performance inevitably degrades with non-Independent and Identically Distributed (non-IID) data. In this paper, we focus on the feature distribution skewed FL scenario, a common non-IID situation in real-world applications where data from different clients exhibit varying underlying distributions. This variation leads to feature shift, which is a key issue of this scenario. While previous works have made notable progress, few pay attention to the data itself, i.e., the root of this issue. The primary goal of this paper is to mitigate feature shift from the perspective of data. To this end, we propose a simple yet remarkably effective input-level data augmentation method, namely FedRDN, which randomly injects the statistical information of the local distribution from the entire federation into the client's data. This is beneficial to improve the generalization of local feature representations, thereby mitigating feature shift. Moreover, our FedRDN is a plug-and-play component, which can be seamlessly integrated into the data augmentation flow with only a few lines of code. Extensive experiments on several datasets show that the performance of various representative FL methods can be further improved by integrating our FedRDN, demonstrating its effectiveness, strong compatibility and generalizability. Code will be released.

A Simple Data Augmentation for Feature Distribution Skewed Federated Learning

TL;DR

This work addresses feature distribution skew in federated learning by proposing FedRDN, a simple, plug-and-play input-level augmentation that injects federation-wide statistical information into local data during training. By randomly pairing local samples with statistics from other clients, FedRDN broadens the effective data distribution and enhances local feature generalization, reducing distribution bias without altering network architectures or incurring significant overhead. Extensive experiments across image classification and medical image segmentation demonstrate consistent improvements for a range of FL methods, with notably low additional communication costs and improved cross-site generalization. The approach has practical impact for deploying FL in heterogeneous environments where feature distributions vary across clients, providing a privacy-preserving, easily adoptable enhancement to existing pipelines.

Abstract

Federated Learning (FL) facilitates collaborative learning among multiple clients in a distributed manner and ensures the security of privacy. However, its performance inevitably degrades with non-Independent and Identically Distributed (non-IID) data. In this paper, we focus on the feature distribution skewed FL scenario, a common non-IID situation in real-world applications where data from different clients exhibit varying underlying distributions. This variation leads to feature shift, which is a key issue of this scenario. While previous works have made notable progress, few pay attention to the data itself, i.e., the root of this issue. The primary goal of this paper is to mitigate feature shift from the perspective of data. To this end, we propose a simple yet remarkably effective input-level data augmentation method, namely FedRDN, which randomly injects the statistical information of the local distribution from the entire federation into the client's data. This is beneficial to improve the generalization of local feature representations, thereby mitigating feature shift. Moreover, our FedRDN is a plug-and-play component, which can be seamlessly integrated into the data augmentation flow with only a few lines of code. Extensive experiments on several datasets show that the performance of various representative FL methods can be further improved by integrating our FedRDN, demonstrating its effectiveness, strong compatibility and generalizability. Code will be released.
Paper Structure (19 sections, 9 equations, 3 figures, 8 tables)

This paper contains 19 sections, 9 equations, 3 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Preliminaries
  5. Federated Random Data Normalization
  6. Privacy Security
  7. Experiments
  8. Experimental Setup
  9. Main Results
  10. Communication Efficiency
  11. Cross-site Generalization performance
  12. Analytical Studies
  13. Conclusion
  14. Limitation
  15. More Details
  16. ...and 4 more sections

Figures (3)

  • Figure 1: Illustration of test performance versus communication rounds on (a) Office-Caltech-10 gong2012geodesic, (b) DomainNet peng2019moment, and (c) ProstateMRI liu2020ms.
  • Figure 2: Illustration of test performance versus local epochs on (a) Office-Caltech-10 gong2012geodesic and (b) DomainNet peng2019moment.
  • Figure 3: T-SNE visualization of features on Office-Caltech-10 gong2012geodesic. The T-SNE is conducted on the test sample features of four clients. We use different colors to mark features from different clients.