Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Optimization to Generalization: Fair Federated Learning against Quality Shift via Inter-Client Sharpness Matching

Nannan Wu, Zhuo Kuang, Zengqiang Yan, Li Yu

TL;DR

This work identifies imaging quality shift as a practical fairness challenge in federated learning for medical imaging and demonstrates that fairness should target generalization rather than solely optimizing training loss. It introduces FedISM, a sharpness-aware FL framework that equalizes inter-client sharpness by combining sharpness-aware local updates with sharpness-driven, quality-aware aggregation, formalized as a minimax objective over client distributions and sharpness metrics. Empirical results on RSNA ICH and ISIC 2019 show FedISM outperforms state-of-the-art fair FL methods in corrupted-image settings while maintaining strong performance on clean data, highlighting improved generalization fairness. The approach is privacy-preserving, lightweight to implement, and robust to different levels of data-quality imbalance, suggesting practical impact for equitable deployment of FL in medical imaging and beyond.

Abstract

Due to escalating privacy concerns, federated learning has been recognized as a vital approach for training deep neural networks with decentralized medical data. In practice, it is challenging to ensure consistent imaging quality across various institutions, often attributed to equipment malfunctions affecting a minority of clients. This imbalance in image quality can cause the federated model to develop an inherent bias towards higher-quality images, thus posing a severe fairness issue. In this study, we pioneer the identification and formulation of this new fairness challenge within the context of the imaging quality shift. Traditional methods for promoting fairness in federated learning predominantly focus on balancing empirical risks across diverse client distributions. This strategy primarily facilitates fair optimization across different training data distributions, yet neglects the crucial aspect of generalization. To address this, we introduce a solution termed Federated learning with Inter-client Sharpness Matching (FedISM). FedISM enhances both local training and global aggregation by incorporating sharpness-awareness, aiming to harmonize the sharpness levels across clients for fair generalization. Our empirical evaluations, conducted using the widely-used ICH and ISIC 2019 datasets, establish FedISM's superiority over current state-of-the-art federated learning methods in promoting fairness. Code is available at https://github.com/wnn2000/FFL4MIA.

From Optimization to Generalization: Fair Federated Learning against Quality Shift via Inter-Client Sharpness Matching

TL;DR

This work identifies imaging quality shift as a practical fairness challenge in federated learning for medical imaging and demonstrates that fairness should target generalization rather than solely optimizing training loss. It introduces FedISM, a sharpness-aware FL framework that equalizes inter-client sharpness by combining sharpness-aware local updates with sharpness-driven, quality-aware aggregation, formalized as a minimax objective over client distributions and sharpness metrics. Empirical results on RSNA ICH and ISIC 2019 show FedISM outperforms state-of-the-art fair FL methods in corrupted-image settings while maintaining strong performance on clean data, highlighting improved generalization fairness. The approach is privacy-preserving, lightweight to implement, and robust to different levels of data-quality imbalance, suggesting practical impact for equitable deployment of FL in medical imaging and beyond.

Abstract

Due to escalating privacy concerns, federated learning has been recognized as a vital approach for training deep neural networks with decentralized medical data. In practice, it is challenging to ensure consistent imaging quality across various institutions, often attributed to equipment malfunctions affecting a minority of clients. This imbalance in image quality can cause the federated model to develop an inherent bias towards higher-quality images, thus posing a severe fairness issue. In this study, we pioneer the identification and formulation of this new fairness challenge within the context of the imaging quality shift. Traditional methods for promoting fairness in federated learning predominantly focus on balancing empirical risks across diverse client distributions. This strategy primarily facilitates fair optimization across different training data distributions, yet neglects the crucial aspect of generalization. To address this, we introduce a solution termed Federated learning with Inter-client Sharpness Matching (FedISM). FedISM enhances both local training and global aggregation by incorporating sharpness-awareness, aiming to harmonize the sharpness levels across clients for fair generalization. Our empirical evaluations, conducted using the widely-used ICH and ISIC 2019 datasets, establish FedISM's superiority over current state-of-the-art federated learning methods in promoting fairness. Code is available at https://github.com/wnn2000/FFL4MIA.
Paper Structure (31 sections, 2 theorems, 19 equations, 7 figures, 8 tables, 1 algorithm)

This paper contains 31 sections, 2 theorems, 19 equations, 7 figures, 8 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Fair Federated Learning
  4. Sharpness of Loss Surface
  5. Methodology
  6. Preliminaries
  7. Previous Solution: Fair Optimization
  8. Measurement of Generalization: Sharpness
  9. Sharpness Matching for Fair Generalization
  10. Experiments
  11. Experimental Setup
  12. Datasets
  13. Model
  14. Implement Details
  15. Evaluation Strategy
  16. ...and 16 more sections

Key Result

Theorem 1

Assuming class distributions of the testing set and all clients' training sets are identical, we have: and where $\mu^*_u = \sum_{k=1}^{K} \mathds{1}_{a_k=u} \lambda^*_k$.

Figures (7)

  • Figure 1: Imaging quality shift across clients. Most clients possess clean images, while others have corrupted images (e.g., exhibiting noise or blur).
  • Figure 2: Motivations behind previous fair optimization and our fair generalization. Fair optimization aims for uniform and low loss values across clients, often resulting in convergence at sharp minima and poor testing performance. Comparatively, we focus on achieving uniform sharpness and converging at flat minima, thus enhancing fair generalization and testing performance.
  • Figure 3: Illustration of FedISM. Our insight is to ensure the uniformity of sharpness across clients, leading to fair generalization.
  • Figure 4: Evaluation across diverse ratios of clean and corrupted clients. Solid lines denote the mean values, while the transparent areas depict the standard deviations. Second best refers to the second best fair FL methods from Section \ref{['sec:sota']}, specifically in terms of their performance on corrupted images.
  • Figure 5: Evaluation on different $q$. Solid lines denote the mean values, while the transparent areas depict the standard deviations. Second best refers to the second best fair FL methods from Section \ref{['sec:sota']}, specifically in terms of their performance on corrupted images.
  • ...and 2 more figures

Theorems & Definitions (3)

  • Theorem 1: Equivalence
  • Theorem 1: Equivalence
  • proof : Proof