Fair Federated Medical Image Classification Against Quality Shift via Inter-Client Progressive State Matching

TL;DR

This work tackles fairness in federated medical image classification under imaging-quality shifts across institutions. It critiques prior approaches for relying on a single convergence state and introduces a comprehensive framework that evaluates and aligns multiple states of convergence (sharpness/perturbed loss) across varying search distances. Building on this, FedISM+ progressively expands the considered state space during training via a dynamic radius $\rho(t)$, combining sharpness-aware local optimization (SALT) with state-aware aggregation (SAGA) and a moving-average scheme to ensure cross-client fairness. Empirical results on RSNA ICH and ISIC 2019 demonstrate that FedISM+ outperforms state-of-the-art fair-FL methods in both test-time accuracy and cross-client fairness, with negligible privacy risks and minimal communication overhead. These findings suggest that multi-state convergence assessment is a practical and effective principle for fair FL in real-world, heterogeneous medical imaging deployments.

Abstract

Despite the potential of federated learning in medical applications, inconsistent imaging quality across institutions-stemming from lower-quality data from a minority of clients-biases federated models toward more common high-quality images. This raises significant fairness concerns. Existing fair federated learning methods have demonstrated some effectiveness in solving this problem by aligning a single 0th- or 1st-order state of convergence (e.g., training loss or sharpness). However, we argue in this work that fairness based on such a single state is still not an adequate surrogate for fairness during testing, as these single metrics fail to fully capture the convergence characteristics, making them suboptimal for guiding fair learning. To address this limitation, we develop a generalized framework. Specifically, we propose assessing convergence using multiple states, defined as sharpness or perturbed loss computed at varying search distances. Building on this comprehensive assessment, we propose promoting fairness for these states across clients to achieve our ultimate fairness objective. This is accomplished through the proposed method, FedISM+. In FedISM+, the search distance evolves over time, progressively focusing on different states. We then incorporate two components in local training and global aggregation to ensure cross-client fairness for each state. This gradually makes convergence equitable for all states, thereby improving fairness during testing. Our empirical evaluations, performed on the well-known RSNA ICH and ISIC 2019 datasets, demonstrate the superiority of FedISM+ over existing state-of-the-art methods for fair federated learning. The code is available at https://github.com/wnn2000/FFL4MIA.

Figures (7)

• Figure 1: Imaging quality shifts across clients, where most clients have high-quality (i.e., clean) images, while others experience corruption, such as noise or motion blur.
• Figure 2: Comparison of fair optimization and fair generalization: Fair optimization typically leads to convergence at sharp minima (with model parameters $\boldsymbol{\theta}_1$), resulting in uniformly low loss values but poor testing performance. In contrast, fair generalization emphasizes uniformly low sharpness (with model parameters $\boldsymbol{\theta}_2$), which improves absolute performance and fairness when testing.
• Figure 3: Illustration of FedISM+. The key insight is to maintain uniformly low sharpness across clients, defined at progressively increasing search distances as training progresses.
• Figure 4: Left: Comparison of FedISM+ with FedISM across different $\rho$ values. Right: Evaluation across different $\tau$ settings. The solid lines indicate the mean values, and the shaded regions represent the standard deviations.
• Figure 5: Left: Evaluation on different $q$. Right: Evaluation on different $\beta$. The solid lines indicate the mean values, and the shaded regions represent the standard deviations. The best fair optimization method from Section \ref{['sec:sota']} is denoted as "Best FO" according to the performance on corrupted images.

Theorems & Definitions (1)

• Theorem 1: Equivalence