Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedBiCross: A Bi-Level Optimization Framework to Tackle Non-IID Challenges in Data-Free One-Shot Federated Learning on Medical Data

Yuexuan Xia, Yinghao Zhang, Yalin Liu, Hong-Ning Dai, Yong Xia

TL;DR

FedBiCross is proposed, a personalized OSFL framework with three stages: clustering clients by model output similarity to form coherent sub-ensembles, bi-level cross-cluster optimization that learns adaptive weights to selectively leverage beneficial cross-cluster knowledge while suppressing negative transfer, and personalized distillation for client-specific adaptation.

Abstract

Data-free knowledge distillation-based one-shot federated learning (OSFL) trains a model in a single communication round without sharing raw data, making OSFL attractive for privacy-sensitive medical applications. However, existing methods aggregate predictions from all clients to form a global teacher. Under non-IID data, conflicting predictions cancel out during averaging, yielding near-uniform soft labels that provide weak supervision for distillation. We propose FedBiCross, a personalized OSFL framework with three stages: (1) clustering clients by model output similarity to form coherent sub-ensembles, (2) bi-level cross-cluster optimization that learns adaptive weights to selectively leverage beneficial cross-cluster knowledge while suppressing negative transfer, and (3) personalized distillation for client-specific adaptation. Experiments on four medical image datasets demonstrate that FedBiCross consistently outperforms state-of-the-art baselines across different non-IID degrees.

FedBiCross: A Bi-Level Optimization Framework to Tackle Non-IID Challenges in Data-Free One-Shot Federated Learning on Medical Data

TL;DR

FedBiCross is proposed, a personalized OSFL framework with three stages: clustering clients by model output similarity to form coherent sub-ensembles, bi-level cross-cluster optimization that learns adaptive weights to selectively leverage beneficial cross-cluster knowledge while suppressing negative transfer, and personalized distillation for client-specific adaptation.

Abstract

Data-free knowledge distillation-based one-shot federated learning (OSFL) trains a model in a single communication round without sharing raw data, making OSFL attractive for privacy-sensitive medical applications. However, existing methods aggregate predictions from all clients to form a global teacher. Under non-IID data, conflicting predictions cancel out during averaging, yielding near-uniform soft labels that provide weak supervision for distillation. We propose FedBiCross, a personalized OSFL framework with three stages: (1) clustering clients by model output similarity to form coherent sub-ensembles, (2) bi-level cross-cluster optimization that learns adaptive weights to selectively leverage beneficial cross-cluster knowledge while suppressing negative transfer, and (3) personalized distillation for client-specific adaptation. Experiments on four medical image datasets demonstrate that FedBiCross consistently outperforms state-of-the-art baselines across different non-IID degrees.
Paper Structure (22 sections, 9 equations, 3 figures, 3 tables, 1 algorithm)

This paper contains 22 sections, 9 equations, 3 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Personalized Federated Learning
  4. One-Shot Federated Learning
  5. Methodology
  6. Stage 1: Client Clustering and Data Generation
  7. Client Clustering via Similarity
  8. Trajectory-Based Synthesis
  9. Noise-Adapted Teachers
  10. Stage 2: Bi-Level Cross-Cluster Optimization
  11. Problem Formulation
  12. Online Optimization
  13. Stage 3: Personalized Knowledge Distillation
  14. Experiments
  15. Experimental Setup
  16. ...and 7 more sections

Figures (3)

  • Figure 1: Overview of FedBiCross. (a) Previous methods (e.g., FedISCA kang2023one) construct a unified ensemble teacher from all clients, ignoring distribution heterogeneity. (b) Our approach consists of three stages: Stage 1 clusters clients by model output similarity and generates cluster-specific synthetic data; Stage 2 performs bi-level optimization to learn adaptive cross-cluster weights; Stage 3 personalizes models for individual clients via local fine-tuning.
  • Figure 2: Ablation on the clustering stage for BloodMNIST. Test accuracy (%) under varying client numbers $N$ and heterogeneity levels $\alpha$.
  • Figure 3: Visualization of synthetic images generated by different methods under non-IID settings with $\alpha=0.3$ and $N=10$: (a) DAFL, (b) DENSE, (c) FedISCA, (d) Co-Boosting, and (e) Ours.