GeFL: Model-Agnostic Federated Learning with Generative Models

TL;DR

The paper tackles federated learning across heterogeneous client architectures by introducing GeFL, which aggregates global knowledge through a federated generative model and augments local training with synthetic data. To address privacy and scalability concerns, it further proposes GeFL-F, a feature-level variant that shares a common feature extractor and feature-generative models to reduce privacy risk and communication load while maintaining performance. Extensive experiments on MNIST, FMNIST, and CIFAR10 across 10 heterogeneous models demonstrate that GeFL and GeFL-F outperform traditional FL baselines and existing model-heterogeneous approaches, with diffusion-based generators often delivering strong gains. The work highlights practical implications for edge deployments and outlines privacy considerations (MND memorization) and resource trade-offs, offering a pathway toward scalable, private, model-agnostic FL in real-world systems.

Abstract

Federated learning (FL) is a distributed training paradigm that enables collaborative learning across clients without sharing local data, thereby preserving privacy. However, the increasing scale and complexity of modern deep models often exceed the computational or memory capabilities of edge devices. Furthermore, clients may be constrained to use heterogeneous model architectures due to hardware variability (e.g., ASICs, FPGAs) or proprietary requirements that prevent the disclosure or modification of local model structures. These practical considerations motivate the need for model-heterogeneous FL, where clients participate using distinct model architectures. In this work, we propose Generative Model-Aided Federated Learning (GeFL), a framework that enables cross-client knowledge sharing via a generative model trained in a federated manner. This generative model captures global data semantics and facilitates local training without requiring model homogeneity across clients. While GeFL achieves strong performance, empirical analysis reveals limitations in scalability and potential privacy leakage due to generative sample memorization. To address these concerns, we propose GeFL-F, which utilizes feature-level generative modeling. This approach enhances scalability to large client populations and mitigates privacy risks. Extensive experiments across image classification tasks demonstrate that both GeFL and GeFL-F offer competitive performance in heterogeneous settings. Code is available at [1].

Figures (15)

• Figure 1: Illustration of GeFL. We propose a model-agnostic FL framework under model-heterogeneity, GeFL, consisting of (i) generative knowledge aggregation which trains a generative model in a federated manner, and (ii) heterogeneous target network training augmented by trained generative models.
• Figure 2: Generated images from FedCVAE (top two rows) and FedDCGAN (bottom two rows) which are trained across the different number of clients
• Figure 3: Illustration of GeFL-F algorithm. Each target network is decoupled into a common feature extractor and a heterogeneous header. There are three main stages in GeFL-F including (i) warming-up phase for training a common feature extractor, and (ii) generative knowledge aggregation which trains feature-generative models using warmed-up feature extractor, and (iii) target network training, especially the heterogeneous headers, using real samples and generated features.
• Figure 4: Scalability in client numbers of GeFL and GeFL-F. GeFL-F exhibits less performance degradation in a large number of clients compared to GeFL.
• Figure 5: Reconstructed samples generated by feature-generative models by model-inversion