GENE-FL: Gene-Driven Parameter-Efficient Dynamic Federated Learning

TL;DR

The paper tackles dynamic agnostic federated learning by introducing GENE-FL, a gene-driven, parameter-efficient framework that condenses large models into lightweight learnGene fragments via learnGene Condensation, Aggregation, and Initialization. It uses per-layer adaptive smooth updates and Fisher-information-based elasticity to preserve generalizable knowledge, while enabling cluster-level aggregation to support rapid initialization of agnostic clients through nearest-cluster learnGene transfer. Empirical results demonstrate substantial communication savings (up to ~4×) and competitive or superior accuracy with only about 9.04 MB per cluster learnGene for initializing new clients, along with strong privacy considerations when sharing learnGene. The work shows that learnGene-based inheritance can significantly improve efficiency and scalability in DAFL, offering practical impact for real-world dynamic FL deployments.

Abstract

Real-world \underline{F}ederated \underline{L}earning systems often encounter \underline{D}ynamic clients with \underline{A}gnostic and highly heterogeneous data distributions (DAFL), which pose challenges for efficient communication and model initialization. To address these challenges, we draw inspiration from the recently proposed Learngene paradigm, which compresses the large-scale model into lightweight, cross-task meta-information fragments. Learngene effectively encapsulates and communicates core knowledge, making it particularly well-suited for DAFL, where dynamic client participation requires communication efficiency and rapid adaptation to new data distributions. Based on this insight, we propose a Gene-driven parameter-efficient dynamic Federated Learning (GENE-FL) framework. First, local models perform quadratic constraints based on parameters with high Fisher values in the global model, as these parameters are considered to encapsulate generalizable knowledge. Second, we apply the strategy of parameter sensitivity analysis in local model parameters to condense the \textit{learnGene} for interaction. Finally, the server aggregates these small-scale trained \textit{learnGene}s into a robust \textit{learnGene} with cross-task generalization capability, facilitating the rapid initialization of dynamic agnostic client models. Extensive experimental results demonstrate that GENE-FL reduces \textbf{4 $\times$} communication costs compared to FEDAVG and effectively initializes agnostic client models with only about \textbf{9.04} MB.

Figures (6)

• Figure 1: Illustration of Dynamic Agnostic Federated Learning and Learngene. In the accumulating, condensing, and inheriting processes of the Learngene, dynamic agnostic federated learning can achieve a corresponding organic integration.
• Figure 2: Illustration of training process of GENE-FL, which includes (I) learnGene Condensation via Smooth Updating, (II) learnGene Collaborative Aggregation, and (III) learnGene Initial Agnostic Client Model.
• Figure 3: Performance curve comparison on CIFAR-10 with $s$ = 4. GENE-FL, initialized by the learnGene, shows the fastest convergence rate, with its test accuracy increasing rapidly and significantly outperforming other methods.
• Figure 4: Higher privacy protection. Reconstructing images under iDLG attacks in FEDAVG, FEDLG, PARFED, and the proposed method. Images are extracted from CIFAR-10 and CIFAR-100 datasets, with corresponding PSNR reported beneath each recovered image.
• Figure 5: Comparison of communication cost curves from client to server over communication rounds on the CIFAR-10 dataset under $s$=5 data partition.