Harmonizing Generalization and Personalization in Ring-topology Decentralized Federated Learning

TL;DR

The paper tackles data heterogeneity in ring-topology decentralized federated learning by proposing DRDFL, a divide-and-conquer framework that jointly optimizes personalization and generalization. It decomposes learning into PersonaNet, which enforces a Gaussian-mixture latent structure for class-specific personalization, and Learngene, which captures globally invariant knowledge via adversarial alignment under a uniform class prior. The approach yields strong performance gains across multiple non-IID settings with dramatically reduced communication (about 0.58M parameters vs hundreds of millions) and includes theoretical convergence guarantees for the non-convex setting. Practically, DRDFL enables robust, scalable, and communication-efficient decentralized learning with secure, distribution-aware initialization for new clients.

Abstract

We introduce Ring-topology Decentralized Federated Learning (RDFL) for distributed model training, aiming to avoid the inherent risks of centralized failure in server-based FL. However, RDFL faces the challenge of low information-sharing efficiency due to the point-to-point communication manner when handling inherent data heterogeneity. Existing studies to mitigate data heterogeneity focus on personalized optimization of models, ignoring that the lack of shared information constraints can lead to large differences among models, weakening the benefits of collaborative learning. To tackle these challenges, we propose a Divide-and-conquer RDFL framework (DRDFL) that uses a feature generation model to extract personalized information and invariant shared knowledge from the underlying data distribution, ensuring both effective personalization and strong generalization. Specifically, we design a \textit{PersonaNet} module that encourages class-specific feature representations to follow a Gaussian mixture distribution, facilitating the learning of discriminative latent representations tailored to local data distributions. Meanwhile, the \textit{Learngene} module is introduced to encapsulate shared knowledge through an adversarial classifier to align latent representations and extract globally invariant information. Extensive experiments demonstrate that DRDFL outperforms state-of-the-art methods in various data heterogeneity settings.

Figures (21)

• Figure 1: Illustration of the optimized learning mechanism in RDFL and comparison of the personalization and generalization performance of various advanced methods. (a) shows the schematic of the ring optimization approach, where client 1 receives the model from client 4, trains and updates it using its private dataset, and then transmits the locally updated model to client 2. The model is iteratively optimized over the ring topology. (b) presents the performance comparison of personalization (Local-T) and generalization (Global-T) on different methods across four clients with different data distributions. The results show that our proposed method effectively achieves both personalization and generalization goals in federated learning scenarios with data heterogeneity.
• Figure 2: Illustration of the DRDFL framework. It adopts a divide-and-conquer strategy, where PersonaNet captures class-specific personalized representations via a Gaussian mixture distribution, enhancing adaptability to local data. Meanwhile, Learngene employs adversarial alignment to ensure cross-client feature consistency and encapsulate globally invariant knowledge. During inference, the decoder reconstructs inputs with noise for robustness, while iterative updates of global latent representations and Learngene enhance information sharing across clients.
• Figure 3: CIFAR-10 with $\beta$ = 0.1
• Figure 4: CIFAR-10 with $\beta$ = 0.4
• Figure 5: CIFAR-10 with $s$ = 4