Personalized Multi-tier Federated Learning
Sourasekhar Banerjee, Ali Dadras, Alp Yurtsever, Monowar Bhuyan
TL;DR
This work addresses data heterogeneity in federated learning by introducing PerMFL, a personalized multi-tier framework that learns a global model, per-team models, and per-device models within a hierarchical cloud-edge structure. The method formulates a three-level optimization with squared Euclidean penalties linking device, team, and global models, and solves it via device-, team-, and server-level updates that exploit Moreau envelopes to balance personalization with collaboration. Theoretical guarantees cover both smooth strongly convex and smooth non-convex losses, yielding linear convergence under suitable conditions and sublinear convergence to first-order stationary points, with explicit parameter bounds. Empirically, PerMFL demonstrates robust performance and fast convergence across MNIST, FMNIST, EMNIST, FEMNIST, CIFAR100, and synthetic non-IID data, often outperforming state-of-the-art FL methods and showing resilience to various team formations and participation scenarios, while reducing global-communication costs through intra-team updates.
Abstract
The key challenge of personalized federated learning (PerFL) is to capture the statistical heterogeneity properties of data with inexpensive communications and gain customized performance for participating devices. To address these, we introduced personalized federated learning in multi-tier architecture (PerMFL) to obtain optimized and personalized local models when there are known team structures across devices. We provide theoretical guarantees of PerMFL, which offers linear convergence rates for smooth strongly convex problems and sub-linear convergence rates for smooth non-convex problems. We conduct numerical experiments demonstrating the robust empirical performance of PerMFL, outperforming the state-of-the-art in multiple personalized federated learning tasks.