Analytic Personalized Federated Meta-Learning

TL;DR

This work tackles the challenge of efficient, personalization-capable learning in gradient-free analytic Federated Learning (AFL) with heterogeneous client data. It introduces FedACnnL, a layer-wise, distributed LS-based FedLearning framework that enables DNN training in AFL, and its personalized extension, pFedACnnL, which forms group-wise meta-models and applies gradient-free local personalization. The proposed methods demonstrate dramatic training-time reductions (up to 83-99% faster) and competitive or state-of-the-art test accuracy across convex and non-convex settings, validated on MNIST, synthetic, and CIFAR-10 datasets with realistic heterogeneity. The adaptive batch-size mechanism further mitigates stragglers without affecting final models, highlighting practical gains in real-world, resource-diverse FL deployments. Overall, the paper provides a scalable, gradient-free pathway to fast, personalized federated learning for deep networks in heterogeneous environments.

Abstract

Analytic Federated Learning (AFL) is an enhanced gradient-free federated learning (FL) paradigm designed to accelerate training by updating the global model in a single step with closed-form least-square (LS) solutions. However, the obtained global model suffers performance degradation across clients with heterogeneous data distribution. Meta-learning is a common approach to tackle this problem by delivering personalized local models for individual clients. Yet, integrating meta-learning with AFL presents significant challenges: First, conventional AFL frameworks cannot support deep neural network (DNN) training which can influence the fast adaption capability of meta-learning for complex FL tasks. Second, the existing meta-learning method requires gradient information, which is not involved in AFL. To overcome the first challenge, we propose an AFL framework, namely FedACnnL, in which a layer-wise DNN collaborative training method is designed by modeling the training of each layer as a distributed LS problem. For the second challenge, we further propose an analytic personalized federated meta-learning framework, namely pFedACnnL. It generates a personalized model for each client by analytically solving a local objective which bridges the gap between the global model and the individual data distribution. FedACnnL is theoretically proven to require significantly shorter training time than the conventional FL frameworks on DNN training while the reduction ratio is $83\%\sim99\%$ in the experiment. Meanwhile, pFedACnnL excels at test accuracy with the vanilla FedACnnL by $4\%\sim8\%$ and it achieves state-of-the-art (SOTA) model performance in most cases of convex and non-convex settings compared with previous SOTA frameworks.

Figures (6)

• Figure 1: The resampling and flattening process in convolutional layers to make the weights in it updatable by the equation \ref{['eq2']}.
• Figure 2: The workflow of ACnnL in the arbitrary iteration. The red dashed box represents the current training layer while the blue dashed box indicates the next layer to be trained.
• Figure 3: The overview of pFedACnnL. The cyan dashed box represents the initialization stage while the brown dashed box and the orange dashed box represent the federated optimization stage and the local personalization stage respectively.
• Figure 4: The averaged total training time of each client in each framework on the MNIST and synthetic datasets.
• Figure 5: The overall test accuracy of all clients versus training round curve during the training procedure on the MNIST and synthetic datasets.