LoLaFL: Low-Latency Federated Learning via Forward-only Propagation
Jierui Zhang, Jianhao Huang, Kaibin Huang
TL;DR
LoLaFL tackles high-latency federated learning in 6G by adopting white-box forward-only neural networks (ReduNet) to enable layer-by-layer, single-layer updates with two nonlinear aggregation schemes. It introduces harmonic-mean-like and covariance-matrix-based aggregations to fuse local layer parameters or low-rank covariances across devices, significantly reducing communication rounds and latency while preserving accuracy. The framework is supported by latency, complexity, and privacy analyses and validated on MNIST, Fashion-MNIST, and CIFAR-100, showing latency reductions of over 87% and 97% with comparable performance, particularly under non-IID data. These results suggest that forward-only, data-structure-driven learning can enable practical, low-latency edge AI in future 6G networks, with potential extensions in improved coding theory and higher compression techniques.
Abstract
Federated learning (FL) has emerged as a widely adopted paradigm for enabling edge learning with distributed data while ensuring data privacy. However, the traditional FL with deep neural networks trained via backpropagation can hardly meet the low-latency learning requirements in the sixth generation (6G) mobile networks. This challenge mainly arises from the high-dimensional model parameters to be transmitted and the numerous rounds of communication required for convergence due to the inherent randomness of the training process. To address this issue, we adopt the state-of-the-art principle of maximal coding rate reduction to learn linear discriminative features and extend the resultant white-box neural network into FL, yielding the novel framework of Low-Latency Federated Learning (LoLaFL) via forward-only propagation. LoLaFL enables layer-wise transmissions and aggregation with significantly fewer communication rounds, thereby considerably reducing latency. Additionally, we propose two \emph{nonlinear} aggregation schemes for LoLaFL. The first scheme is based on the proof that the optimal NN parameter aggregation in LoLaFL should be harmonic-mean-like. The second scheme further exploits the low-rank structures of the features and transmits the low-rank-approximated covariance matrices of features to achieve additional latency reduction. Theoretic analysis and experiments are conducted to evaluate the performance of LoLaFL. In comparison with traditional FL, the two nonlinear aggregation schemes for LoLaFL can achieve reductions in latency of over 87\% and 97\%, respectively, while maintaining comparable accuracies.