Federated Learning Framework via Distributed Mutual Learning
Yash Gupta
TL;DR
The paper addresses privacy and bandwidth limitations in traditional federated learning by introducing a loss-based distributed mutual learning framework that exchanges loss predictions on a public test set instead of sharing model weights. It formalizes a loss objective $L = L_{model} + L_{KL}$ with $L_{KL} = \frac{1}{K-1}\sum_{j \neq i}^{K} P_i \log\frac{P_i}{P_j}$ to drive inter-client alignment across rounds. The proposed method is evaluated on a face-mask detection task using a VisionNet-like CNN and demonstrates higher unseen-data accuracy and better generalization than weight-sharing baselines, while preserving privacy and reducing bandwidth. The work suggests that distributed mutual learning can enable privacy-preserving, bandwidth-efficient federated learning in IoT contexts, with future directions including non-IID data handling and diverse architectures.
Abstract
Federated Learning often relies on sharing full or partial model weights, which can burden network bandwidth and raise privacy risks. We present a loss-based alternative using distributed mutual learning. Instead of transmitting weights, clients periodically share their loss predictions on a public test set. Each client then refines its model by combining its local loss with the average Kullback-Leibler divergence over losses from other clients. This collaborative approach both reduces transmission overhead and preserves data privacy. Experiments on a face mask detection task demonstrate that our method outperforms weight-sharing baselines, achieving higher accuracy on unseen data while providing stronger generalization and privacy benefits.