Central Server Free Federated Learning over Single-sided Trust Social Networks
Chaoyang He, Conghui Tan, Hanlin Tang, Shuang Qiu, Ji Liu
TL;DR
Addresses federated learning without a central server on directed social graphs with single-sided trust. Introduces Online Push-Sum (OPS), exchanging models rather than gradients and using a row-stochastic confusion matrix to accommodate asymmetric topologies. Provides a regret bound that captures adversarial and stochastic loss components and proves per-node models converge at rate O(1/T). Demonstrates through experiments on SUSY and Room Occupancy that OPS outperforms decentralized online learning under asymmetric networks and scales well with network size and density while preserving privacy advantages.
Abstract
Federated learning has become increasingly important for modern machine learning, especially for data privacy-sensitive scenarios. Existing federated learning mostly adopts the central server-based architecture or centralized architecture. However, in many social network scenarios, centralized federated learning is not applicable (e.g., a central agent or server connecting all users may not exist, or the communication cost to the central server is not affordable). In this paper, we consider a generic setting: 1) the central server may not exist, and 2) the social network is unidirectional or of single-sided trust (i.e., user A trusts user B but user B may not trust user A). We propose a central server free federated learning algorithm, named Online Push-Sum (OPS) method, to handle this challenging but generic scenario. A rigorous regret analysis is also provided, which shows very interesting results on how users can benefit from communication with trusted users in the federated learning scenario. This work builds upon the fundamental algorithm framework and theoretical guarantees for federated learning in the generic social network scenario.