Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Peer-to-Peer Deep Learning for Beyond-5G IoT

Srinivasa Pranav, José M. F. Moura

TL;DR

This work tackles the scalability and privacy limitations of server-based federated learning in beyond-5G IoT by introducing P2PL, a fully distributed device-to-device deep learning framework. It combines a max norm synchronization initialization with alternating on-device training and neighbor consensus, using device-specific mixing weights to accommodate nonuniform data and network topology, and demonstrates that 100 devices can achieve test performance on par with federated and centralized training. Key contributions include showing faster convergence with max norm synchronization, relaxing doubly stochastic mixing assumptions, and proving robustness to diverse graphs, intermittent communications, and non-IID data distributions. The approach promises privacy-preserving, scalable DL for large, heterogeneous IoT networks without centralized servers, reducing latency and single points of failure while maintaining performance.

Abstract

We present P2PL, a practical multi-device peer-to-peer deep learning algorithm that, unlike the federated learning paradigm, does not require coordination from edge servers or the cloud. This makes P2PL well-suited for the sheer scale of beyond-5G computing environments like smart cities that otherwise create range, latency, bandwidth, and single point of failure issues for federated approaches. P2PL introduces max norm synchronization to catalyze training, retains on-device deep model training to preserve privacy, and leverages local inter-device communication to implement distributed consensus. Each device iteratively alternates between two phases: 1) on-device learning and 2) peer-to-peer cooperation where they combine model parameters with nearby devices. We empirically show that all participating devices achieve the same test performance attained by federated and centralized training -- even with 100 devices and relaxed singly stochastic consensus weights. We extend these experimental results to settings with diverse network topologies, sparse and intermittent communication, and non-IID data distributions.

Peer-to-Peer Deep Learning for Beyond-5G IoT

TL;DR

This work tackles the scalability and privacy limitations of server-based federated learning in beyond-5G IoT by introducing P2PL, a fully distributed device-to-device deep learning framework. It combines a max norm synchronization initialization with alternating on-device training and neighbor consensus, using device-specific mixing weights to accommodate nonuniform data and network topology, and demonstrates that 100 devices can achieve test performance on par with federated and centralized training. Key contributions include showing faster convergence with max norm synchronization, relaxing doubly stochastic mixing assumptions, and proving robustness to diverse graphs, intermittent communications, and non-IID data distributions. The approach promises privacy-preserving, scalable DL for large, heterogeneous IoT networks without centralized servers, reducing latency and single points of failure while maintaining performance.

Abstract

We present P2PL, a practical multi-device peer-to-peer deep learning algorithm that, unlike the federated learning paradigm, does not require coordination from edge servers or the cloud. This makes P2PL well-suited for the sheer scale of beyond-5G computing environments like smart cities that otherwise create range, latency, bandwidth, and single point of failure issues for federated approaches. P2PL introduces max norm synchronization to catalyze training, retains on-device deep model training to preserve privacy, and leverages local inter-device communication to implement distributed consensus. Each device iteratively alternates between two phases: 1) on-device learning and 2) peer-to-peer cooperation where they combine model parameters with nearby devices. We empirically show that all participating devices achieve the same test performance attained by federated and centralized training -- even with 100 devices and relaxed singly stochastic consensus weights. We extend these experimental results to settings with diverse network topologies, sparse and intermittent communication, and non-IID data distributions.
Paper Structure (16 sections, 6 equations, 2 figures, 3 tables, 1 algorithm)

This paper contains 16 sections, 6 equations, 2 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Centralized and Federated Deep Learning
  4. Peer-to-Peer Deep Learning
  5. Peer-to-Peer Deep Learning
  6. Optimization Problem Setup
  7. Device-to-Device Communication and Connectivity
  8. Peer-to-Peer Learning Algorithm
  9. Experimental Results and Discussion
  10. Experimental Setup
  11. Insights from a Complete Communication Graph
  12. Test Performance Robust to Diverse Communication Graphs
  13. Doubly Stochastic Mixing Weights: Metropolis-Hastings
  14. Pathological Non-IID Data Distribution
  15. Link Failure Resiliency and Sparse Gossip Communication
  16. ...and 1 more sections

Figures (2)

  • Figure 1: Federated setting (left); Peer-to-Peer setting (right).
  • Figure 2: P2PL convergence on various communication graphs ($K{=}100$). Average test accuracy vs. rounds. Rounds to convergence in parentheses. Avg. accuracy may exceed $97\%$ before min. accuracy.