Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GRANITE : a Byzantine-Resilient Dynamic Gossip Learning Framework

Yacine Belal, Mohamed Maouche, Sonia Ben Mokhtar, Anthony Simonet-Boulogne

TL;DR

This work introduces GRANITE, a framework for robust learning over sparse, dynamic graphs in the presence of a fraction of Byzantine nodes, and improves learning speed via adaptive filtering of poisoned models and obtains these results in up to 9 times sparser graphs than dictated by current theory.

Abstract

Gossip Learning (GL) is a decentralized learning paradigm where users iteratively exchange and aggregate models with a small set of neighboring peers. Recent GL approaches rely on dynamic communication graphs built and maintained using Random Peer Sampling (RPS) protocols. Thanks to graph dynamics, GL can achieve fast convergence even over extremely sparse topologies. However, the robustness of GL over dy- namic graphs to Byzantine (model poisoning) attacks remains unaddressed especially when Byzantine nodes attack the RPS protocol to scale up model poisoning. We address this issue by introducing GRANITE, a framework for robust learning over sparse, dynamic graphs in the presence of a fraction of Byzantine nodes. GRANITE relies on two key components (i) a History-aware Byzantine-resilient Peer Sampling protocol (HaPS), which tracks previously encountered identifiers to reduce adversarial influence over time, and (ii) an Adaptive Probabilistic Threshold (APT), which leverages an estimate of Byzantine presence to set aggregation thresholds with formal guarantees. Empirical results confirm that GRANITE maintains convergence with up to 30% Byzantine nodes, improves learning speed via adaptive filtering of poisoned models and obtains these results in up to 9 times sparser graphs than dictated by current theory.

GRANITE : a Byzantine-Resilient Dynamic Gossip Learning Framework

TL;DR

This work introduces GRANITE, a framework for robust learning over sparse, dynamic graphs in the presence of a fraction of Byzantine nodes, and improves learning speed via adaptive filtering of poisoned models and obtains these results in up to 9 times sparser graphs than dictated by current theory.

Abstract

Gossip Learning (GL) is a decentralized learning paradigm where users iteratively exchange and aggregate models with a small set of neighboring peers. Recent GL approaches rely on dynamic communication graphs built and maintained using Random Peer Sampling (RPS) protocols. Thanks to graph dynamics, GL can achieve fast convergence even over extremely sparse topologies. However, the robustness of GL over dy- namic graphs to Byzantine (model poisoning) attacks remains unaddressed especially when Byzantine nodes attack the RPS protocol to scale up model poisoning. We address this issue by introducing GRANITE, a framework for robust learning over sparse, dynamic graphs in the presence of a fraction of Byzantine nodes. GRANITE relies on two key components (i) a History-aware Byzantine-resilient Peer Sampling protocol (HaPS), which tracks previously encountered identifiers to reduce adversarial influence over time, and (ii) an Adaptive Probabilistic Threshold (APT), which leverages an estimate of Byzantine presence to set aggregation thresholds with formal guarantees. Empirical results confirm that GRANITE maintains convergence with up to 30% Byzantine nodes, improves learning speed via adaptive filtering of poisoned models and obtains these results in up to 9 times sparser graphs than dictated by current theory.

Paper Structure

This paper contains 36 sections, 4 theorems, 32 equations, 7 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Dynamic GL
  4. Robust Learning
  5. Byzantine-resilient Random Peer Sampling
  6. Preliminaries
  7. Dynamic Gossip Learning
  8. Random Peer-Sampling service
  9. Byzantine-Resilient Learning and Robust Aggregators
  10. Problem Setup and Notation
  11. Threat Model
  12. Assumptions
  13. Notation
  14. GRANITE
  15. System Overview
  16. ...and 21 more sections

Key Result

Theorem 5.1

Let $C({t})$ denote the number of unique honest identifiers known to a node at time $t$, and assume that correct identifiers arrive at an overall rate $\alpha$. We Then have:

Figures (7)

  • Figure 1: F1-Score comparison between GRANITE and three flavors of BASALT under $F=2$ and CS aggregator over MNIST dataset.
  • Figure 2: F1-Score comparison between GRANITE and three flavors of BASALT under $F=2$ and CS aggregator over the Purchase100 dataset.
  • Figure 3: F1-Score analysis of GRANITE under FOE and ALIE attacks with different robust aggregators on the MNIST dataset.
  • Figure 4: F1-Score analysis of GRANITE under FOE and ALIE attacks with different robust aggregators on the Purchase100 dataset.
  • Figure 5: Evolution of $f_{in}$ over rounds and flooding attack with different values of $F$.
  • ...and 2 more figures

Theorems & Definitions (7)

  • Definition 3.1
  • Definition 3.2
  • Definition 3.3
  • Theorem 5.1: Exponential Convergence of Known-Honest IDs
  • Corollary 5.2
  • Lemma 5.3: APT Filtering Guarantee
  • Theorem 8.1: Chernoff Bound – Upper Tail