Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sensitivity Curve Maximization: Attacking Robust Aggregators in Distributed Learning

Christian A. Schroth, Stefan Vlaski, Abdelhak M. Zoubir

TL;DR

The paper addresses Byzantine threats in distributed learning and shows that robust aggregators (mean, median, M-estimators, geometric median, etc.) remain vulnerable. It introduces Sensitivity Curve Maximization (SCM) to derive optimal outliers that maximize the disruption of an aggregator, and extends this with aligned, time-coupled attacks (SASCM) to accumulate impact across rounds. The authors formalize the SCM framework, derive attack patterns for 1D/2D aggregators, and demonstrate effectiveness through simulations on linear regression and MNIST datasets, highlighting vulnerabilities of a wide range of schemes. The results indicate that absolute robustness against adversarial timing and direction is difficult to achieve, motivating future defenses that consider temporal alignment and attack directionality.

Abstract

In distributed learning agents aim at collaboratively solving a global learning problem. It becomes more and more likely that individual agents are malicious or faulty with an increasing size of the network. This leads to a degeneration or complete breakdown of the learning process. Classical aggregation schemes are prone to breakdown at small contamination rates, therefore robust aggregation schemes are sought for. While robust aggregation schemes can generally tolerate larger contamination rates, many have been shown to be susceptible to carefully crafted malicious attacks. In this work, we show how the sensitivity curve (SC), a classical tool from robust statistics, can be used to systematically derive optimal attack patterns against arbitrary robust aggregators, in most cases rendering them ineffective. We show the effectiveness of the proposed attack in multiple simulations.

Sensitivity Curve Maximization: Attacking Robust Aggregators in Distributed Learning

TL;DR

The paper addresses Byzantine threats in distributed learning and shows that robust aggregators (mean, median, M-estimators, geometric median, etc.) remain vulnerable. It introduces Sensitivity Curve Maximization (SCM) to derive optimal outliers that maximize the disruption of an aggregator, and extends this with aligned, time-coupled attacks (SASCM) to accumulate impact across rounds. The authors formalize the SCM framework, derive attack patterns for 1D/2D aggregators, and demonstrate effectiveness through simulations on linear regression and MNIST datasets, highlighting vulnerabilities of a wide range of schemes. The results indicate that absolute robustness against adversarial timing and direction is difficult to achieve, motivating future defenses that consider temporal alignment and attack directionality.

Abstract

In distributed learning agents aim at collaboratively solving a global learning problem. It becomes more and more likely that individual agents are malicious or faulty with an increasing size of the network. This leads to a degeneration or complete breakdown of the learning process. Classical aggregation schemes are prone to breakdown at small contamination rates, therefore robust aggregation schemes are sought for. While robust aggregation schemes can generally tolerate larger contamination rates, many have been shown to be susceptible to carefully crafted malicious attacks. In this work, we show how the sensitivity curve (SC), a classical tool from robust statistics, can be used to systematically derive optimal attack patterns against arbitrary robust aggregators, in most cases rendering them ineffective. We show the effectiveness of the proposed attack in multiple simulations.

Paper Structure

This paper contains 17 sections, 2 theorems, 49 equations, 16 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Decentralized Learning and Robust Aggregation
  3. Sensitivity Curve
  4. Optimal Attack Pattern
  5. Aggregation Schemes
  6. Attack Schemes
  7. SASCM Attack Examples
  8. Simulations
  9. Choice of Parameters
  10. Linear Regression
  11. MNIST Dataset
  12. Results
  13. Baseline without attack
  14. Analysis of robust aggregators under SASCM attack
  15. Analysis of different alignment strategies
  16. ...and 2 more sections

Key Result

Theorem 1

For a given aggregator $\mathbf{agg}(\cdot)$ and a given sample $\mathcal{Y}$, the sample $\mathcal{Z}^{\star} = \{\boldsymbol{z}^{\star}, \dots, \boldsymbol{z}^{\star}\}$ with $P$ identical samples, which maximizes the distance $\mathrm{dist}(\mathbf{agg}(\mathcal{Y} \cup \mathcal{Z}), \mathbf{agg}

Figures (16)

  • Figure 1: Properties of the for $r=1$ (adapted from Hampel.1986).
  • Figure 2: Multiple rounds of learning at agent $k$ without aligned . The red dashed line indicates the and $\bar{\boldsymbol{w}}_{k,i} = \mathbf{agg}\left(\{\boldsymbol{\phi}_{\ell,i}\}_{\ell \in \mathcal{H}_{k}}\right)$ indicates the honest aggregation result.
  • Figure 3: Overview of for different aggregation schemes for $r=1$.
  • Figure 4: Overview of for different aggregation schemes for $r=1$.
  • Figure 5: Euclidean norm of 2D- for different aggregation schemes. Values larger than 70 are clipped. Mean of underlying data is at $(0,0)$.
  • ...and 11 more figures

Theorems & Definitions (6)

  • Definition 1: Optimal Attack
  • Theorem 1: Sensitivity Curve Maximization
  • proof
  • Theorem 2: Aligned Sensitivity Curve Maximization
  • proof
  • Definition 2: Simplified Aligned Sensitivity Curve Maximization