Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Attacking Byzantine Robust Aggregation in High Dimensions

Sarthak Choudhary, Aashish Kolluri, Prateek Saxena

TL;DR

The paper addresses the challenge of computing a robust, mean-like statistic in high-dimensional settings under an $\\epsilon$-fraction of Byzantine corruptions. It analyzes strong, dimension-independent aggregation bounds and introduces HiDRA, an untargeted poisoning attack that circumvents practical defenses by exploiting a fundamental computational bottleneck in maximum-variance-direction calculations. The authors prove near-optimal bias bounds of $\\Omega(\\sqrt{\\epsilon d})$ per chunk and demonstrate via extensive experiments that HiDRA can cause drastic drops in model accuracy across standard benchmarks, even when aggregators provide strong theoretical guarantees. The work highlights a critical gap between information-theoretic analyses and practical realizations, suggesting that new defense strategies must address high-dimensional computational challenges to restore robustness in real-world ML training.

Abstract

Training modern neural networks or models typically requires averaging over a sample of high-dimensional vectors. Poisoning attacks can skew or bias the average vectors used to train the model, forcing the model to learn specific patterns or avoid learning anything useful. Byzantine robust aggregation is a principled algorithmic defense against such biasing. Robust aggregators can bound the maximum bias in computing centrality statistics, such as mean, even when some fraction of inputs are arbitrarily corrupted. Designing such aggregators is challenging when dealing with high dimensions. However, the first polynomial-time algorithms with strong theoretical bounds on the bias have recently been proposed. Their bounds are independent of the number of dimensions, promising a conceptual limit on the power of poisoning attacks in their ongoing arms race against defenses. In this paper, we show a new attack called HIDRA on practical realization of strong defenses which subverts their claim of dimension-independent bias. HIDRA highlights a novel computational bottleneck that has not been a concern of prior information-theoretic analysis. Our experimental evaluation shows that our attacks almost completely destroy the model performance, whereas existing attacks with the same goal fail to have much effect. Our findings leave the arms race between poisoning attacks and provable defenses wide open.

Attacking Byzantine Robust Aggregation in High Dimensions

TL;DR

The paper addresses the challenge of computing a robust, mean-like statistic in high-dimensional settings under an -fraction of Byzantine corruptions. It analyzes strong, dimension-independent aggregation bounds and introduces HiDRA, an untargeted poisoning attack that circumvents practical defenses by exploiting a fundamental computational bottleneck in maximum-variance-direction calculations. The authors prove near-optimal bias bounds of per chunk and demonstrate via extensive experiments that HiDRA can cause drastic drops in model accuracy across standard benchmarks, even when aggregators provide strong theoretical guarantees. The work highlights a critical gap between information-theoretic analyses and practical realizations, suggesting that new defense strategies must address high-dimensional computational challenges to restore robustness in real-world ML training.

Abstract

Training modern neural networks or models typically requires averaging over a sample of high-dimensional vectors. Poisoning attacks can skew or bias the average vectors used to train the model, forcing the model to learn specific patterns or avoid learning anything useful. Byzantine robust aggregation is a principled algorithmic defense against such biasing. Robust aggregators can bound the maximum bias in computing centrality statistics, such as mean, even when some fraction of inputs are arbitrarily corrupted. Designing such aggregators is challenging when dealing with high dimensions. However, the first polynomial-time algorithms with strong theoretical bounds on the bias have recently been proposed. Their bounds are independent of the number of dimensions, promising a conceptual limit on the power of poisoning attacks in their ongoing arms race against defenses. In this paper, we show a new attack called HIDRA on practical realization of strong defenses which subverts their claim of dimension-independent bias. HIDRA highlights a novel computational bottleneck that has not been a concern of prior information-theoretic analysis. Our experimental evaluation shows that our attacks almost completely destroy the model performance, whereas existing attacks with the same goal fail to have much effect. Our findings leave the arms race between poisoning attacks and provable defenses wide open.
Paper Structure (55 sections, 12 theorems, 41 equations, 10 figures, 1 table, 7 algorithms)

This paper contains 55 sections, 12 theorems, 41 equations, 10 figures, 1 table, 7 algorithms.

Table of Contents

  1. Introduction
  2. Our work.
  3. Experimental results.
  4. Computational Bottleneck is fundamental.
  5. Our contribution.
  6. Background & Problem Setup
  7. The Problem: Byzantine Robust Aggregation
  8. Application: Untargeted Poisoning in SGD
  9. Aggregation in SGD.
  10. Threat model.
  11. Untargeted model poisoning attacks.
  12. Robust Aggregators
  13. Robust aggregators with weak bounds on bias.
  14. Robust aggregators with strong bounds on bias.
  15. Computational bottleneck.
  16. ...and 40 more sections

Key Result

Theorem 1

HiDRA , as outlined in Algorithm alg:our_attack_algo, will result in a bias of $\Omega(\sqrt{\epsilon d})\cdot ||\Sigma||_2^{\frac{1}{2}}$ against Alg. alg:brahm in the worst case.

Figures (10)

  • Figure 1: Left: Gaussian samples with 0.1 ($\epsilon$) fraction of arbitrarily corrupted data, highlighting the mean shift post-corruption where the dotted circle is the standard deviation ($\sigma$) boundary around the mean. Middle: Trimmed mean by dimension, establishing dimension-wise thresholds to contain corrupted mean within an order of $d$. Right: Strong robust aggregator defenses, applying a single threshold based on variance to restrict corrupted mean to a constant distance.
  • Figure 2: FILTERING diakonikolas2017being
  • Figure 3: HiDRA : Corruptions crafted within variance threshold, yet biasing the mean to an order of $\Omega(\sqrt\epsilon)$
  • Figure 4: Bias vs. # of Dimensions against FILTERING (top) and NO-REGRET (bottom) strong aggregators.
  • Figure 5: Impact of HiDRA on accuracy against FILTERING
  • ...and 5 more figures

Theorems & Definitions (19)

  • Theorem 1
  • proof
  • Lemma 1.1
  • Lemma 1.2
  • Lemma 1.3
  • Lemma 2.1
  • Lemma 2.2
  • Theorem 2
  • proof
  • Lemma 2.1
  • ...and 9 more