Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Conflict of Robustness and Learning in Collaborative Machine Learning

Mathilde Raynal, Carmela Troncoso

TL;DR

This paper formalizes robustness in Collaborative Machine Learning (CML) and demonstrates a fundamental conflict between robustness and learning. It introduces two robust-aggregator families (distance-based and behavior-based) and a formal game-based framework comprising the robustness-indistinguishability (R-IND) and learning existential unforgeability (L-EUF) games to analyze the trade-offs. The authors prove that distance-based approaches cannot reliably distinguish malicious updates without hindering learning, and behavior-based methods shift the burden to local data representativeness, which in turn limits the benefits of collaboration. Empirical validation on MNIST and real-world-like health and autonomous-driving tasks shows that existing robust aggregators can be manipulated by strategic adversaries, emphasizing the need to rethink robustness design in CML. The work provides a principled foundation for future research toward robust yet learnable collaborative systems, and highlights the limits of current defenses in privacy-preserving, networked learning settings.

Abstract

Collaborative Machine Learning (CML) allows participants to jointly train a machine learning model while keeping their training data private. In many scenarios where CML is seen as the solution to privacy issues, such as health-related applications, safety is also a primary concern. To ensure that CML processes produce models that output correct and reliable decisions \emph{even in the presence of potentially untrusted participants}, researchers propose to use \textit{robust aggregators} to filter out malicious contributions that negatively influence the training process. In this work, we formalize the two prevalent forms of robust aggregators in the literature. We then show that neither can provide the intended protection: either they use distance-based metrics that cannot reliably identify malicious inputs to training; or use metrics based on the behavior of the loss function which create a conflict with the ability of CML participants to learn, i.e., they cannot eliminate the risk of compromise without preventing learning.

On the Conflict of Robustness and Learning in Collaborative Machine Learning

TL;DR

This paper formalizes robustness in Collaborative Machine Learning (CML) and demonstrates a fundamental conflict between robustness and learning. It introduces two robust-aggregator families (distance-based and behavior-based) and a formal game-based framework comprising the robustness-indistinguishability (R-IND) and learning existential unforgeability (L-EUF) games to analyze the trade-offs. The authors prove that distance-based approaches cannot reliably distinguish malicious updates without hindering learning, and behavior-based methods shift the burden to local data representativeness, which in turn limits the benefits of collaboration. Empirical validation on MNIST and real-world-like health and autonomous-driving tasks shows that existing robust aggregators can be manipulated by strategic adversaries, emphasizing the need to rethink robustness design in CML. The work provides a principled foundation for future research toward robust yet learnable collaborative systems, and highlights the limits of current defenses in privacy-preserving, networked learning settings.

Abstract

Collaborative Machine Learning (CML) allows participants to jointly train a machine learning model while keeping their training data private. In many scenarios where CML is seen as the solution to privacy issues, such as health-related applications, safety is also a primary concern. To ensure that CML processes produce models that output correct and reliable decisions \emph{even in the presence of potentially untrusted participants}, researchers propose to use \textit{robust aggregators} to filter out malicious contributions that negatively influence the training process. In this work, we formalize the two prevalent forms of robust aggregators in the literature. We then show that neither can provide the intended protection: either they use distance-based metrics that cannot reliably identify malicious inputs to training; or use metrics based on the behavior of the loss function which create a conflict with the ability of CML participants to learn, i.e., they cannot eliminate the risk of compromise without preventing learning.
Paper Structure (27 sections, 2 theorems, 6 equations, 11 figures)

This paper contains 27 sections, 2 theorems, 6 equations, 11 figures.

Table of Contents

  1. Introduction
  2. Robustness in Collaborative Learning
  3. Systematization of Robust Aggregators
  4. Distance-based Evaluation
  5. Examples of distance-based robust aggregators
  6. Behavior-based Evaluation
  7. Examples of behavior-based robust aggregators
  8. Out of Scope Robust Aggregators
  9. CML goals: Robustness and Learning
  10. Robustness Indistinguishability
  11. Users' Learning Potential
  12. Distance is a bad Proxy for Maliciousness
  13. Space for Learning is space for Manipulation
  14. Distance to win the R-IND game
  15. Advantage to win R-IND Compose during Training
  16. ...and 12 more sections

Key Result

Theorem 5.1

When $R(\theta, \mathcal{D})$ is distance-based, there exist a equivalence reduction from the R-IND game to the L-EUF game, meaning that if, for a participant $\mathtt{P}$ and a threshold $\delta^*$, $Adv_\mathtt{P}^\text{R-IND}(\delta^*; S_\mathtt{P}, \mathcal{D})=1$, then $Adv_\mathtt{P}^\text{L-E

Figures (11)

  • Figure 1: R-IND Game
  • Figure 2: L-EUF Game
  • Figure 3: Accuracy (top) and L2 distance to adversarial target (bottom) under the state-override attack on the MNIST task (non-IID setup, left; IID setup, right).
  • Figure 4: Accuracy during benign training (top) and L2 distance to the adversarial target under the state-override attack (bottom) on the MNIST task for $\delta=0.5$ (left), $\delta =1$ (middle), and $\delta=5$ (right).
  • Figure 5: Evolution of $\delta$ during benign training (baseline) and under the dissensus attack.
  • ...and 6 more figures

Theorems & Definitions (6)

  • Definition 3.1: Distance-based robustness
  • Definition 3.2: Behavior-based robustness
  • Theorem 5.1
  • proof : Proof Sketch
  • Theorem 6.1
  • proof : Proof Sketch