Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ADI: Adversarial Dominating Inputs in Vertical Federated Learning Systems

Qi Pang, Yuanyuan Yuan, Shuai Wang, Wenting Zheng

TL;DR

This study reveals new VFL attack opportunities, promoting the identification of unknown threats before breaches and building more secure VFL systems, and conducts an in-depth study on the influence of critical parameters and settings in synthesizing ADIs.

Abstract

Vertical federated learning (VFL) system has recently become prominent as a concept to process data distributed across many individual sources without the need to centralize it. Multiple participants collaboratively train models based on their local data in a privacy-aware manner. To date, VFL has become a de facto solution to securely learn a model among organizations, allowing knowledge to be shared without compromising privacy of any individuals. Despite the prosperous development of VFL systems, we find that certain inputs of a participant, named adversarial dominating inputs (ADIs), can dominate the joint inference towards the direction of the adversary's will and force other (victim) participants to make negligible contributions, losing rewards that are usually offered regarding the importance of their contributions in federated learning scenarios. We conduct a systematic study on ADIs by first proving their existence in typical VFL systems. We then propose gradient-based methods to synthesize ADIs of various formats and exploit common VFL systems. We further launch greybox fuzz testing, guided by the saliency score of ``victim'' participants, to perturb adversary-controlled inputs and systematically explore the VFL attack surface in a privacy-preserving manner. We conduct an in-depth study on the influence of critical parameters and settings in synthesizing ADIs. Our study reveals new VFL attack opportunities, promoting the identification of unknown threats before breaches and building more secure VFL systems.

ADI: Adversarial Dominating Inputs in Vertical Federated Learning Systems

TL;DR

This study reveals new VFL attack opportunities, promoting the identification of unknown threats before breaches and building more secure VFL systems, and conducts an in-depth study on the influence of critical parameters and settings in synthesizing ADIs.

Abstract

Vertical federated learning (VFL) system has recently become prominent as a concept to process data distributed across many individual sources without the need to centralize it. Multiple participants collaboratively train models based on their local data in a privacy-aware manner. To date, VFL has become a de facto solution to securely learn a model among organizations, allowing knowledge to be shared without compromising privacy of any individuals. Despite the prosperous development of VFL systems, we find that certain inputs of a participant, named adversarial dominating inputs (ADIs), can dominate the joint inference towards the direction of the adversary's will and force other (victim) participants to make negligible contributions, losing rewards that are usually offered regarding the importance of their contributions in federated learning scenarios. We conduct a systematic study on ADIs by first proving their existence in typical VFL systems. We then propose gradient-based methods to synthesize ADIs of various formats and exploit common VFL systems. We further launch greybox fuzz testing, guided by the saliency score of ``victim'' participants, to perturb adversary-controlled inputs and systematically explore the VFL attack surface in a privacy-preserving manner. We conduct an in-depth study on the influence of critical parameters and settings in synthesizing ADIs. Our study reveals new VFL attack opportunities, promoting the identification of unknown threats before breaches and building more secure VFL systems.
Paper Structure (34 sections, 3 theorems, 32 equations, 17 figures, 15 tables, 2 algorithms)

This paper contains 34 sections, 3 theorems, 32 equations, 17 figures, 15 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Related Work and Problem Statement
  4. FL Security and ADI Positioning
  5. Threat Model
  6. Proof on the Existence of ADIs in VFL
  7. Extension & Practical Consideration
  8. Blackbox ADI Synthesis
  9. A Schematic View of ADI Generation
  10. Gradient Estimation in the Blackbox Setting
  11. Uncover ADIs with Greybox Fuzzing
  12. Design of Greybox Fuzz Testing
  13. Implementation & Evaluation Setup
  14. Evaluation
  15. Gradient-Based ADI Synthesis
  16. ...and 19 more sections

Key Result

Corollary 1

With fixed input $X_{\mathcal{A}}^*$ and varying input $X_{\mathcal{B}}$ following any distribution $D_{\mathcal{B}}$ whose density function is $p(X_{\mathcal{B}})$. For any $\epsilon$, there exists $X_{\mathcal{A}}^*$ that the output variance of the VFL system (SplitNN, HeteroLR, and their extensio

Figures (17)

  • Figure 1: ADIs from FinTech dominate the VFL predictions.
  • Figure 2: Architecture of a VFL system.
  • Figure 3: Architecture of SplitNN.
  • Figure 4: A schematic view of ADI generation.
  • Figure 5: Saliency-aware mutation.
  • ...and 12 more figures

Theorems & Definitions (8)

  • Definition 1: ADIs
  • Corollary 1: Variance Bound of VFL systems
  • Definition 2: Practical Assessment of ADIs
  • Corollary 2: Variance of HeteroLR
  • proof
  • Corollary 3: Variance of SplitNN
  • proof
  • proof