Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rethinking Byzantine Robustness in Federated Recommendation from Sparse Aggregation Perspective

Zhongjian Zhang, Mengmei Zhang, Xiao Wang, Lingjuan Lyu, Bo Yan, Junping Du, Chuan Shi

TL;DR

The paper tackles the Byzantine robustness of federated recommender systems under sparse aggregation, where each item embedding is updated only by its interacting users. It introduces Spattack, a family of attacks designed to exploit the per-item sparse updates, with four variants based on attacker knowledge and capability. The authors provide a theoretical analysis showing that tailed items in a power-law degree distribution are particularly vulnerable, and demonstrate through extensive experiments on ML100K, ML1M, and Steam that Spattack can prevent convergence and defeat common defenses, with strong transferability to FR models like FedGNN. The work highlights a critical privacy-preserving yet security-challenging aspect of FR and calls for robust tail-item aggregation strategies to enhance real-world applicability.

Abstract

To preserve user privacy in recommender systems, federated recommendation (FR) based on federated learning (FL) emerges, keeping the personal data on the local client and updating a model collaboratively. Unlike FL, FR has a unique sparse aggregation mechanism, where the embedding of each item is updated by only partial clients, instead of full clients in a dense aggregation of general FL. Recently, as an essential principle of FL, model security has received increasing attention, especially for Byzantine attacks, where malicious clients can send arbitrary updates. The problem of exploring the Byzantine robustness of FR is particularly critical since in the domains applying FR, e.g., e-commerce, malicious clients can be injected easily by registering new accounts. However, existing Byzantine works neglect the unique sparse aggregation of FR, making them unsuitable for our problem. Thus, we make the first effort to investigate Byzantine attacks on FR from the perspective of sparse aggregation, which is non-trivial: it is not clear how to define Byzantine robustness under sparse aggregations and design Byzantine attacks under limited knowledge/capability. In this paper, we reformulate the Byzantine robustness under sparse aggregation by defining the aggregation for a single item as the smallest execution unit. Then we propose a family of effective attack strategies, named Spattack, which exploit the vulnerability in sparse aggregation and are categorized along the adversary's knowledge and capability. Extensive experimental results demonstrate that Spattack can effectively prevent convergence and even break down defenses under a few malicious clients, raising alarms for securing FR systems.

Rethinking Byzantine Robustness in Federated Recommendation from Sparse Aggregation Perspective

TL;DR

The paper tackles the Byzantine robustness of federated recommender systems under sparse aggregation, where each item embedding is updated only by its interacting users. It introduces Spattack, a family of attacks designed to exploit the per-item sparse updates, with four variants based on attacker knowledge and capability. The authors provide a theoretical analysis showing that tailed items in a power-law degree distribution are particularly vulnerable, and demonstrate through extensive experiments on ML100K, ML1M, and Steam that Spattack can prevent convergence and defeat common defenses, with strong transferability to FR models like FedGNN. The work highlights a critical privacy-preserving yet security-challenging aspect of FR and calls for robust tail-item aggregation strategies to enhance real-world applicability.

Abstract

To preserve user privacy in recommender systems, federated recommendation (FR) based on federated learning (FL) emerges, keeping the personal data on the local client and updating a model collaboratively. Unlike FL, FR has a unique sparse aggregation mechanism, where the embedding of each item is updated by only partial clients, instead of full clients in a dense aggregation of general FL. Recently, as an essential principle of FL, model security has received increasing attention, especially for Byzantine attacks, where malicious clients can send arbitrary updates. The problem of exploring the Byzantine robustness of FR is particularly critical since in the domains applying FR, e.g., e-commerce, malicious clients can be injected easily by registering new accounts. However, existing Byzantine works neglect the unique sparse aggregation of FR, making them unsuitable for our problem. Thus, we make the first effort to investigate Byzantine attacks on FR from the perspective of sparse aggregation, which is non-trivial: it is not clear how to define Byzantine robustness under sparse aggregations and design Byzantine attacks under limited knowledge/capability. In this paper, we reformulate the Byzantine robustness under sparse aggregation by defining the aggregation for a single item as the smallest execution unit. Then we propose a family of effective attack strategies, named Spattack, which exploit the vulnerability in sparse aggregation and are categorized along the adversary's knowledge and capability. Extensive experimental results demonstrate that Spattack can effectively prevent convergence and even break down defenses under a few malicious clients, raising alarms for securing FR systems.
Paper Structure (33 sections, 2 theorems, 7 equations, 27 figures, 10 tables)

This paper contains 33 sections, 2 theorems, 7 equations, 27 figures, 10 tables.

Table of Contents

  1. Introduction
  2. Background and Preliminary
  3. Centralized Recommendation
  4. Federated Recommendation
  5. Byzantine Attack and Defense
  6. Methodology
  7. Problem Definition
  8. Byzantine Robustness Analysis
  9. Spattack: Byzantine Attack Strategies
  10. Experiment
  11. Experimental Setup
  12. Attack Performance Evaluation (RQ1)
  13. Attack Effectiveness under Defense (RQ2)
  14. Transferability of Attack (RQ3)
  15. Hyperparameter Analysis (RQ4)
  16. ...and 18 more sections

Key Result

Proposition 1

For each item $v_j$, let $\{\nabla \boldsymbol{v}^{i,t}_j| \text{ user } i \in \mathcal{U}_{v_j}\}$ be the set of benign gradient vectors at epoch $t$. Consider a Mean aggregator averaging updates for each element. Let $\nabla \boldsymbol{\tilde{v}}_j$ be a malicious update with arbitrary values in

Figures (27)

  • Figure 1: Comparisons between dense aggregation of general FL and the unique sparse aggregation of FR.
  • Figure 2: Analysis of Byzantine robustness in general FL and FR. Under Byzantine attacks, a robust aggregator can filter outliers in general FL, but fails to defend for tailed items' embedding.
  • Figure 3: Performance of Spattack against multiple defense strategies under different ratios of Byzantine clients.
  • Figure 7: Performance of Spattack starting with different epochs on Steam dataset.
  • Figure 8: Attack performance under differential privacy.
  • ...and 22 more figures

Theorems & Definitions (5)

  • Definition 1
  • Proposition 1
  • proof
  • Proposition 2
  • proof