Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LiD-FL: Towards List-Decodable Federated Learning

Hong Liu, Liren Shan, Han Bao, Ronghui You, Yuhao Yi, Jiancheng Lv

TL;DR

LiD-FL proposes a list-decodable federated learning framework that preserves a list of global models and uses randomized sampling plus a voting procedure to endure Byzantine adversaries without requiring an honest majority. It provides a convergence theorem for strongly convex and smooth losses under i.i.d. local data and demonstrates robust performance on both convex and non-convex tasks (LR and CNN) across FEMNIST and CIFAR-10 under diverse attacks. Empirically, LiD-FL outperforms standard robust FL baselines in worst-case accuracy, shows stability with larger lists, and benefits from lightweight aggregators, confirming practical robustness in adversarially-rich environments. The work highlights a principled, privacy-friendly path for Byzantine-tolerant FL and motivates further study into non-convex guarantees, data heterogeneity, and advanced aggregation rules.

Abstract

Federated learning is often used in environments with many unverified participants. Therefore, federated learning under adversarial attacks receives significant attention. This paper proposes an algorithmic framework for list-decodable federated learning, where a central server maintains a list of models, with at least one guaranteed to perform well. The framework has no strict restriction on the fraction of honest workers, extending the applicability of Byzantine federated learning to the scenario with more than half adversaries. Under proper assumptions on the loss function, we prove a convergence theorem for our method. Experimental results, including image classification tasks with both convex and non-convex losses, demonstrate that the proposed algorithm can withstand the malicious majority under various attacks.

LiD-FL: Towards List-Decodable Federated Learning

TL;DR

LiD-FL proposes a list-decodable federated learning framework that preserves a list of global models and uses randomized sampling plus a voting procedure to endure Byzantine adversaries without requiring an honest majority. It provides a convergence theorem for strongly convex and smooth losses under i.i.d. local data and demonstrates robust performance on both convex and non-convex tasks (LR and CNN) across FEMNIST and CIFAR-10 under diverse attacks. Empirically, LiD-FL outperforms standard robust FL baselines in worst-case accuracy, shows stability with larger lists, and benefits from lightweight aggregators, confirming practical robustness in adversarially-rich environments. The work highlights a principled, privacy-friendly path for Byzantine-tolerant FL and motivates further study into non-convex guarantees, data heterogeneity, and advanced aggregation rules.

Abstract

Federated learning is often used in environments with many unverified participants. Therefore, federated learning under adversarial attacks receives significant attention. This paper proposes an algorithmic framework for list-decodable federated learning, where a central server maintains a list of models, with at least one guaranteed to perform well. The framework has no strict restriction on the fraction of honest workers, extending the applicability of Byzantine federated learning to the scenario with more than half adversaries. Under proper assumptions on the loss function, we prove a convergence theorem for our method. Experimental results, including image classification tasks with both convex and non-convex losses, demonstrate that the proposed algorithm can withstand the malicious majority under various attacks.
Paper Structure (35 sections, 3 theorems, 17 equations, 5 figures, 9 tables, 3 algorithms)

This paper contains 35 sections, 3 theorems, 17 equations, 5 figures, 9 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Main contributions.
  3. Related work.
  4. Outline.
  5. Preliminaries
  6. Problem Setup
  7. Byzantine-Robust Federated Learning
  8. List-Decodable Federated Learning
  9. Remark I. Guarantee of the voting procedure.
  10. Remark II. Applying aggregation rules in LiD-FL.
  11. Analysis
  12. Experiments
  13. Experimental Setup
  14. Dataset.
  15. Byzantine attacks.
  16. ...and 20 more sections

Key Result

Theorem 4.2

Suppose $\gamma$ fraction of all $m$ clients are uncorrupted. Assume each loss function $f_i$ is $\alpha$-stronly convex and $\beta$-smooth, and the variance of the gradient noise on each client is bounded by $\sigma^2$. With a list size of $q \geq \lfloor 1/\gamma \rfloor$, Algorithm alg:lidFl guar holds with probability at least $1-\delta T$, where $\varepsilon = q (1-\delta)(4\eta\beta + \sigma

Figures (5)

  • Figure 1: Test accuracy of different methods on CIFAR-10 at a Byzantine fraction of $0.6$, with the CNN global model.
  • Figure 2: Test accuracy of various methods on FEMNIST, with the LR global model.
  • Figure 3: Test accuracy of various methods on FEMNIST, with the CNN global model.
  • Figure 4: Performance comparison of LiD-FL with different list size on FEMNIST at a Byzantine fraction of $0.6$.
  • Figure 5: Performance comparison between naive LiD-FL and LiD-FL with aggregators.

Theorems & Definitions (10)

  • Definition 4.1
  • Theorem 4.2
  • Lemma 4.3
  • proof
  • Lemma 4.4
  • proof
  • proof : Proof of Theorem \ref{['thm:main']}
  • Definition A.1
  • Definition A.2
  • Definition A.3