Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Capacity of Hierarchical Secure Coded Gradient Aggregation with Straggling Communication Links

Qinyi Lu, Jiale Cheng, Wei Kang, Nan Liu

TL;DR

This work addresses secure gradient aggregation in a hierarchical, user–helper–master network with straggling links. It introduces the hierarchical secure coded gradient aggregation (HCGA) problem under resiliency $N_r$ and collusion parameter $T$, and develops a Vandermonde-based encoding scheme together with an extended inter-helper sharing mechanism. A matching information-theoretic converse shows that secure aggregation is infeasible for $N_r\le T$, and for $N_r>T$ the optimal per-link rates are $R_X=R_Y=\tfrac{1}{N_r-T}$, providing a clean capacity-like result. The results have practical implications for designing robust, privacy-preserving distributed learning systems over multi-hop, heterogeneous networks where stragglers and collusion are key concerns.

Abstract

The growing privacy concerns in distributed learning have led to the widespread adoption of secure aggregation techniques in distributed machine learning systems, such as federated learning. Motivated by a coded gradient aggregation problem in a user-helper-master hierarchical network setting with straggling communication links, we formulate a new secure hierarchical coded gradient aggregation problem. In our setting, \( K \) users communicate with the master through an intermediate layer of \( N \) helpers, who can communicate with each other. With a resiliency threshold of \( N_r \) for straggling communication links, and at most \( T \) colluding helpers and any number of colluding users, the master aims to recover the sum of all users' gradients while remaining unaware of any individual gradient that exceeds the expected sum. In addition, helpers cannot infer more about users' gradients than what is already known by the colluding users. We propose an achievable scheme where users' upload messages are based on a globally known Vandermonde matrix, and helper communication is facilitated using an extended Vandermonde matrix with special structural properties. A matching converse bound is also derived, establishing the optimal result for this hierarchical coded gradient aggregation problem.

Capacity of Hierarchical Secure Coded Gradient Aggregation with Straggling Communication Links

TL;DR

This work addresses secure gradient aggregation in a hierarchical, user–helper–master network with straggling links. It introduces the hierarchical secure coded gradient aggregation (HCGA) problem under resiliency and collusion parameter , and develops a Vandermonde-based encoding scheme together with an extended inter-helper sharing mechanism. A matching information-theoretic converse shows that secure aggregation is infeasible for , and for the optimal per-link rates are , providing a clean capacity-like result. The results have practical implications for designing robust, privacy-preserving distributed learning systems over multi-hop, heterogeneous networks where stragglers and collusion are key concerns.

Abstract

The growing privacy concerns in distributed learning have led to the widespread adoption of secure aggregation techniques in distributed machine learning systems, such as federated learning. Motivated by a coded gradient aggregation problem in a user-helper-master hierarchical network setting with straggling communication links, we formulate a new secure hierarchical coded gradient aggregation problem. In our setting, users communicate with the master through an intermediate layer of helpers, who can communicate with each other. With a resiliency threshold of for straggling communication links, and at most colluding helpers and any number of colluding users, the master aims to recover the sum of all users' gradients while remaining unaware of any individual gradient that exceeds the expected sum. In addition, helpers cannot infer more about users' gradients than what is already known by the colluding users. We propose an achievable scheme where users' upload messages are based on a globally known Vandermonde matrix, and helper communication is facilitated using an extended Vandermonde matrix with special structural properties. A matching converse bound is also derived, establishing the optimal result for this hierarchical coded gradient aggregation problem.

Paper Structure

This paper contains 18 sections, 4 theorems, 63 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Paper Organization
  3. Notations
  4. System Model
  5. Users Uploading Phase
  6. Sharing and Computation Phase
  7. Reconstruction Phase
  8. Main Result
  9. Proof of Theorem \ref{['Theo1']}: Achievability when $N_r > T$
  10. Motivating Example: $K=2,N=4,N_r = 3,T=1$
  11. General Proof for Arbitrary $K,N,N_r,T$
  12. Proof of Lemma 1
  13. Proof of Lemma 2
  14. Converse Proof
  15. $N_r \le T$: Proof of $\mathcal{R}^* = \emptyset$
  16. ...and 3 more sections

Key Result

Theorem 1

For the hierarchical secure coded gradient aggregation problem with $K$ users, $N$ helpers, a resiliency threshold of $N_r$, at most $T$ colluding helpers, and any number of colluding users, the optimal rate region is given by

Figures (1)

  • Figure 1: Hierarchical distributed learning system with straggling communication links.

Theorems & Definitions (9)

  • Theorem 1
  • Remark 1
  • Remark 2
  • Remark 3
  • Remark 4
  • Lemma 1
  • Lemma 2
  • Remark 5
  • Lemma 3