Private Aggregation in Hierarchical Wireless Federated Learning with Partial and Full Collusion

TL;DR

A hierarchical wireless system architecture in which the clients are connected to base stations; the base stations are connected to the federator either directly or through relays, and suitable private aggregation schemes tailored for these settings whose communication costs are multiplicative factors away from the derived bounds are introduced.

Abstract

In federated learning, a federator coordinates the training of a model, e.g., a neural network, on privately owned data held by several participating clients. The gradient descent algorithm, a well-known and popular iterative optimization procedure, is run to train the model. Every client computes partial gradients based on their local data and sends them to the federator, which aggregates the results and updates the model. Privacy of the clients' data is a major concern. In fact, it is shown that observing the partial gradients can be enough to reveal the clients' data. Existing literature focuses on private aggregation schemes that tackle the privacy problem in federated learning in settings where all users are connected to each other and to the federator. In this paper, we consider a hierarchical wireless system architecture in which the clients are connected to base stations; the base stations are connected to the federator either directly or through relays. We examine settings with and without relays, and derive fundamental limits on the communication cost under information-theoretic privacy with different collusion assumptions. We introduce suitable private aggregation schemes tailored for these settings whose communication costs are multiplicative factors away from the derived bounds.

Figures (4)

• Figure 1: The wireless architecture for federated learning in the simplified setting. Arrows correspond to point-to-point links. The user equipments of the clients (termed UEs) are connected to the federator and to each other via base stations. Each UE is connected to several base stations. Each client is connected to at least $z_{\text{BS}}\xspace+1$ base stations.
• Figure 2: Example of wireless architecture for federated learning without relays, where UEs are connected to the federator via base stations. Each UE is connected to at least $z_{\text{BS}}\xspace+1$ base stations, with $z_{\text{BS}}\xspace=1$. Solid black links describe aggregated shares. We only depict shares corresponding to evaluation point $\alpha_1$, represented by the leftmost lines of each client. The other two lines describe the shares corresponding to evaluation points $\alpha_2$ and $\alpha_3$ (for client $1$ and $2$). Shares of UEs $1$ and $2$ are aggregated by the three leftmost base stations; shares of UE $3$ are not aggregated. The base stations aggregate the keys, so that the federator cannot decode partial sums of the $g_i$'s.
• Figure 3: Communication cost of our scheme normalized by $n\xspace d\xspace$ as a function of $v_{i\xspace}\xspace$ for $n\xspace=10^4$, $b\xspace=100$ and $z_{\text{BS}}\xspace = 3$.
• Figure 4: Example of generalized hierarchical architecture for federated learning, where UEs are connected to the federator via base stations and relay nodes. Each UE is connected to several base stations. One base station is connected to multiple relay nodes. Each relay (R) is connected to the federator. Solid black links describe aggregated shares. We only depict shares corresponding to evaluation point $\alpha_1$, represented by the leftmost lines of each client. The other two lines describe the shares corresponding to evaluation points $\alpha_2$ and $\alpha_3$. Shares of UEs $1$ and $2$ are aggregated by the first three base stations from the left. The shares of UE $3$ are aggregated by the leftmost three relays, whereas the shares of UE $4$ cannot be aggregated.

Theorems & Definitions (25)

• Definition 1: $({n_\mathrm{s}}, {k_\mathrm{s}}, {z_\mathrm{s}})$ Threshold Secret Sharing
• Proposition 1: Minimum Combined Share Size
• Theorem 1: Lower Bound on the Communication Cost
• proof
• Theorem 2: Privacy of the Scheme
• Remark 1
• Theorem 3: Communication Cost with Partial Collusion
• Remark 2
• Theorem 4: Communication Cost with Full Collusion
• proof : Proof of \ref{['thm:communication_relayed']}