Asynchronous Secure Federated Learning with Byzantine aggregators

TL;DR

This work tackles privacy-preserving federated learning in asynchronous networks with potentially fully Byzantine aggregators. It introduces aggregator replication, cluster-based two-stage aggregation, LWE-based masking, secret sharing, and differential privacy, all without requiring consensus among aggregators. A verifiable shuffling mechanism and an inclusion scheme ensure fair client participation and mitigate bias, while certification via PVAHSS and threshold signatures validates the aggregation. Empirical results on MNIST show convergence under strong heterogeneity and privacy constraints, with the inclusion mechanism reducing DP noise and improving utility, albeit with a trade-off between the number of aggregators and privacy overhead.

Abstract

Privacy-preserving federated averaging is a central approach for protecting client privacy in federated learning. In this paper, we study this problem in an asynchronous communications setting with malicious aggregators. We propose a new solution to provide federated averaging in this model while protecting the client's data privacy through secure aggregation and differential privacy. Our solution maintains the same performance as the state of the art across all metrics. The main contributions of this paper are threefold. First, unlike existing single- or multi-server solutions, we consider malicious aggregation servers that may manipulate the model to leak clients' data or halt computation. To tolerate this threat, we replicate the aggregators, allowing a fraction of them to be corrupted. Second, we propose a new privacy preservation protocol for protocols in asynchronous communication models with Byzantine aggregators. In this protocol, clients mask their values and add Gaussian noise to their models. In contrast with previous works, we use the replicated servers to unmask the models, while ensuring the liveness of training even if aggregators misbehave. Third, the asynchronous communication model introduces new challenges not present in existing approaches. In such a setting, faster clients may contribute more frequently, potentially reducing their privacy and biasing the training. To address this, we introduce an inclusion mechanism that ensures uniform client participation and balanced privacy budgets. Interestingly, the solution presented in this paper does not rely on agreement between aggregators. Thus, we circumvent the known impossibility of consensus in asynchronous settings where processes might crash. Additionally, this feature increases availability, as a consensus-based algorithm only progresses in periods of low latency.

Figures (4)

• Figure 1: Overview of one training round in the proposed federated learning protocol. (1) Each client receives the aggregated global model from the previous round, together with a proof of correct aggregation. (2) The client verifies the proof and performs a local gradient descent step on its private data. (3) It then produces a privacy-preserving update by masking its model and adding calibrated DP noise, and sends it to its coordinator. Aggregators collect masked updates from their cluster and, once enough contributions are received, include $\rho < k$ clients to mitigate delay bias and reduce the DP noise budget. (4) They aggregate the included masked updates, collaboratively remove the masks on the cluster-level sum, and perform a second inter-cluster aggregation to obtain the global model for the next round. (5) Finally, aggregators jointly produce a threshold signature as a proof of correct aggregation, determine the new client-to-cluster assignment and send the global model to the clients newly assigned. Only one client-to-aggregator and one aggregator-to-client communication occur per round, with no inter-client communication.
• Figure 2: Communication flow for aggregator $a_1$. The aggregator $a_1$ waits for enough updates, then includes $\rho$ out of $k$ updates (1). It sends each share of the $\rho$ updates to the concerned aggregators which aggregate the shares and produce a proof of non-tampering (2). Each aggregator sends back the aggregated shares and proofs to $a_1$, which can verify the proof, reconstruct the mask, unmask the cluster-level aggregated update, and aggregate the proofs of non-tampering (3). Then, $a_1$ sends the proof and the aggregate to the other aggregators, who do the same thing with their own aggregates (4). Once enough intra-cluster aggregates and proofs are received, $a_1$ shares them with the aggregators for them to verify and sign the sum of the intra-cluster aggregate (5). Once $a_1$ receives enough such signatures, it begins a new aggregation round (6).
• Figure 3: Privacy preserving update sharing: Aggregator shares its model with a client (1). The client first trains the received model on local data (2), then adds noise $e_i^\tau$ to the resulting update and masks it with $s_i^{\tau}$ (3). Shares are then produced using a $(n_a, n_a-t_a)$-SS scheme (4). Those shares are then encrypted and sent to their recipient, using $a_1$ as a proxy (5). Finally, aggregators receive their shares, they can decrypt them and begin the aggregation protocol (6).
• Figure 4: (a) Accuracy progression over 300 rounds of aggregation with skewed datasets, where fast clients have classes $'0'$ to $'4'$ and slow clients classes $'5'$ to $'9'$. (b) Number of rounds required to reach 80% accuracy with $\rho=64$ and various privacy budgets. (c) Number of rounds required to reach 80% accuracy with $\epsilon=8$ and various values of $\rho$.

Theorems & Definitions (1)

• Definition 1