Communication-Efficient and Privacy-Adaptable Mechanism -- a Federated Learning Scheme with Convergence Analysis

TL;DR

This work advances federated learning by presenting CEPAM, a unified mechanism that jointly achieves communication efficiency and privacy protection through rejection-sampled universal quantization (RSUQ) and its layered form (LRSUQ). It provides a rigorous convergence analysis under standard FL assumptions and derives DP guarantees for Gaussian and Laplace adaptations, with quantization-induced distortion factored into the optimization analysis. Empirical results on MNIST with a CNN show CEPAM-Gaussian and CEPAM-Laplace outperform baselines by up to about 1 percentage point in test accuracy, while delivering meaningful privacy guarantees. The findings indicate that CEPAM offers a practical pathway to robust, privacy-aware, and communication-efficient FL, with potential extensions to nonconvex settings and personalized FL.

Abstract

Federated learning enables multiple parties to jointly train learning models without sharing their own underlying data, offering a practical pathway to privacy-preserving collaboration under data-governance constraints. Continued study of federated learning is essential to address key challenges in it, including communication efficiency and privacy protection between parties. A recent line of work introduced a novel approach called the Communication-Efficient and Privacy-Adaptable Mechanism (CEPAM), which achieves both objectives simultaneously. CEPAM leverages the rejection-sampled universal quantizer (RSUQ), a randomized vector quantizer whose quantization error is equivalent to a prescribed noise, which can be tuned to customize privacy protection between parties. In this work, we theoretically analyze the privacy guarantees and convergence properties of CEPAM. Moreover, we assess CEPAM's utility performance through experimental evaluations, including convergence profiles compared with other baselines, and accuracy-privacy trade-offs between different parties.

Figures (5)

• Figure 1: Schematic of CEPAM in the FL framework. The upper part demonstrates initialization and aggregation of model updates between server and clients across global communication rounds. The lower part details one global communication round, illustrating the sequence of local updates over local iterations at clients and server-side aggregation at the end of one FL communication round. At the beginning of each communication round, client $k$ receives the global parameter vector $\mathbf{W}_{t}$ from the server and set $\mathbf{W}_{t}^{k}=\mathbf{W}_t$. Next, client $k$ performs $\tau -1$ steps of local SGD, clips the gradient $\nabla F_{k}^{j_{t+\tau}^{k}}(\mathbf{W}_{t+\tau-1}^k)$, and encodes $\tilde{\mathbf{X}}_{t+\tau-1}^{k}$. The server then aggregates $K$ estimated local gradients $\hat{\mathbf{X}}_{t+\tau-1}^{k}$ and perform one step of global SGD according to $\mathbf{W}_{t+\tau} = \mathbf{W}_{t}-\eta_{t+\tau-1}\sum_{k \in \mathcal{K}}p_k\hat{\mathbf{X}}_{t+\tau-1}^{k}$.
• Figure 2: Convergence profile of different FL schemes for Gaussian
• Figure 3: Convergence profile of different FL schemes for Laplace
• Figure 4: Accuracy and Privacy Trade-off of Gaussian
• Figure 5: Accuracy and Privacy Trade-off of Laplace

Theorems & Definitions (14)

• Definition 1
• Definition 2
• Definition 3
• Proposition 4
• Definition 5: $(\epsilon, \delta)$-differential privacy Dwork06DP
• Lemma 6
• proof
• Theorem 7
• Theorem 8
• Proposition 9