FedMPDD: Communication-Efficient Federated Learning with Privacy Preservation Attributes via Projected Directional Derivative

TL;DR

FedMPDD tackles the dual challenges of communication efficiency and gradient privacy in federated learning by encoding client gradients via multi-projected directional derivatives. By drawing $m$ random projection directions per round and aggregating via a Johnson–Lindenstrauss-based framework, it maintains gradient fidelity with a distortion parameter $\varepsilon$ while reducing uplink messages to $\mathcal{O}(m)$ scalars per client. The method delivers an $O(1/\sqrt{K})$ convergence rate with a tunable privacy–accuracy–communication trade-off controlled by $m$, leveraging the rank-deficient projection to inherently obscure exact gradients against gradient-inversion attacks. Empirical results across standard benchmarks demonstrate substantial communication savings and consistent privacy protection, making FedMPDD a scalable and private alternative to traditional gradient compression in bandwidth-limited FL deployments.

Abstract

This paper introduces \texttt{FedMPDD} (\textbf{Fed}erated Learning via \textbf{M}ulti-\textbf{P}rojected \textbf{D}irectional \textbf{D}erivatives), a novel algorithm that simultaneously optimizes bandwidth utilization and enhances privacy in Federated Learning. The core idea of \texttt{FedMPDD} is to encode each client's high-dimensional gradient by computing its directional derivatives along multiple random vectors. This compresses the gradient into a much smaller message, significantly reducing uplink communication costs from $\mathcal{O}(d)$ to $\mathcal{O}(m)$, where $m \ll d$. The server then decodes the aggregated information by projecting it back onto the same random vectors. Our key insight is that averaging multiple projections overcomes the dimension-dependent convergence limitations of a single projection. We provide a rigorous theoretical analysis, establishing that \texttt{FedMPDD} converges at a rate of $\mathcal{O}(1/\sqrt{K})$, matching the performance of FedSGD. Furthermore, we demonstrate that our method provides some inherent privacy against gradient inversion attacks due to the geometric properties of low-rank projections, offering a tunable privacy-utility trade-off controlled by the number of projections. Extensive experiments on benchmark datasets validate our theory and demonstrates our results.

Figures (19)

• Figure 1: SSIM scores from GIA yu2025gi on LeNet using FedMPDD with $m = 600$ remain consistently low (below $0.04$) over $100$ training epochs, demonstrating that privacy protection is independent of the training stage.
• Figure 2: GIA attack yu2025gi visualization: SSIM scores (left) and reconstructed CIFAR-10 samples (right). LDP with small noise (columns 3--4) and QSGD (column 5) show significant data leakage, while FedMPDD (columns 6--7) demonstrates stronger privacy.
• Figure 3: Training loss and accuracy curves versus communication rounds and number of transmitted bits for the LeNet model on the MNIST dataset (i.i.d.).
• Figure 4: Training loss and accuracy curves versus communication rounds and number of transmitted bits for the logistic model on the MNIST dataset (i.i.d.).
• Figure 5: Training loss and accuracy curves versus communication rounds and number of transmitted bits for the logistic model on the MNIST dataset (non-i.i.d.).

Theorems & Definitions (19)

• Definition 1: projected directional derivative
• Lemma 1: Variance Reduction via Distribution Choice; proof in Appendix \ref{['sec:proofs']}
• Theorem 1: Convergence Bound of FedPDD Algorithm
• Lemma 2: JL Bound for Multi-Projected Directional Derivatives
• Theorem 2: Convergence Bound of FedMPDD Algorithm
• Remark 1
• Remark 2: Computational Cost of FedMPDD
• Remark 3: Communication Reduction and Efficiency in FedMPDD
• Definition 2: Threat model
• Lemma 3: Gradient reconstruction error; proof in Appendix \ref{['sec:proofs']}