Private Networked Federated Learning for Nonsmooth Objectives

TL;DR

The developed private networked federated learning algorithm has a competitive privacy accuracy trade-off and handles nonsmooth and non-strongly convex problems and handles nonsmooth and non-strongly convex problems.

Abstract

This paper develops a networked federated learning algorithm to solve nonsmooth objective functions. To guarantee the confidentiality of the participants with respect to each other and potential eavesdroppers, we use the zero-concentrated differential privacy notion (zCDP). Privacy is achieved by perturbing the outcome of the computation at each client with a variance-decreasing Gaussian noise. ZCDP allows for better accuracy than the conventional $(ε, δ)$-DP and stronger guarantees than the more recent Rényi-DP by assuming adversaries aggregate all the exchanged messages. The proposed algorithm relies on the distributed Alternating Direction Method of Multipliers (ADMM) and uses the approximation of the augmented Lagrangian to handle nonsmooth objective functions. The developed private networked federated learning algorithm has a competitive privacy accuracy trade-off and handles nonsmooth and non-strongly convex problems. We provide complete theoretical proof for the privacy guarantees and the algorithm's convergence to the exact solution. We also prove under additional assumptions that the algorithm converges in $O(1/n)$ ADMM iterations. Finally, we observe the performance of the algorithm in a series of numerical simulations.

Figures (3)

• Figure 1: Learning curves (left) and privacy-accuracy trade-off (right) on the elastic net problem.
• Figure 2: Learning curve on the least absolute deviation problem.
• Figure 3: Learning curves (left) and privacy-accuracy trade-off (right) on the smooth ridge regression problem.

Theorems & Definitions (12)

• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof