Privacy-Preserving Distributed Online Mirror Descent for Nonconvex Optimization
Yingjie Zhou, Tao Li
TL;DR
This work tackles privacy-preserving distributed online nonconvex optimization over time-varying directed graphs. It integrates the Laplace differential privacy mechanism with a distributed mirror descent framework to protect node data while tracking a stationary point of time-varying nonconvex objectives on a compact set $\Omega$. The authors prove per-iteration $\epsilon$-differential privacy by bounding the algorithm's sensitivity and provide a sublinear regret bound of order $O(\sqrt{T})$, showing that privacy protections do not degrade the convergence rate's order. Numerical simulations on a distributed localization problem validate the theoretical results and illustrate the privacy-utility trade-off under different privacy budgets and Bregman divergences.
Abstract
We investigate the distributed online nonconvex optimization problem with differential privacy over time-varying networks. Each node minimizes the sum of several nonconvex functions while preserving the node's differential privacy. We propose a privacy-preserving distributed online mirror descent algorithm for nonconvex optimization, which uses the mirror descent to update decision variables and the Laplace differential privacy mechanism to protect privacy. Unlike the existing works, the proposed algorithm allows the cost functions to be nonconvex, which is more applicable. Based upon these, we prove that if the communication network is $B$-strongly connected and the constraint set is compact, then by choosing the step size properly, the algorithm guarantees $ε$-differential privacy at each time. Furthermore, we prove that if the local cost functions are $β$-smooth, then the regret over time horizon $T$ grows sublinearly while preserving differential privacy, with an upper bound $O(\sqrt{T})$. Finally, the effectiveness of the algorithm is demonstrated through numerical simulations.