Lower Bounds and Optimal Algorithms for Non-Smooth Convex Decentralized Optimization over Time-Varying Networks
Dmitry Kovalev, Ekaterina Borodich, Alexander Gasnikov, Dmitrii Feoktistov
TL;DR
The paper addresses non-smooth convex decentralized optimization over time-varying networks by formulating a precise objective $p(x)=\frac{1}{n}\sum_{i=1}^n f_i(x)+\frac{r}{2}\|x\|^2$ and introducing a saddle-point reformulation to tackle consensus constraints. It proves the first lower bounds on both decentralized communication and subgradient computation complexities in this setting and presents an optimal algorithm that matches these bounds. The algorithm leverages a Forward-Backward scheme with acceleration and error-feedback within a saddle-point framework, delivering complexity guarantees $K=O(\frac{\chi M}{\sqrt{r\epsilon}})$ and $KT=O(\frac{M^2}{r\epsilon})$ for $r>0$, and $K=O(\frac{\chi MR}{\epsilon})$, $KT=O(\frac{M^2R^2}{\epsilon^2})$ for $r=0$, thus outperforming prior time-varying-network methods without resorting to smoothing. These results close a major gap in decentralized optimization theory and inform the design of efficient distributed solvers under unreliable or dynamic network conditions.
Abstract
We consider the task of minimizing the sum of convex functions stored in a decentralized manner across the nodes of a communication network. This problem is relatively well-studied in the scenario when the objective functions are smooth, or the links of the network are fixed in time, or both. In particular, lower bounds on the number of decentralized communications and (sub)gradient computations required to solve the problem have been established, along with matching optimal algorithms. However, the remaining and most challenging setting of non-smooth decentralized optimization over time-varying networks is largely underexplored, as neither lower bounds nor optimal algorithms are known in the literature. We resolve this fundamental gap with the following contributions: (i) we establish the first lower bounds on the communication and subgradient computation complexities of solving non-smooth convex decentralized optimization problems over time-varying networks; (ii) we develop the first optimal algorithm that matches these lower bounds and offers substantially improved theoretical performance compared to the existing state of the art.