Distributed Optimization via Energy Conservation Laws in Dilated Coordinates

TL;DR

This work introduces a unified energy-conservation framework for distributed convex optimization via dilated coordinates, enabling explicit convergence rates tied to the time-dilation factor. It derives a distributed accelerated gradient flow (AGM-ODE) with a continuous-time rate of $O(1/t^{2-\beta})$ for any small $\beta>0$, and designs a rate-matching discretization using a semi-second-order symplectic Euler scheme that attains $O(1/k^{2-\beta})$ in iterations. A detailed discretization analysis provides step-size rules ensuring rate preservation, supported by Lyapunov-like arguments in a dilated space. Empirical results on large-scale, ill-conditioned problems (e.g., MNIST logistic regression across a ring network) demonstrate accelerated convergence relative to state-of-the-art distributed methods, highlighting practical impact for private-data multi-agent optimization. The framework offers a general tool for analyzing and designing distributed optimization algorithms with provable, tunable acceleration properties.

Abstract

Optimizing problems in a distributed manner is critical for systems involving multiple agents with private data. Despite substantial interest, a unified method for analyzing the convergence rates of distributed optimization algorithms is lacking. This paper introduces an energy conservation approach for analyzing continuous-time dynamical systems in dilated coordinates. Instead of directly analyzing dynamics in the original coordinate system, we establish a conserved quantity, akin to physical energy, in the dilated coordinate system. Consequently, convergence rates can be explicitly expressed in terms of the inverse time-dilation factor. Leveraging this generalized approach, we formulate a novel second-order distributed accelerated gradient flow with a convergence rate of $O\left(1/t^{2-ε}\right)$ in time $t$ for $ε>0$. We then employ a semi second-order symplectic Euler discretization to derive a rate-matching algorithm with a convergence rate of $O\left(1/k^{2-ε}\right)$ in $k$ iterations. To the best of our knowledge, this represents the most favorable convergence rate for any distributed optimization algorithm designed for smooth convex optimization. Its accelerated convergence behavior is benchmarked against various state-of-the-art distributed optimization algorithms on practical, large-scale problems.

Figures (2)

• Figure 1: (a) $\left\lVert{\tilde{{\cal L}}} X_k\right\rVert$, (b) $\left\lVert\nabla F(X_k)\right\rVert$, and (c) $F(X_k)$ of the discretized distributed AGM algorithm \ref{['eqn:xplus0_def']}-\ref{['eqn:xk_def']} with different values of $\beta$ for solving the binary logistic regression problem on MNIST dataset.
• Figure 2: Comparison of (a) $\left\lVert{\tilde{{\cal L}}} X_k\right\rVert$, (b) $\left\lVert\nabla F(X_k)\right\rVert$, and (c) $F(X_k)$ of different distributed optimization algorithms for solving the binary logistic regression problem on MNIST dataset.

Theorems & Definitions (6)

• Definition 1
• Definition 2
• Theorem 1
• proof
• Theorem 2
• proof