Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decentralized Distributed Optimization for Saddle Point Problems

Alexander Rogozin, Aleksandr Beznosikov, Darina Dvinskikh, Dmitry Kovalev, Pavel Dvurechensky, Alexander Gasnikov

TL;DR

This work addresses distributed convex-concave saddle-point problems on networks by introducing a decentralized Mirror-Prox method that enforces consensus via dual multipliers. It provides non-asymptotic convergence guarantees with explicit dependence on the network condition number $\chi$, along with sharp lower bounds that establish optimality in the Euclidean proximal setting. The framework is applied to Wasserstein barycenters by reformulating WB as a saddle-point problem and demonstrating both theoretical guarantees and extensive numerical experiments across diverse network topologies. The results show that the proposed approach attains high-precision WB without the instability issues seen in entropy-regularized methods, highlighting its practical impact for distributed optimal transport tasks.

Abstract

We consider distributed convex-concave saddle point problems over arbitrary connected undirected networks and propose a decentralized distributed algorithm for their solution. The local functions distributed across the nodes are assumed to have global and local groups of variables. For the proposed algorithm we prove non-asymptotic convergence rate estimates with explicit dependence on the network characteristics. To supplement the convergence rate analysis, we propose lower bounds for strongly-convex-strongly-concave and convex-concave saddle-point problems over arbitrary connected undirected networks. We illustrate the considered problem setting by a particular application to distributed calculation of non-regularized Wasserstein barycenters.

Decentralized Distributed Optimization for Saddle Point Problems

TL;DR

This work addresses distributed convex-concave saddle-point problems on networks by introducing a decentralized Mirror-Prox method that enforces consensus via dual multipliers. It provides non-asymptotic convergence guarantees with explicit dependence on the network condition number , along with sharp lower bounds that establish optimality in the Euclidean proximal setting. The framework is applied to Wasserstein barycenters by reformulating WB as a saddle-point problem and demonstrating both theoretical guarantees and extensive numerical experiments across diverse network topologies. The results show that the proposed approach attains high-precision WB without the instability issues seen in entropy-regularized methods, highlighting its practical impact for distributed optimal transport tasks.

Abstract

We consider distributed convex-concave saddle point problems over arbitrary connected undirected networks and propose a decentralized distributed algorithm for their solution. The local functions distributed across the nodes are assumed to have global and local groups of variables. For the proposed algorithm we prove non-asymptotic convergence rate estimates with explicit dependence on the network characteristics. To supplement the convergence rate analysis, we propose lower bounds for strongly-convex-strongly-concave and convex-concave saddle-point problems over arbitrary connected undirected networks. We illustrate the considered problem setting by a particular application to distributed calculation of non-regularized Wasserstein barycenters.

Paper Structure

This paper contains 29 sections, 16 theorems, 72 equations, 5 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Problem statement
  3. Decentralized setup and communication matrix
  4. Distributed problem reformulation
  5. Distributed algorithm for saddle-point problems
  6. Algorithm
  7. Localizing the solution and smoothness assumptions
  8. Main result
  9. Discussion for the Euclidean case
  10. Lower bounds for distributed saddle point problems
  11. Application to the Wasserstein barycenters
  12. Saddle point problem formulation
  13. Convergence guarantee
  14. Discussion of the convergence results.
  15. Numerical experiments
  16. ...and 14 more sections

Key Result

Theorem 2.4

Introduce where $\bar{\alpha}, \bar{\beta}\in (1, +\infty) \cup \left\{ +\infty \right\}$. Problem eq:problem_initial is equivalent to

Figures (5)

  • Figure 1: Convergence of the DMP algorithm on different network architectures.
  • Figure 2: Numerical instability of entropy-regularized based methods
  • Figure 3: Barycenters of letter 'B' found by the DMP (left), the IBP (middle) and the ADCWB (right).
  • Figure :
  • Figure :

Theorems & Definitions (18)

  • Theorem 2.4
  • Lemma 2.5
  • Corollary 2.6
  • Lemma 3.1
  • Definition 3.2
  • Lemma 3.4
  • Theorem 3.5
  • Corollary 3.6
  • Corollary 3.7
  • Remark 1
  • ...and 8 more