NPGA: A Unified Algorithmic Framework for Decentralized Constraint-Coupled Optimization
Jingwang Li, Housheng Su
TL;DR
This work tackles decentralized constraint-coupled optimization by introducing NPGA, a nested primal-dual gradient algorithm that achieves linear convergence under the weakest known conditions and acts as a unified framework via configurable network matrices. NPGA recasts the problem into a dual form and solves it in a decentralized fashion using an inexact dual gradient, yielding multiple algorithm variants (e.g., DCPA, DCDA) as special cases. Theoretical results establish R-linear convergence under two regimes: (i) $g_i=0$ with full row-rank $A$, and (ii) $h$ being $l_h$-smooth, with explicit step-size bounds that ensure contraction. Empirical studies in vertical federated learning on ridge, logistic, and elastic-net regression demonstrate NPGA's superior convergence speed and reduced communication compared to existing methods, validating its practical value for distributed learning and optimization.
Abstract
This work focuses on a class of general decentralized constraint-coupled optimization problems. We propose a novel nested primal-dual gradient algorithm (NPGA), which can achieve linear convergence under the weakest known condition, and its theoretical convergence rate surpasses all known results. More importantly, NPGA serves not only as an algorithm but also as a unified algorithmic framework, encompassing various existing algorithms as special cases. By designing different network matrices, we can derive numerous versions of NPGA and analyze their convergences by leveraging the convergence results of NPGA conveniently, thereby enabling the design of more efficient algorithms. Finally, we conduct numerical experiments to compare the convergence rates of NPGA and existing algorithms, providing empirical evidence for the superior performance of NPGA.