Projection-based Prediction-Correction Method for Distributed Consensus Optimization
Han Long
TL;DR
This work tackles distributed consensus optimization under convex constraints by formulating the problem as a variational inequality and introducing an Adaptive Projection-based Prediction-Correction Method (PPCM) inspired by proximal point algorithms. PPCM combines a prediction step and a correction step based on projections, with per-agent preconditioners and adaptive step sizes to ensure contraction and convergence to VI solutions. Theoretical convergence guarantees are established, and extensive distributed experiments on linear least squares, logistic regression, and SVM demonstrate that PPCM achieves substantial speedups (often 4x–12x) while preserving high accuracy, outperforming the benchmark WAGM and built-in solvers. The method’s decentralized structure, simple parameter tuning, and robust performance suggest significant practical impact for large-scale distributed optimization in networks.
Abstract
Within the realm of industrial technology, optimization methods play a pivotal role and are extensively applied across various sectors, including transportation engineering, robotics, and machine learning. With the surge in data volumes, there is an increasing demand for solving large-scale problems, which in turn has spurred the development of distributed optimization methods. These methods rely on the collaborative efforts of numerous dispersed devices to achieve the collective goals of the system. This study focuses on the exploration of distributed consensus optimization problems with convex set constraints within networks. The paper introduces a novel Adaptive Projection Prediction-Correction Method (PPCM), inspired by the proximal point algorithm and incorporating the theory of variational inequalities. As a contraction algorithm with notable convergence performance, PPCM is particularly suited for decentralized network environments. Moreover, the selection of parameters for this method is both straightforward and intuitive, avoiding the complexities of intricate parameter tuning. Comprehensive theoretical analysis and empirical testing have validated the effectiveness of PPCM. When applied to problems such as distributed linear least squares, logistic regression, and support vector machines, PPCM demonstrates superior performance, achieving computation speeds over ten times faster than built-in Python functions while maintaining high precision. In conclusion, this research provides a valuable distributed consensus optimization technique, both theoretically and practically.