Bundle EXTRA for Decentralized Optimization

Abstract

Decentralized primal-dual methods are widely used for solving decentralized optimization problems, but their updates often rely on the potentially crude first-order Taylor approximations of the objective functions, which can limit convergence speed. To overcome this, we replace the first-order Taylor approximation in the primal update of EXTRA, which can be interpreted as a primal-dual method, with a more accurate multi-cut bundle model, resulting in a fully decentralized bundle EXTRA method. The bundle model incorporates historical information to improve the approximation accuracy, potentially leading to faster convergence. Under mild assumptions, we show that a KKT residual converges to zero. Numerical experiments on decentralized least-squares problems demonstrate that, compared to EXTRA, the bundle EXTRA method converges faster and is more robust to step-size choices.

Figures (3)

• Figure 2: Surrogate function $\tilde{f}^k_i$.
• Figure 3: Convergence of bundle EXTRA and EXTRA.
• Figure 4: The parameter robustness of bundle EXTRA and EXTRA methods with step-size $\alpha= 0.003\times 2^t$.

Theorems & Definitions (5)

• Remark 1
• Theorem 1
• proof
• Corollary 1
• proof