Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Block Decomposable Methods for Large-Scale Optimization Problems

Leandro Farias Maia

TL;DR

The work tackles large-scale, block-structured optimization by developing adaptive proximal ADMM (A-ADMM) and inexact block-proximal mappings. It provides a parameter-free, inexact proximal framework that achieves stationary solutions with favorable iteration complexity ${\cal O}(B \max\{\rho^{-3}, \eta^{-3}\})$ and offers both static and adaptive prox-step variants, including AB-IPP for adaptivity. Convergence guarantees are extended to randomized BCD methods over Hölder-smooth functions, with matching rates across nonconvex, convex, and strongly convex regimes. Numerical experiments on distributed quadratic programs and nonconvex QPs with box constraints illustrate that A-ADMM substantially outperforms state-of-the-art proximal ADMM variants and related baselines in both iterations and time. Overall, the dissertation advances practical, scalable optimization by unifying adaptive proximal updates, inexact subproblem solutions, and randomized block schemes under rigorous convergence and complexity analyses.

Abstract

This dissertation explores block decomposable methods for large-scale optimization problems. It focuses on alternating direction method of multipliers (ADMM) schemes and block coordinate descent (BCD) methods. Specifically, it introduces a new proximal ADMM algorithm and proposes two BCD methods. The first part of the research presents a new proximal ADMM algorithm. This method is adaptive to all problem parameters and solves the proximal augmented Lagrangian (AL) subproblem inexactly. This adaptiveness facilitates the highly efficient application of the algorithm to a broad swath of practical problems. The inexact solution of the proximal AL subproblem overcomes many key challenges in the practical applications of ADMM. The resultant algorithm obtains an approximate solution of an optimization problem in a number of iterations that matches the state-of-the-art complexity for the class of proximal ADMM schemes. The second part of the research focuses on an inexact proximal mapping for the class of block proximal gradient methods. Key properties of this operator is established, facilitating the derivation of convergence rates for the proposed algorithm. Under two error decreases conditions, the algorithm matches the convergence rate of its exactly computed counterpart. Numerical results demonstrate the superior performance of the algorithm under a dynamic error regime over a fixed one. The dissertation concludes by providing convergence guarantees for the randomized BCD method applied to a broad class of functions, known as Hölder smooth functions. Convergence rates are derived for non-convex, convex, and strongly convex functions. These convergence rates match those furnished in the existing literature for the Lipschtiz smooth setting.

Block Decomposable Methods for Large-Scale Optimization Problems

TL;DR

The work tackles large-scale, block-structured optimization by developing adaptive proximal ADMM (A-ADMM) and inexact block-proximal mappings. It provides a parameter-free, inexact proximal framework that achieves stationary solutions with favorable iteration complexity and offers both static and adaptive prox-step variants, including AB-IPP for adaptivity. Convergence guarantees are extended to randomized BCD methods over Hölder-smooth functions, with matching rates across nonconvex, convex, and strongly convex regimes. Numerical experiments on distributed quadratic programs and nonconvex QPs with box constraints illustrate that A-ADMM substantially outperforms state-of-the-art proximal ADMM variants and related baselines in both iterations and time. Overall, the dissertation advances practical, scalable optimization by unifying adaptive proximal updates, inexact subproblem solutions, and randomized block schemes under rigorous convergence and complexity analyses.

Abstract

This dissertation explores block decomposable methods for large-scale optimization problems. It focuses on alternating direction method of multipliers (ADMM) schemes and block coordinate descent (BCD) methods. Specifically, it introduces a new proximal ADMM algorithm and proposes two BCD methods. The first part of the research presents a new proximal ADMM algorithm. This method is adaptive to all problem parameters and solves the proximal augmented Lagrangian (AL) subproblem inexactly. This adaptiveness facilitates the highly efficient application of the algorithm to a broad swath of practical problems. The inexact solution of the proximal AL subproblem overcomes many key challenges in the practical applications of ADMM. The resultant algorithm obtains an approximate solution of an optimization problem in a number of iterations that matches the state-of-the-art complexity for the class of proximal ADMM schemes. The second part of the research focuses on an inexact proximal mapping for the class of block proximal gradient methods. Key properties of this operator is established, facilitating the derivation of convergence rates for the proposed algorithm. Under two error decreases conditions, the algorithm matches the convergence rate of its exactly computed counterpart. Numerical results demonstrate the superior performance of the algorithm under a dynamic error regime over a fixed one. The dissertation concludes by providing convergence guarantees for the randomized BCD method applied to a broad class of functions, known as Hölder smooth functions. Convergence rates are derived for non-convex, convex, and strongly convex functions. These convergence rates match those furnished in the existing literature for the Lipschtiz smooth setting.
Paper Structure (19 sections, 12 theorems, 78 equations, 1 table, 4 algorithms)

This paper contains 19 sections, 12 theorems, 78 equations, 1 table, 4 algorithms.

Table of Contents

  1. INTRODUCTION
  2. Block Decomposable Methods
  3. AN ADAPTIVE PROXIMAL ADMM FOR NONCONVEX LINEARLY-CONSTRAINED COMPOSITE PROGRAMS$^*$
  4. Overview
  5. Introduction
  6. Chapter's Organization
  7. Notation and Basic Definitions
  8. Assumptions for Problem \ref{['initial.problem']} and an Inexact Solution Concept of \ref{['eq:x_update']}
  9. Assumptions for Problem \ref{['initial.problem']}
  10. An Inexact Solution Concept for \ref{['eq:x_update']}
  11. A Static P-ADMM
  12. A Block IPP Black-Box: B-IPP
  13. S-ADMM: A Static Version of A-ADMM
  14. The Proof of S-ADMM's Complexity Theorem (Theorem \ref{['thm:static.complexity']})
  15. An A-ADMM With Fixed Prox Stepsizes
  16. ...and 4 more sections

Key Result

Proposition 2.3.2

Assume that for some $(\tilde{M},\tilde{\mu}) \in \mathbb{R}^2_{++}$, the above functional pair $(\psi_{\text{s}},\psi_{\text{n}})$ is such that $\nabla \psi_{\text{s}}(\cdot)$ is $\tilde{M}$-Lipschitz continuous and $(\psi_s+\psi_n)$ is $\tilde{\mu}$-strongly convex. Then, Algorithm 1 in HeMonteiro

Theorems & Definitions (23)

  • Definition 2.3.1
  • Proposition 2.3.2
  • Proposition 2.4.1
  • proof
  • Theorem 2.4.2: S-ADMM Complexity
  • Lemma 2.5.1
  • proof
  • Lemma 2.5.2
  • proof
  • Lemma 2.5.3
  • ...and 13 more