Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Faster Stochastic ADMM for Nonsmooth Composite Convex Optimization in Hilbert Space

Weihua Deng, Haiming Song, Hao Wang, Jinda Yang

TL;DR

The strong convergence of the proposed ADMM algorithm in the strongly convex case is proved, and the faster nonergodic convergence rates are shown in terms of functional values and feasibility violation for both strongly convex and general convex cases.

Abstract

In this paper, a stochastic alternating direction method of multipliers (ADMM) is proposed for a class of nonsmooth composite and stochastic convex optimization problems in Hilbert space, motivated by optimization problems constrained by partial differential equation (PDE) with random coefficients. We prove the strong convergence of the proposed ADMM algorithm in the strongly convex case, and show the faster nonergodic convergence rates in terms of functional values and feasibility violation for both strongly convex and general convex cases. We demonstrate the application of the proposed method to solve certain model problems, along with its associated probability bound of large deviation. Some preliminary numerical results illustrate the efficiency of our method.

Faster Stochastic ADMM for Nonsmooth Composite Convex Optimization in Hilbert Space

TL;DR

The strong convergence of the proposed ADMM algorithm in the strongly convex case is proved, and the faster nonergodic convergence rates are shown in terms of functional values and feasibility violation for both strongly convex and general convex cases.

Abstract

In this paper, a stochastic alternating direction method of multipliers (ADMM) is proposed for a class of nonsmooth composite and stochastic convex optimization problems in Hilbert space, motivated by optimization problems constrained by partial differential equation (PDE) with random coefficients. We prove the strong convergence of the proposed ADMM algorithm in the strongly convex case, and show the faster nonergodic convergence rates in terms of functional values and feasibility violation for both strongly convex and general convex cases. We demonstrate the application of the proposed method to solve certain model problems, along with its associated probability bound of large deviation. Some preliminary numerical results illustrate the efficiency of our method.
Paper Structure (16 sections, 5 theorems, 93 equations, 10 figures, 2 tables, 1 algorithm)

This paper contains 16 sections, 5 theorems, 93 equations, 10 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Motivation
  3. Algorithmic framework
  4. Related works
  5. Our goals
  6. Organization
  7. Convergence analysis
  8. Preliminaries
  9. Strongly convex case
  10. General convex case
  11. Application to PDE--constrained optimization under uncertainty
  12. Elliptic optimal control problem
  13. Numerical experiments
  14. Parameter settings
  15. Experiment results
  16. ...and 1 more sections

Key Result

Lemma 2.1

Let the sequences $\left\lbrace (v_{k}, s_{k}, \psi_{k}) \right\rbrace$ and $\left\lbrace (u_{k}, z_{k}, \lambda_{k}) \right\rbrace$ be generated by Algorithm alg:Stochastic Linearized ADMM. Then for any $\lambda \in U$, we have

Figures (10)

  • Figure 1: Objective values of Algorithm \ref{['alg:Stochastic Linearized ADMM']} and SG-type methods for different $(\alpha,\beta)$ pairs in strongly convex case.
  • Figure 2: Objective values of Algorithm \ref{['alg:Stochastic Linearized ADMM']} for different $(\alpha,\beta)$ pairs in the strongly convex case with different sampled stochastic gradients $G_{k}$.
  • Figure 3: Control variables for several values of $\beta$ with fixed $\alpha = 10^{-4}$ in strongly convex case. Top row: principal view; bottom row: top view.
  • Figure 4: State variables for several values of $\beta$ with fixed $\alpha = 10^{-4}$ in strongly convex case. Top row: principal view; bottom row: top view.
  • Figure 5: High-probability convergence of Algorithm \ref{['alg:Stochastic Linearized ADMM']} for different $(\alpha,\beta)$ pairs in the strongly convex case.
  • ...and 5 more figures

Theorems & Definitions (13)

  • Remark 2.1
  • Remark 2.2
  • Lemma 2.1
  • proof
  • Theorem 2.1
  • proof
  • Theorem 2.2
  • proof
  • Theorem 2.3
  • proof
  • ...and 3 more