Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Randomized Block Coordinate DC Programming

Hoomaan Maskan, Paniz Halvachi, Suvrit Sra, Alp Yurtsever

TL;DR

The paper develops a randomized block-coordinate extension of the Difference of Convex Algorithm (DCA) for DC programs with separable structure, proving a non-asymptotic $O(n/k)$ convergence rate in expectation with respect to a Forward–Backward envelope–based stationarity measure. It also leverages the DCA–EM connection to formulate a randomized block-coordinate EM method, with analogous nonasymptotic guarantees. The methods are analyzed for both differentiable and non-differentiable concave terms, with faster linear rates under a generalized Polyak–Łojasiewicz condition. Empirical results on logistic regression with nonconvex sparsity and nonconvex quadratic programming, plus a Block EM application to one-bit MIMO, demonstrate practical scalability and competitive performance against standard DCA and RCSD baselines.

Abstract

We introduce an extension of the Difference of Convex Algorithm (DCA) in the form of a randomized block coordinate approach for problems with separable structure. For $n$ coordinate-blocks and $k$ iterations, our main result proves a non-asymptotic convergence rate of $O(n/k)$ in expectation, with respect to a stationarity measure based on a Forward-Backward envelope. Furthermore, leveraging the connection between DCA and Expectation Maximization (EM), we propose a randomized block coordinate EM algorithm.

Randomized Block Coordinate DC Programming

TL;DR

The paper develops a randomized block-coordinate extension of the Difference of Convex Algorithm (DCA) for DC programs with separable structure, proving a non-asymptotic convergence rate in expectation with respect to a Forward–Backward envelope–based stationarity measure. It also leverages the DCA–EM connection to formulate a randomized block-coordinate EM method, with analogous nonasymptotic guarantees. The methods are analyzed for both differentiable and non-differentiable concave terms, with faster linear rates under a generalized Polyak–Łojasiewicz condition. Empirical results on logistic regression with nonconvex sparsity and nonconvex quadratic programming, plus a Block EM application to one-bit MIMO, demonstrate practical scalability and competitive performance against standard DCA and RCSD baselines.

Abstract

We introduce an extension of the Difference of Convex Algorithm (DCA) in the form of a randomized block coordinate approach for problems with separable structure. For coordinate-blocks and iterations, our main result proves a non-asymptotic convergence rate of in expectation, with respect to a stationarity measure based on a Forward-Backward envelope. Furthermore, leveraging the connection between DCA and Expectation Maximization (EM), we propose a randomized block coordinate EM algorithm.

Paper Structure

This paper contains 22 sections, 8 theorems, 95 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Related Work on BDCA
  4. Related Work on Block EM
  5. Other Work on Nonconvex BC Methods
  6. Block Coordinate DCA
  7. Convergence Guarantees for BDCA
  8. Guarantees under Non-differentiable Concave Terms
  9. Faster Rates Under a Generalized Polyak-Łojasiewicz Condition
  10. Block EM Algorithm
  11. Numerical Experiments
  12. Logistic Regression with Nonconvex Sparsity Regularization
  13. Nonconvex Quadratic Programming with Negative $\ell_1$ Regularization
  14. Conclusions
  15. Regularized EM
  16. ...and 7 more sections

Key Result

Lemma 1

Let $\mathcal{M} \subseteq \mathbb{R}^m$ be a closed convex set, and let $f,g,h : \mathcal{M} \to \mathbb{R}$ be lower-semicontinuous convex functions, where $f$ is $L$-smooth and $h$ is continuously differentiable. Then, $\mathrm{gap}^L_{\mathcal{M}}(\bm{y}) \geq 0$ for all $\bm{y}\in\mathcal{M}$,

Figures (3)

  • Figure 1: Comparison of DC and Bdca for logistic regression with nonconvex sparsity regularization in \ref{['eqn:classification-reg-lK']}, in terms of the objective value and the gap (as defined in \ref{['def:gap']}), on Rcv1 and Real-sim datasets, with $\rho = 0.1$. Epoch represents iteration $\times$ block-size$/m$.
  • Figure 2: Comparison of DC and Bdca for logistic regression with nonconvex sparsity regularization in \ref{['eqn:classification-reg-lK']}, in terms of the objective value and the gap (as defined in \ref{['def:gap']}), on Rcv1 and Real-sim datasets, with $\rho = 10$. Epoch represents iteration $\times$ block-size$/m$.
  • Figure SM3: Comparison of Block EM and classical EM in terms of BER vs. $E_b/N$.

Theorems & Definitions (20)

  • Definition 1
  • Lemma 1
  • proof
  • Remark 2
  • Theorem 3
  • proof
  • Remark 4
  • Remark 5
  • Lemma 6
  • Theorem 7
  • ...and 10 more