Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Relaxation Method for Nonsmooth Nonlinear Optimization with Binary Constraints

Lianghai Xiao, Yitian Qian, Shaohua Pan

TL;DR

The paper tackles nonsmooth binary optimization by embedding the binary constraint into a DC-penalized, lifted SDP framework and solving via a Burer–Monteiro factorization together with Moreau envelope smoothing (DCRA). It proves that inner MM iterations achieve an $\varepsilon$-stationary point in $O(\varepsilon^{-2})$ and that the outer penalty scheme terminates in finite steps, with a rank-one projection yielding an approximately feasible binary solution and an explicit optimality-gap bound. Empirically, DCRA demonstrates superior accuracy and scalability on synthetic data, binary compressed sensing, and supervised hashing, compared with state-of-the-art baselines. The method offers a principled, scalable alternative for broad classes of nonsmooth binary problems, with solid theoretical guarantees and practical efficacy. Overall, this work advances nonsmooth binary optimization by combining DC relaxation, SDP lifting, and MM-style smoothing to achieve both quality and scalability.

Abstract

We study binary optimization problems of the form \( \min_{x\in\{-1,1\}^n} f(Ax-b) \) with possibly nonsmooth loss \(f\). Following the lifted rank-one semidefinite programming (SDP) approach\cite{qian2023matrix}, we develop a majorization-minimization algorithm by using the difference-of-convexity (DC) reformuation for the rank-one constraint and the Moreau envelop for the nonsmooth loss. We provide global complexity guarantees for the proposed \textbf{D}ifference of \textbf{C}onvex \textbf{R}elaxation \textbf{A}lgorithm (DCRA) and show that it produces an approximately feasible binary solution with an explicit bound on the optimality gap. Numerical experiments on synthetic and real datasets confirm that our method achieves superior accuracy and scalability compared with existing approaches.

A Relaxation Method for Nonsmooth Nonlinear Optimization with Binary Constraints

TL;DR

The paper tackles nonsmooth binary optimization by embedding the binary constraint into a DC-penalized, lifted SDP framework and solving via a Burer–Monteiro factorization together with Moreau envelope smoothing (DCRA). It proves that inner MM iterations achieve an -stationary point in and that the outer penalty scheme terminates in finite steps, with a rank-one projection yielding an approximately feasible binary solution and an explicit optimality-gap bound. Empirically, DCRA demonstrates superior accuracy and scalability on synthetic data, binary compressed sensing, and supervised hashing, compared with state-of-the-art baselines. The method offers a principled, scalable alternative for broad classes of nonsmooth binary problems, with solid theoretical guarantees and practical efficacy. Overall, this work advances nonsmooth binary optimization by combining DC relaxation, SDP lifting, and MM-style smoothing to achieve both quality and scalability.

Abstract

We study binary optimization problems of the form \( \min_{x\in\{-1,1\}^n} f(Ax-b) \) with possibly nonsmooth loss . Following the lifted rank-one semidefinite programming (SDP) approach\cite{qian2023matrix}, we develop a majorization-minimization algorithm by using the difference-of-convexity (DC) reformuation for the rank-one constraint and the Moreau envelop for the nonsmooth loss. We provide global complexity guarantees for the proposed \textbf{D}ifference of \textbf{C}onvex \textbf{R}elaxation \textbf{A}lgorithm (DCRA) and show that it produces an approximately feasible binary solution with an explicit bound on the optimality gap. Numerical experiments on synthetic and real datasets confirm that our method achieves superior accuracy and scalability compared with existing approaches.
Paper Structure (20 sections, 6 theorems, 79 equations, 8 figures, 2 tables, 3 algorithms)

This paper contains 20 sections, 6 theorems, 79 equations, 8 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Contributions
  4. Organization
  5. Preliminaries
  6. Reformulation
  7. Algorithmic Framework
  8. Convergence Analysis
  9. Numerical Experiment
  10. Parameter sensitivity analysis
  11. Performance on randomly generated problems
  12. Performance on binary compressed sensing
  13. Performance on supervised hashing Code Learning
  14. Conclusion
  15. Proof of Proposition \ref{['prop:inner-descent']}
  16. ...and 5 more sections

Key Result

Proposition 1

Let $\{V^{k,l}\}_{l\in\mathbb{N}}\subset\mathcal{S}$ be the sequence generated by Algorithm AlgA for a fixed $k\in \mathbb{N}$. Then the following statements hold:

Figures (8)

  • Figure 1: Parameter sensitivity analysis of the proposed algorithm for two problem sizes $(n,r)=(500,300)$ and $(n,r)=(1000,600)$. For each parameter setting, results are averaged over 10 trials, with error bars indicating one standard deviation. Left panels: objective value; Right panels: runtime. Both axes use logarithmic scales where appropriate.
  • Figure 2: Phase diagrams for $\mu = 0, 2, 10$ cases.
  • Figure 3: Precision vs Retrieval Samples
  • Figure 4: Precision vs. Recall 16
  • Figure 5: Precision vs. Recall 32
  • ...and 3 more figures

Theorems & Definitions (8)

  • Definition 1
  • Definition 2
  • Proposition 1
  • Theorem 1
  • Lemma 1
  • Proposition 2
  • Corollary 1
  • Theorem 2