Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the B-subdifferential of proximal operators of affine-constrained $\ell_1$ regularizer

Xudong Li, Meixia Lin, Kim-Chuan Toh

TL;DR

This work addresses optimization with an affine-constrained $\ell_1$ regularizer by deriving the proximal operator and its ${\rm B}$-subdifferential, enabling fast second-order methods. It introduces a dual-variable formulation and an efficient one-dimensional root-finding procedure for the proximal map, complemented by a complete B-subdifferential analysis for both $c\neq 0$ and $c=0$ cases. A double-loop algorithm combining a preconditioned proximal point framework with a semismooth Newton inner loop is proposed, with practical implementation details and convergence guarantees. Numerical experiments on microbiome compositional data analysis and sparse subspace clustering demonstrate superior efficiency and accuracy compared to state-of-the-art solvers, highlighting the practical impact for structured sparse learning under affine constraints.

Abstract

In this work, we study the affine-constrained $\ell_1$ regularizers, which frequently arise in statistical and machine learning problems across a variety of applications, including microbiome compositional data analysis and sparse subspace clustering. With the aim of developing scalable second-order methods for solving optimization problems involving such regularizers, we analyze the associated proximal mapping and characterize its generalized differentiability, with a focus on its B-subdifferential. The revealed structured sparsity in the B-subdifferential enables us to design efficient algorithms within the proximal point framework. Extensive numerical experiments on real applications, including comparisons with state-of-the-art solvers, further demonstrate the superior performance of our approach. Our findings provide new insights into the sensitivity and stability properties of affine-constrained nonsmooth regularizers, and contribute to the development of fast second-order methods for a class of structured, constrained sparse learning problems.

On the B-subdifferential of proximal operators of affine-constrained $\ell_1$ regularizer

TL;DR

This work addresses optimization with an affine-constrained regularizer by deriving the proximal operator and its -subdifferential, enabling fast second-order methods. It introduces a dual-variable formulation and an efficient one-dimensional root-finding procedure for the proximal map, complemented by a complete B-subdifferential analysis for both and cases. A double-loop algorithm combining a preconditioned proximal point framework with a semismooth Newton inner loop is proposed, with practical implementation details and convergence guarantees. Numerical experiments on microbiome compositional data analysis and sparse subspace clustering demonstrate superior efficiency and accuracy compared to state-of-the-art solvers, highlighting the practical impact for structured sparse learning under affine constraints.

Abstract

In this work, we study the affine-constrained regularizers, which frequently arise in statistical and machine learning problems across a variety of applications, including microbiome compositional data analysis and sparse subspace clustering. With the aim of developing scalable second-order methods for solving optimization problems involving such regularizers, we analyze the associated proximal mapping and characterize its generalized differentiability, with a focus on its B-subdifferential. The revealed structured sparsity in the B-subdifferential enables us to design efficient algorithms within the proximal point framework. Extensive numerical experiments on real applications, including comparisons with state-of-the-art solvers, further demonstrate the superior performance of our approach. Our findings provide new insights into the sensitivity and stability properties of affine-constrained nonsmooth regularizers, and contribute to the development of fast second-order methods for a class of structured, constrained sparse learning problems.

Paper Structure

This paper contains 17 sections, 14 theorems, 84 equations, 6 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Computation of the proximal mapping
  3. Characterization of the B-subdifferential of ${\rm Prox}_{\lambda q_{\mu,c}}(\cdot)$
  4. Lipschitz continuity of the dual multiplier
  5. B-subdifferential of the dual multiplier mapping
  6. B-subdifferential of ${\rm Prox}_{\lambda q_{\mu,c}}(\cdot)$
  7. Discussion of the case $c = 0$
  8. Double-loop algorithm for affine-constrained sparse optimization
  9. Outer loop: preconditioned PPA
  10. Inner loop: semismooth Newton method
  11. Implementation details
  12. Numerical experiments
  13. Microbiome compositional data analysis
  14. Regression analysis
  15. Classification analysis
  16. ...and 2 more sections

Key Result

Proposition 1

A necessary and sufficient optimality condition for eq: def_prox_qx_C is the existence of a dual multiplier $w\in \mathbb{R}$ such that Once such a scalar $w$ is identified, then

Figures (6)

  • Figure 1: Comparison of path generation between our algorithm and SparseReg's path following algorithm on microbiome compositional regression $(m=932,n=1000)$.
  • Figure 2: Comparison of path generation between our algorithm and SparseReg's path following algorithm on microbiome compositional regression $(m=932,n=3000)$.
  • Figure 3: Runtime comparison of path generation between our algorithm and SparseReg's ADMM on microbiome compositional regression with varying sizes.
  • Figure 4: Runtime of our algorithm with or without AS, for path generation on microbiome compositional regression $(m=932,n=209,356)$.
  • Figure 5: Comparison of path generation between our algorithm and ECLasso on microbiome compositional classification $(m=50,n=60)$.
  • ...and 1 more figures

Theorems & Definitions (27)

  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Proposition 3
  • proof
  • Proposition 4
  • proof
  • Lemma 1
  • Proposition 5
  • ...and 17 more