Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Riemannian Optimization on the Oblique Manifold for Sparse Simplex Constraints via Multiplicative Updates

Flavia Esposito, Andersen Ang

TL;DR

The paper tackles sparse simplex-constrained, low-rank optimization by reformulating the problem on the oblique rotation manifold $\mathcal{OB}(r,n)$ with $\bm{H}=\bm{A}\odot\bm{A}$. It develops a Riemannian Multiplicative Update (RMU) algorithm that preserves nonnegativity and the unit-sum constraint through a metric-retraction step and a split-gradient update, with $Q=\bm{W}^T\bm{W}$ and $P=\bm{W}^T\bm{X}$ used to build the Riemannian gradient and handle the non-differentiable $\ell_1$ term via a subgradient. The method is shown to converge to stationary points on $\mathcal{OB}(r,n)$ and to perform favorably against a Riemannian Conjugate Gradient method and Euclidean heuristics in synthetic experiments, particularly at large scales where RMU's lower per-iteration cost and intrinsic constraint handling yield speed and stability advantages. This makes RMU a practical tool for structured low-rank problems in machine learning, signal processing, and computational biology where sparse simplex constraints are important.)

Abstract

Low-rank optimization problems with sparse simplex constraints involve variables that must satisfy nonnegativity, sparsity, and sum-to-one conditions, making their optimization particularly challenging due to the interplay between low-rank structures and constraints. These problems arise in various applications, including machine learning, signal processing, environmental fields, and computational biology. In this paper, we propose a novel manifold optimization approach to tackle these problems efficiently. Our method leverages the geometry of oblique rotation manifolds to reformulate the problem and introduces a new Riemannian optimization method based on Riemannian gradient descent that strictly maintains the simplex constraints. By exploiting the underlying manifold structure, our approach improves optimization efficiency. Experiments on synthetic datasets compared to standard Euclidean and Riemannian methods show the effectiveness of the proposed method.

Riemannian Optimization on the Oblique Manifold for Sparse Simplex Constraints via Multiplicative Updates

TL;DR

The paper tackles sparse simplex-constrained, low-rank optimization by reformulating the problem on the oblique rotation manifold with . It develops a Riemannian Multiplicative Update (RMU) algorithm that preserves nonnegativity and the unit-sum constraint through a metric-retraction step and a split-gradient update, with and used to build the Riemannian gradient and handle the non-differentiable term via a subgradient. The method is shown to converge to stationary points on and to perform favorably against a Riemannian Conjugate Gradient method and Euclidean heuristics in synthetic experiments, particularly at large scales where RMU's lower per-iteration cost and intrinsic constraint handling yield speed and stability advantages. This makes RMU a practical tool for structured low-rank problems in machine learning, signal processing, and computational biology where sparse simplex constraints are important.)

Abstract

Low-rank optimization problems with sparse simplex constraints involve variables that must satisfy nonnegativity, sparsity, and sum-to-one conditions, making their optimization particularly challenging due to the interplay between low-rank structures and constraints. These problems arise in various applications, including machine learning, signal processing, environmental fields, and computational biology. In this paper, we propose a novel manifold optimization approach to tackle these problems efficiently. Our method leverages the geometry of oblique rotation manifolds to reformulate the problem and introduces a new Riemannian optimization method based on Riemannian gradient descent that strictly maintains the simplex constraints. By exploiting the underlying manifold structure, our approach improves optimization efficiency. Experiments on synthetic datasets compared to standard Euclidean and Riemannian methods show the effectiveness of the proposed method.

Paper Structure

This paper contains 12 sections, 2 theorems, 15 equations, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Manifold formulation
  3. Formulation in oblique rotation manifold
  4. Oblique rotation matrices manifold
  5. Riemaniann Multiplicative Update
  6. Algorithm for minimizing \ref{['riemanianfunc_4']} over $\mathcal{O} \mathcal{B}(r,n)$
  7. Numerical Experiments
  8. Comparative methods
  9. Experimental settings
  10. Evaluation metrics
  11. Results and Discussion
  12. Conclusion

Key Result

Lemma 1

The matrix $\bm{H}$ stays in the simplex if and only if $\bm{H} = \bm{A} \odot \bm{A}$ with $\bm{A}\in \mathcal{O} \mathcal{B}(r,n)$.

Figures (2)

  • Figure 1: The plot of the (nonconvex) $\ell_{0.5}^{0.5}-$quasi-norm and the simplex constraint. Left: the surface of $\ell_{0.5}^{0.5}-$quasi-norm in $\mathbb{R}^2$ and the simplex constraint (in green). Right: the level set plot. Minimizing the $\ell_{0.5}^{0.5}-$quasi-norm on the green simplex gives the solution sitting on the corner of the simplex, leading to sparsity.
  • Figure 2: A typical convergence plot of the four methods. Here in terms of iterations, the convergence of RCG has the best performance, however, it takes the highest amount of computational time for performing one iteration (shown by a larger gap between two consecutive points on the right).

Theorems & Definitions (7)

  • Remark 1: Nonconvex simplex
  • Lemma 1
  • proof
  • Remark 2: Why $\ell_{1/2}^{1/2}$-quasi-norm.
  • Remark 3: Nonconvexity of the objective functions
  • Theorem 1: esposito2024chordal
  • Remark 4: Subgradient