Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accelerated alternating minimization algorithm for low-rank approximations in the Chebyshev norm

Stanislav Morozov, Dmitry Zheltkov, Alexander Osinsky

Abstract

Nowadays, low-rank approximations of matrices are an important component of many methods in science and engineering. Traditionally, low-rank approximations are considered in unitary invariant norms, however, recently element-wise approximations have also received significant attention in the literature. In this paper, we propose an accelerated alternating minimization algorithm for solving the problem of low-rank approximation of matrices in the Chebyshev norm. Through the numerical evaluation we demonstrate the effectiveness of the proposed procedure for large-scale problems. We also theoretically investigate the alternating minimization method and introduce the notion of a $2$-way alternance of rank $r$. We show that the presence of a $2$-way alternance of rank $r$ is the necessary condition of the optimal low-rank approximation in the Chebyshev norm and that all limit points of the alternating minimization method satisfy this condition.

Accelerated alternating minimization algorithm for low-rank approximations in the Chebyshev norm

Abstract

Nowadays, low-rank approximations of matrices are an important component of many methods in science and engineering. Traditionally, low-rank approximations are considered in unitary invariant norms, however, recently element-wise approximations have also received significant attention in the literature. In this paper, we propose an accelerated alternating minimization algorithm for solving the problem of low-rank approximation of matrices in the Chebyshev norm. Through the numerical evaluation we demonstrate the effectiveness of the proposed procedure for large-scale problems. We also theoretically investigate the alternating minimization method and introduce the notion of a -way alternance of rank . We show that the presence of a -way alternance of rank is the necessary condition of the optimal low-rank approximation in the Chebyshev norm and that all limit points of the alternating minimization method satisfy this condition.
Paper Structure (19 sections, 24 theorems, 153 equations, 4 figures, 1 table, 4 algorithms)

This paper contains 19 sections, 24 theorems, 153 equations, 4 figures, 1 table, 4 algorithms.

Table of Contents

  1. Introduction
  2. Related work
  3. Preliminaries
  4. Equioscillation theorem
  5. Accelerated uniform approximation algorithm
  6. On the problem of size $(r+1) \times r$
  7. Iterative update of the current set of indices
  8. Final algorithm
  9. Alternating minimization method
  10. Method description
  11. Rank-$r$ alternance
  12. Numerical evaluation
  13. 2-way alternance of rank $r$
  14. Hilbert matrix
  15. Identity matrix
  16. ...and 4 more sections

Key Result

Theorem 3.2

\newlabeltheorem:exists_unique_cont0 Let $V \in \mathbb{R}^{n \times r}$ be a Chebyshev matrix and $a \in \mathbb{R}^n$. Then the solution to the problem (eq:uniform_approx_problem) exists, is unique and continuously depends on the matrix $V$ and right-hand side $a$.

Figures (4)

  • Figure 1: An example of low-rank approximation of a grayscale image. The left image corresponds to the original picture of size $64\times 64$, the middle image contains the approximation of rank $8$, the right image demonstrates the 2-way alternance of rank $r$. Blue pixels correspond to the positions where the maximum absolute values in the residual is not reached, yellow where it is reached with the positive signs and red where it is reached with the negative sign.
  • Figure 2: Approximation error of the Hilbert matrix with alternating minimization method (CAM), alternating projections (AP) and SVD. The left panel corresponds to the matrix of size $n=2,048$. The iterative procedures were started from $20$ random initial points. The colored domains correspond to the maximal and minimal values over the initial points and the dotted curves correspond to the median value. The right panel contains the results for the matrix of size $n=32,768$.
  • Figure 3: Approximation error of the identity matrix with alternating minimization method (CAM) and alternating projections (AP). The matrix size is $n=2,048$. The right plot corresponds to the zoom for the small ranks. The iterative procedures were started from $20$ random initial points. The colored domains correspond to the maximal and minimal values over the initial points and the dotted curves correspond to the median value. The plots also contain the known theoretical upper and lower bounds.
  • Figure 4: Approximation error for the function-generated matrix with alternating minimization method (CAM), alternating projections (AP) and SVD. The matrix size is $n=4,096$. The iterative procedures were started from $20$ random initial points. The colored domains correspond to the maximal and minimal values over the initial points and the dotted curves correspond to the median value.

Theorems & Definitions (46)

  • Definition 3.1
  • Theorem 3.2
  • Lemma 3.3
  • Definition 3.4
  • Theorem 3.5
  • Theorem 3.6
  • Theorem 3.7
  • Lemma 3.8
  • Proof 1
  • Lemma 3.9
  • ...and 36 more