Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rectangular Rotational Invariant Estimator for High-Rank Matrix Estimation

Farzad Pourkamali, Nicolas Macris

TL;DR

This work proposes an estimator which is optimal among the class of rectangular rotational invariant estimators and can be applied irrespective of the prior on the signal, and proves the optimality of the proposed estimator for Gaussian noise.

Abstract

We consider estimating a matrix from noisy observations coming from an arbitrary additive bi-rotational invariant perturbation. We propose an estimator which is optimal among the class of rectangular rotational invariant estimators and can be applied irrespective of the prior on the signal. For the particular case of Gaussian noise, we prove the optimality of the proposed estimator, and we find an explicit expression for the MMSE in terms of the limiting singular value distribution of the observation matrix. Moreover, we prove a formula linking the asymptotic mutual information and the limit of a log-spherical integral of rectangular matrices. We also provide numerical checks for our results for general bi-rotational invariant noise, as well as Gaussian noise, which match our theoretical predictions.

Rectangular Rotational Invariant Estimator for High-Rank Matrix Estimation

TL;DR

This work proposes an estimator which is optimal among the class of rectangular rotational invariant estimators and can be applied irrespective of the prior on the signal, and proves the optimality of the proposed estimator for Gaussian noise.

Abstract

We consider estimating a matrix from noisy observations coming from an arbitrary additive bi-rotational invariant perturbation. We propose an estimator which is optimal among the class of rectangular rotational invariant estimators and can be applied irrespective of the prior on the signal. For the particular case of Gaussian noise, we prove the optimality of the proposed estimator, and we find an explicit expression for the MMSE in terms of the limiting singular value distribution of the observation matrix. Moreover, we prove a formula linking the asymptotic mutual information and the limit of a log-spherical integral of rectangular matrices. We also provide numerical checks for our results for general bi-rotational invariant noise, as well as Gaussian noise, which match our theoretical predictions.
Paper Structure (50 sections, 16 theorems, 196 equations, 7 figures)

This paper contains 50 sections, 16 theorems, 196 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Finite rank.
  3. Sub-linear rank regime.
  4. Linear rank regime.
  5. Main contributions.
  6. Denoising Model and Rotational Invariant Estimators
  7. Rectangular RIE Class
  8. Algorithmic RIE
  9. Bayes Optimality and MMSE
  10. Gaussian noise
  11. RIE
  12. Algorithm
  13. MMSE
  14. Mutual information
  15. Numerical simulations
  16. ...and 35 more sections

Key Result

Proposition 1

For $1 \leq j \leq N$, and for any $\epsilon > 0$ such that $[\gamma_j -\epsilon, \gamma_j +\epsilon] \cap \{ \gamma_1, \cdots, \gamma_N \} = \{ \gamma_j\}$, the optimal singular values Oracle-sv satisfy

Figures (7)

  • Figure 1: Performance of the algorithmic RIE based on \ref{['rect-RIE']} as compared to the oracle one. Signal matrix $\mathbf{S} \in \mathbb{R}^{N \times M}$ has i.i.d. Gaussian entries of variance $1/N$. Results are averaged over 10 runs (error bars are invisible). Average relative error is also reported. In both examples, the Hilbert transform of the observation is computed numerically using Cauchy kernel method in potters2020first.
  • Figure 2: Validity of the estimation \ref{['L(z)-approx']}. Plots are average of 100 experiments and $95\%$ confidence interval is also depicted. The signal matrix $\mathbf{S} \in \mathbb{R}^{N \times M}$ has i.i.d. Gaussian entries of variance $1/N$, and $M = N/\alpha_0$. the expressions are evaluated for $z = 1 + \mathsf{i}/\sqrt{N}$. In the left panel, the relative error \ref{['rel-error-tr-relation']} is plotted for various values of $N$. On the same simulations, the error term is plotted in the right panel which behaves as $N^{-1/2}$ which matches with the conjecture of remark \ref{['error-term-imz']}.
  • Figure 3: Performance of the RIE \ref{['rect-RIE']} for the Gaussian signal and noise. Signal and noise matrices $\mathbf{S}, \mathbf{Z} \in \mathbb{R}^{N \times M}$ have i.i.d. Gaussian entries of variance $1/N$. The MMSE is plotted for two aspect ratios $\alpha = 1, 1/2$, and the RIE \ref{['rect-RIE']} is applied to $N = 1000$. Results are averaged over 10 runs.
  • Figure 4: Performance of the RIE using \ref{['optimal-sv-approx-alg']} for the Gaussian signal. The formula \ref{['optimal-sv-approx-alg']} is used to estimate the optimal singular values with $z = \gamma_j + \mathsf{i} N^{-\epsilon}$. RIE is applied to $N = 1000, M=2000, \lambda = 2$, and results are averaged over 10 runs.
  • Figure 5: Signal with Bernoulli spectrum. MSE is normalized by the norm of the signal, $1-p$. The RIE is applied to $N=1000, M =2000$, and the results are averaged over 10 runs (error bars might be invisible).
  • ...and 2 more figures

Theorems & Definitions (29)

  • Remark 1
  • Proposition 1
  • Theorem 2: Estimation of $L(z)$
  • Remark 2
  • Remark 3: On imaginary part of $z$ in \ref{['optimal-sv-approx']}
  • Remark 4: Relation to the formula \ref{['rect-RIE']}
  • Remark 5
  • Theorem 4: Mutual Information
  • Remark 6
  • Lemma 5.1
  • ...and 19 more