Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Structured covariance estimation via tensor-train decomposition

Artsiom Patarusau, Nikita Puchkin, Maxim Rakhuba, Fedor Noskov

TL;DR

The paper tackles high-dimensional covariance estimation by imposing a tensor-train structured Kronecker model $\Sigma = \sum_{j=1}^J \sum_{k=1}^K U_j \otimes V_{jk} \otimes W_k$, enabling dimension-free analysis via the rearrangement $\mathcal{R}(\Sigma) \in \mathbb{R}^{p^2 \times q^2 \times r^2}$. It introduces the HarTTh algorithm, a TT-SVD/HOOI-inspired iterative estimator that recovers the best $(J,K)$-TT approximation of $\mathcal{R}(\Sigma)$ from noisy observations, with estimator $\\tilde{\\Sigma} = \\mathcal{R}^{-1}(\\widehat{\\mathcal{T}})$. The authors establish non-asymptotic, high-probability bounds that decompose error into a bias term due to model misspecification and a variance term that scales as $\\|\\Sigma\\|$ times $\\sqrt{(J \\mathtt{r}_1^2(\\Sigma) + JK \\mathtt{r}_2^2(\\Sigma) + K \\mathtt{r}_3^2(\\Sigma) + \\log(1/\\delta))/n}$, with additional remainder terms. Numerical experiments demonstrate competitive performance and computational efficiency of HarTTh relative to TT-HOSVD, Tucker-based methods, and PRLS, illustrating practical viability for structured covariance estimation in multiway data.

Abstract

We consider a problem of covariance estimation from a sample of i.i.d. high-dimensional random vectors. To avoid the curse of dimensionality we impose an additional assumption on the structure of the covariance matrix $Σ$. To be more precise we study the case when $Σ$ can be approximated by a sum of double Kronecker products of smaller matrices in a tensor train (TT) format. Our setup naturally extends widely known Kronecker sum and CANDECOMP/PARAFAC models but admits richer interaction across modes. We suggest an iterative polynomial time algorithm based on TT-SVD and higher-order orthogonal iteration (HOOI) adapted to Tucker-2 hybrid structure. We derive non-asymptotic dimension-free bounds on the accuracy of covariance estimation taking into account hidden Kronecker product and tensor train structures. The efficiency of our approach is illustrated with numerical experiments.

Structured covariance estimation via tensor-train decomposition

TL;DR

The paper tackles high-dimensional covariance estimation by imposing a tensor-train structured Kronecker model , enabling dimension-free analysis via the rearrangement . It introduces the HarTTh algorithm, a TT-SVD/HOOI-inspired iterative estimator that recovers the best -TT approximation of from noisy observations, with estimator . The authors establish non-asymptotic, high-probability bounds that decompose error into a bias term due to model misspecification and a variance term that scales as times , with additional remainder terms. Numerical experiments demonstrate competitive performance and computational efficiency of HarTTh relative to TT-HOSVD, Tucker-based methods, and PRLS, illustrating practical viability for structured covariance estimation in multiway data.

Abstract

We consider a problem of covariance estimation from a sample of i.i.d. high-dimensional random vectors. To avoid the curse of dimensionality we impose an additional assumption on the structure of the covariance matrix . To be more precise we study the case when can be approximated by a sum of double Kronecker products of smaller matrices in a tensor train (TT) format. Our setup naturally extends widely known Kronecker sum and CANDECOMP/PARAFAC models but admits richer interaction across modes. We suggest an iterative polynomial time algorithm based on TT-SVD and higher-order orthogonal iteration (HOOI) adapted to Tucker-2 hybrid structure. We derive non-asymptotic dimension-free bounds on the accuracy of covariance estimation taking into account hidden Kronecker product and tensor train structures. The efficiency of our approach is illustrated with numerical experiments.

Paper Structure

This paper contains 23 sections, 12 theorems, 232 equations, 2 figures, 8 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contribution.
  3. Paper structure.
  4. Notation.
  5. Main results
  6. Experiments
  7. Conclusion
  8. Additional notations and basic tools
  9. Proof sketch for Theorem \ref{['theorem: Sigma estimator performance']}
  10. Proof of Theorem \ref{['theorem: Sigma estimator performance']}
  11. Proof of Lemma \ref{['lemma: m1 norm upper bound']}
  12. Proof of Lemma \ref{['lemma: sup U times noise']}
  13. Proof of Lemma \ref{['lemma: U star times noise bound']}
  14. Proof of Lemma \ref{['lemma: leading term bound']}
  15. Proof of Theorem \ref{['theorem: sensetivity analysis']}
  16. ...and 8 more sections

Key Result

Theorem 2.2

Fix $\delta \in (0, 1)$. Grant Assumption as:orlicz. Suppose that singular values $\sigma_J(\mathtt{m}_1(\mathcal{R}(\Sigma))$, $\sigma_K(\mathtt{m}_3(\mathcal{R}(\Sigma))$ satisfy Then, we have with probability at least $1 - \delta$, provided $n \geqslant \mathtt{R}_\delta$, where and $\mathtt{R}_\delta$ and remainder terms $\widetilde{\diamondsuit}_2, \widetilde{r}_T$ are defined in Table tab

Figures (2)

  • Figure 1: Spectrum of the objectives in case of random sub-components. As one can see, dense spectrum of matrix $S$ with noise become separable for matricizations.
  • Figure 2: Performance of tensor decomposition algorithms and spectrum behavior under noise increase.

Theorems & Definitions (22)

  • Theorem 2.2
  • Theorem B.1
  • proof : Proof of Theorem \ref{['theorem: Sigma estimator performance']}
  • Lemma C.1
  • Lemma C.2
  • Lemma C.3
  • Lemma C.4
  • proof
  • Lemma C.5
  • proof
  • ...and 12 more