Stochastic Trace and Diagonal Estimator for Tensors

TL;DR

The paper investigates implicit estimation of the trace and diagonal entries of N-order tensors via tensor-vector queries, filling a gap in extending Hutchinson-type estimators from matrices to tensors. It introduces unbiased estimators for both diagonal elements and the trace, along with variance bounds and concentration analyses under Rademacher and Gaussian inputs, and validates the theory with simulations. The proposed methods generalize the classical matrix estimators, recovering Hutchinson and Bekas results when N=2, and offer practical tools for efficient tensor-structured data analysis in settings where explicit tensor construction is expensive. Potential applications span hypergraph spectral theory, quantum computing, and other domains dealing with high-order tensors.

Abstract

We consider the problem of estimating the trace and diagonal entries of an N-order tensor (where $N \geq 2$) under the framework where the tensor can only be accessed through tensor-vector multiplication. The aim is to estimate the tensor's diagonal entries and trace by minimizing the number of tensor-vector queries. The seminal work of Hutchinson and its extended version due to Bekas et al. give unbiased estimates of the trace and diagonal elements of a given matrix, respectively, using matrix-vector queries. However, to the best of our knowledge, no analogous results are known for estimating the trace and diagonal entries of higher-order tensors using tensor-vector queries. This paper addresses this gap and presents unbiased estimators for the trace and diagonal entries of a tensor under this model. Our proposed methods can be seen as generalizations of Hutchinson's and Bekas et al.'s estimators and reduce to their estimators when N = 2. We provide a rigorous theoretical analysis of our proposals and complement it with supporting simulations.

Figures (5)

• Figure 1: Analysis of the tightness of the upper bound (Equation \ref{['eq:eqn_tr_rad_var_proof']}) w.r.t. the exact variance expression (Equation \ref{['eq:eqn_symptotic_tr_rad']}) for different combinations of $N$ and $d$. $N$ denotes the order of the tensor. The exact variance to upper bound ratio being closer to $1$ means the upper bound is tight.
• Figure 2: Variance analysis of Rademacher and Normal distribution based diagonal estimators via boxplots using the relative errors observed in $100$ runs. $N$ and $\alpha$ denote the order of the tensor and the ratio of the sum of the square of diagonal entries and the square Frobenius norm of the tensor, respectively. The smaller interquartile range is an indication of a smaller variance.
• Figure 3: Comparison among the Rademacher and the normal distribution based diagonal estimators on the basis of mean absolute relative error over $100$ experimental runs for $N \in \{2,3,4\}$. $N$ and $\alpha$ denote the order of the tensor and the ratio of the sum of the squares of diagonal entries and the square Frobenius norm of the tensor, respectively. The smaller value of the mean absolute relative error indicates better estimates.
• Figure 4: Variance analysis of Rademacher and Normal distribution based trace estimators via boxplots using the relative errors observed in $100$ runs. $N$ and $\alpha$ denote the order of the tensor and the ratio of the sum of the square of diagonal entries and the square Frobenius norm of the tensor, respectively. The smaller interquartile range is an indication of a smaller variance.
• Figure 5: Comparison among the Rademacher and the normal distribution based trace estimators on the basis of mean absolute relative error over $100$ experimental runs for $N \in \{2,3,4\}$. $N$ and $\alpha$ denote the order of the tensor and the ratio of the sum of the square of diagonal entries and the square Frobenius norm of the tensor, respectively. The smaller value of mean absolute relative error indicates better estimates.

Theorems & Definitions (27)

• Theorem 1: Hutchinson Trace Estimator hutchinson1989stochastic
• Theorem 2: Diagonal entries estimator of a matrix bekas2007estimator
• Definition 3
• Definition 4
• Definition 5: Hadamard Product kolda2009tensor
• Definition 6: Diagonal Elements and trace of a Tensor kolda2009tensorqi2017tensor
• Definition 7: Mode-$n$ Tensor Vector Multiplication kolda2009tensor
• Definition 8: Rademacher Distribution
• Definition 9: $(\epsilon, \delta)$ -Approximator
• Theorem 10: Hypercontractivity Concentration Inequality schudy2012concentration