Statistical Rounding Error Analysis for Random Matrix Computations

TL;DR

This work replaces deterministic worst-case rounding error bounds with a probabilistic framework for random matrix computations, modeling relative errors as independent random variables bounded by the unit roundoff $u$. By deriving the mean and variance of rounding errors for inner products and extending to matrix-vector, matrix-matrix, and Wishart-related tasks, it provides approximate closed-form expressions that are shown to be two or more orders of magnitude tighter than traditional bounds. The approach is validated through extensive simulations across precisions and input distributions, including specific Wishart settings for ZF detection and LS problems, and demonstrates robust accuracy except in cases with highly dependent inputs where the model breaks down. The results offer tighter, statistically grounded insights into numerical stability for random matrices in wireless, signal processing, and machine learning contexts, potentially guiding algorithm design and precision selection.

Abstract

The conventional rounding error analysis provides worst-case bounds with an associated failure probability and ignores the statistical property of the rounding errors. In this paper, we develop a new statistical rounding error analysis for random matrix computations. Such computations have numerous applications in the field of wireless communications, signal processing, and machine learning. By assuming the relative errors are independent random variables, we derive the approximate closed-form expressions for the expectation and variance of the rounding errors in various key computations for random matrices. Numerical experiments validate the accuracy of our derivations and demonstrate that our analytical expressions are generally at least two orders of magnitude tighter than alternative worst-case bounds, exemplified through the inner products.

Figures (7)

• Figure 1: Comparison between simulated variance and analytical variance, i.e., \ref{['eq:inner_e']}, of the rounding error for the computation in single precision of the inner product with different input distribution.
• Figure 2: Comparison between the analytical results and other worst-case bounds for the computation in single precision of the inner product with different input distributions. Here, $\lambda = 1$ and $\zeta = 10^{-16}$.
• Figure 3: Comparison between simulated variance and analytical variance of the rounding error for the computation in lower precision of the inner product with different input distribution with random Gaussian $\mathcal{N}\left(0,1\right)$ vectors.
• Figure 4: Computation in single precision of the inner product $s={\bf x}^T{\bf y}$ with dependent random vectors of dimension $n = 10^8$.
• Figure 5: Comparison between simulated autocorrelation matrix and analytical autocorrelation matrix, i.e., \ref{['eq:mm_var']}, of the rounding error for the computation in single precision of the matrix-matrix product with different dimensions using the second-row second-column element ${\bf R}_{\Delta {\bf C}}(2,2)$ as an example.

Theorems & Definitions (14)

• Definition 1: Probabilistic floating-point arithmetic constantinides2021rigorous
• Lemma 1: Expectation and variance of products of random variables frishman1975arithmetic
• Lemma 2: Expectation and variance of random variables satisfying Wishart distribution
• Lemma 3: Expectation and variance of random variables satisfying student distribution
• Lemma 4: Expectation and variance of elements in random matrices satisfying Wishart distribution gupta2018matrix
• Lemma 5: Bienaymé--Chebyshev inequality heyde2012ij
• Theorem 1: Inner products
• Corollary 1: Probabilistic bounds for inner products
• Theorem 2: Matrix-vector products
• Theorem 3: Matrix-matrix products