Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Probabilistic Examination of Least Squares Error in Low-bitwidth Cholesky Decomposition

Alexander Osinsky, Roman Bychkov, Mikhail Trefilov, Vladimir Lyashev, Andrey Ivanov

TL;DR

This paper addresses the impact of low-bitwidth arithmetic on linear LS solutions used in massive MIMO detectors, focusing on Cholesky-based computations of $X = (H^H H)^{-1} H^H Y$. It introduces a probabilistic bound by deriving a tighter scalar-product bound and modeling round-off as Gaussian within the RANDSVD channel ensemble, linking the LS error to the conditioning of $H$. The authors show the bound closely tracks the observed half-precision LS error (within about $1$ dB), demonstrating practical usefulness for precision planning and allocation in LS detectors. The work provides a framework to reduce precision where possible while maintaining performance, enhancing computational efficiency in large-scale MIMO systems. Overall, the method enables principled bitwidth optimization for Cholesky-based LS solutions in communications and related applications.

Abstract

In this paper, we propose a new approach to justify a round-off error impact on the accuracy of the linear least squares (LS) solution using Cholesky decomposition. This decomposition is widely employed to inverse a matrix in the linear detector of the Multi-User multi-antenna receiver. The proposed stochastic bound is much closer to actual errors than other numerical bounds. It was tested with a half-precision format and validated in realistic scenarios. Experimental results demonstrate our approach predicts errors very close to those achieved by simulations. The proposed approach can be employed to analyze the resulting round-off error in many other applications.

Probabilistic Examination of Least Squares Error in Low-bitwidth Cholesky Decomposition

TL;DR

This paper addresses the impact of low-bitwidth arithmetic on linear LS solutions used in massive MIMO detectors, focusing on Cholesky-based computations of . It introduces a probabilistic bound by deriving a tighter scalar-product bound and modeling round-off as Gaussian within the RANDSVD channel ensemble, linking the LS error to the conditioning of . The authors show the bound closely tracks the observed half-precision LS error (within about dB), demonstrating practical usefulness for precision planning and allocation in LS detectors. The work provides a framework to reduce precision where possible while maintaining performance, enhancing computational efficiency in large-scale MIMO systems. Overall, the method enables principled bitwidth optimization for Cholesky-based LS solutions in communications and related applications.

Abstract

In this paper, we propose a new approach to justify a round-off error impact on the accuracy of the linear least squares (LS) solution using Cholesky decomposition. This decomposition is widely employed to inverse a matrix in the linear detector of the Multi-User multi-antenna receiver. The proposed stochastic bound is much closer to actual errors than other numerical bounds. It was tested with a half-precision format and validated in realistic scenarios. Experimental results demonstrate our approach predicts errors very close to those achieved by simulations. The proposed approach can be employed to analyze the resulting round-off error in many other applications.
Paper Structure (11 sections, 7 theorems, 47 equations, 2 figures)

This paper contains 11 sections, 7 theorems, 47 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Problem definition
  3. Contribution
  4. Known results
  5. Better scalar product bound
  6. More assumptions
  7. Cholesky decomposition error
  8. Effect of round-off error on the LS detector
  9. Simulation results
  10. Conclusion
  11. Acknowledgment

Key Result

Theorem 1

If Cholesky decomposition of $A \in \mathbb{R}^{N \times N}$ with rounding to the nearest runs to completion and outputs some factors $\tilde{L}$ and $\tilde{L}^H$, then where $u = 2^{-b-1}$, $b$ is the number of mantissa bits.

Figures (2)

  • Figure 1: Dependency of Cholesky round-off error on condition number for 32x32 matrix H.
  • Figure 2: Dependency of Cholesky round-off error on condition number for 64x12 matrices H from RANDSVD ensemble and QuaDRiGa simulations.

Theorems & Definitions (12)

  • Theorem 1: accuracy, Theorem 10.3
  • Theorem 2: HighamPHD
  • Remark 1
  • Definition 1
  • Theorem 3: Binet-Cauchy
  • Corollary 1
  • Proposition 1
  • proof
  • Definition 2
  • Proposition 2: GaussProduct
  • ...and 2 more