Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Probabilistic Analysis of Least Squares, Orthogonal Projection, and QR Factorization Algorithms Subject to Gaussian Noise

Ali Lotfi, Julien Langou, Mohammad Meysami

TL;DR

This work addresses the gap on the condition number increase for a matrix B without assuming perfect orthonormality, even when a column is not perfectly orthogonal to the span of B, by deriving bounds on the condition number increase for a matrix B without assuming perfect orthonormality.

Abstract

In this paper, we extend the work of Liesen et al. (2002), which analyzes how the condition number of an orthonormal matrix Q changes when a column is added ([Q, c]), particularly focusing on the perpendicularity of c to the span of Q. Their result, presented in Theorem 2.3 of Liesen et al. (2002), assumes exact arithmetic and orthonormality of Q, which is a strong assumption when applying these results to numerical methods such as QR factorization algorithms. In our work, we address this gap by deriving bounds on the condition number increase for a matrix B without assuming perfect orthonormality, even when a column is not perfectly orthogonal to the span of B. This framework allows us to analyze QR factorization methods where orthogonalization is imperfect and subject to Gaussian noise. We also provide results on the performance of orthogonal projection and least squares under Gaussian noise, further supporting the development of this theory.

Probabilistic Analysis of Least Squares, Orthogonal Projection, and QR Factorization Algorithms Subject to Gaussian Noise

TL;DR

This work addresses the gap on the condition number increase for a matrix B without assuming perfect orthonormality, even when a column is not perfectly orthogonal to the span of B, by deriving bounds on the condition number increase for a matrix B without assuming perfect orthonormality.

Abstract

In this paper, we extend the work of Liesen et al. (2002), which analyzes how the condition number of an orthonormal matrix Q changes when a column is added ([Q, c]), particularly focusing on the perpendicularity of c to the span of Q. Their result, presented in Theorem 2.3 of Liesen et al. (2002), assumes exact arithmetic and orthonormality of Q, which is a strong assumption when applying these results to numerical methods such as QR factorization algorithms. In our work, we address this gap by deriving bounds on the condition number increase for a matrix B without assuming perfect orthonormality, even when a column is not perfectly orthogonal to the span of B. This framework allows us to analyze QR factorization methods where orthogonalization is imperfect and subject to Gaussian noise. We also provide results on the performance of orthogonal projection and least squares under Gaussian noise, further supporting the development of this theory.
Paper Structure (12 sections, 14 theorems, 73 equations, 2 figures)

This paper contains 12 sections, 14 theorems, 73 equations, 2 figures.

Table of Contents

  1. Introduction
  2. History
  3. Overview
  4. Motivation and Problem Solving Strategies
  5. Notation
  6. Condition Number Growth Induced by Adding a Column Under No Assumption
  7. Probabilistic Analysis of the Norm of a Vector Subject to Gaussian Noise
  8. Numerical Test of Theorem \ref{['thm:Malcum-Bessel']}
  9. Orthogonal Projection Subject to Gaussian Noise
  10. Least Squares Subject to Gaussian Noise
  11. Imperfect Orthogonalization and Perfect Normalization in QR Factorization Algorithms
  12. Conclusion and Future Research

Key Result

Theorem 1.1

Suppose that $[c, B] \in \mathbb{R}^{N,n+1}$ has full column rank, and $r \neq 0$ is the residual of the least squares problem ($\|r\|_2 = \min_{u} \|c - Bu\|_2$). Let $B = QR$ be a QR-factorization of the matrix $B$, and let $\gamma > 0$ be a real parameter. Then: where $\|r\|_2 = \|c - By\|_2$. Furthermore, where $\alpha = 1 + \gamma^2 \|c\|_2^2$.

Figures (2)

  • Figure 1: Comparison of numerical versus theoretical results to test Theorem \ref{['thm:Malcum-Bessel']} ($\varepsilon = 0.9$). The red triangles represent the empirical data obtained from numerical experiments and the blue line corresponds to the theoretical predictions.
  • Figure 2: Comparison of numerical versus theoretical results to test Theorem \ref{['thm:Malcum-Bessel']} ($\varepsilon=1.5$).

Theorems & Definitions (29)

  • Theorem 1.1
  • Lemma 3.1
  • proof
  • Theorem 3.2
  • proof
  • Definition 4.1: Bessel function
  • Definition 4.2: Generalized Marcum-Q function
  • Theorem 4.3
  • proof
  • Corollary 4.4
  • ...and 19 more