Fast QR updating methods for statistical applications

TL;DR

This paper introduces fast R updating algorithms specifically designed for statistical applications, including regression, filtering, and model selection, where data structures change frequently, and achieves a substantial reduction in computational time without compromising accuracy.

Abstract

This paper introduces fast R updating algorithms specifically designed for statistical applications, including regression, filtering, and model selection, where data structures change frequently. Although traditional QR decomposition is essential for matrix operations, it becomes computationally intensive when dynamically updating the design matrix in statistical models. The proposed algorithms efficiently update the R matrix without the need for recalculation of Q, thereby significantly reducing computational costs in practical computational scenarios. The provision of scalable solutions for high-dimensional regression models is a key strength of these algorithms, enhancing the feasibility of large-scale statistical analyses and model selection in data-intensive fields. A thorough simulation study and the analysis of real-world data demonstrate that the methods achieve a substantial reduction in computational time without compromising accuracy. The discussion illustrates the benefits of these algorithms across a wide range of models and applications in statistics and machine learning.

Figures (20)

• Figure 1: Graphical representation of the effect of Givens rotations on $\widetilde{\mathbf{R}}$ in order to obtain the new $\mathbf{R}$. The boxed row/column are added cells, the column in gray is deleted. $\odot$ indicates cells that are zeroed, $\oplus$ indicates cells that were $0$ and after the update assume values different from $0$, finally $\times$ indicate cells whose value is modified from the starting one $(+)$ as a consequence of the update.
• Figure 2: Logarithm of the exact computational costs of adding (left panels) or deleting (right panels) 1, 5 or 10 columns or rows. Top row: $N=1000$ and $p \in \{20, 50, 100, 200, 500, 800\}$. Bottom row: $p=100$ and $N \in \{200, 500, 800, 1000, 2000, 5000\}$.
• Figure 3: Mean AUC with $1$ standard error bands of the MPM computed by the RJ (with R update) and the BoomSS algorithms. 40 repetitions for each setting with independent covariates.
• Figure 4: Mean F1 score with 1 standard errors bands of the MPM and the MaP model computed by the RJ (with R update) and the BoomSS algorithms. 40 repetitions for each setting with independent covariates.
• Figure 5: Logarithm of the mean computational time (in seconds) with $1$ standard error bands for $50,000$ draws from the RJ (with R update) and the BoomSS algorithms. 40 repetitions for each setting with independent covariates.

Theorems & Definitions (21)

• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof
• proof