Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

High-precision randomized iterative methods for the random feature method

Jingrun Chen, Longze Tan

TL;DR

The CSQRP-LSQR and CSSVDP-LSQR methods are developed, which open up the applicability of the random feature method for PDEs over complicated geometries with high-complexity solutions.

Abstract

This paper focuses on solving large-scale, ill-conditioned, and overdetermined sparse least squares problems that arise from numerical partial differential equations (PDEs), mainly from the random feature method. To address these difficulties, we introduce (1) a count sketch technique to sketch the original matrix to a smaller matrix; (2) a QR factorization or a singular value decomposition for the smaller matrix to obtain the preconditioner, which is multiplied to the original matrix from the right-hand side; (3) least squares iterative solvers to solve the preconditioned least squares system. Therefore, the methods we develop are termed CSQRP-LSQR and CSSVDP-LSQR. Under mild assumptions, we prove that the preconditioned problem holds a condition number whose upper bound is independent of the condition number of the original matrix, and provide error estimates for both methods. Ample numerical experiments, including least squares problems arising from two-dimensional and three-dimensional PDEs and the Florida Sparse Matrix Collection, are conducted. Both methods are comparable to or even better than direct methods in accuracy and are computationally more efficient for large-scale problems. This opens up the applicability of the random feature method for PDEs over complicated geometries with high-complexity solutions.

High-precision randomized iterative methods for the random feature method

TL;DR

The CSQRP-LSQR and CSSVDP-LSQR methods are developed, which open up the applicability of the random feature method for PDEs over complicated geometries with high-complexity solutions.

Abstract

This paper focuses on solving large-scale, ill-conditioned, and overdetermined sparse least squares problems that arise from numerical partial differential equations (PDEs), mainly from the random feature method. To address these difficulties, we introduce (1) a count sketch technique to sketch the original matrix to a smaller matrix; (2) a QR factorization or a singular value decomposition for the smaller matrix to obtain the preconditioner, which is multiplied to the original matrix from the right-hand side; (3) least squares iterative solvers to solve the preconditioned least squares system. Therefore, the methods we develop are termed CSQRP-LSQR and CSSVDP-LSQR. Under mild assumptions, we prove that the preconditioned problem holds a condition number whose upper bound is independent of the condition number of the original matrix, and provide error estimates for both methods. Ample numerical experiments, including least squares problems arising from two-dimensional and three-dimensional PDEs and the Florida Sparse Matrix Collection, are conducted. Both methods are comparable to or even better than direct methods in accuracy and are computationally more efficient for large-scale problems. This opens up the applicability of the random feature method for PDEs over complicated geometries with high-complexity solutions.
Paper Structure (16 sections, 5 theorems, 63 equations, 17 figures, 32 tables, 2 algorithms)

This paper contains 16 sections, 5 theorems, 63 equations, 17 figures, 32 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. The random feature method
  4. The $\ell_2$-subspace embedding
  5. The proposed methods
  6. Numerical experiments
  7. Setup and notation
  8. Condition number, oversampling factor, randomized and Krylov methods
  9. Condition number and oversampling factor
  10. Results of randomized iterative and Krylov subspace methods
  11. Three-dimensional PDEs
  12. Least squares problems with rank deficiency or infinite condition number
  13. Concluding remarks
  14. Results for PDEs without explicit solutions
  15. Hyper-parameters in the RFM
  16. ...and 1 more sections

Key Result

Lemma 2.1

(MM) Suppose that $\mathbf{S} \in \mathrm{R}^{s \times m}$ is a count sketch transform with $s=\left(n^2+n\right) /\left(\delta \varepsilon^2\right)$, where $0<\delta, \epsilon<1$. Then with probability at least $1-\delta$, we have that the inequality L2_subspaace embedding holds.

Figures (17)

  • Figure 1: The Timoshenko beam problem.
  • Figure 2: Sparsity pattern. Left: $\mathbf{A}$, middle: $\mathbf{S}\mathbf{A}$, right: $\mathbf{G}\mathbf{A}$.
  • Figure 3: Singular value distribution of $\mathbf{A}$, $\mathbf{A}\mathbf{R}^{-1}$, and $\mathbf{AP}$ for the Timoshenko beam problem.
  • Figure 4: Distribution of absolute errors in $u$, $v$, and $p$ for two-dimensional Stokes flow problem using the CSSVDP-LSQR method with $\gamma=3$.
  • Figure 5: A two-dimensional complex geometry.
  • ...and 12 more figures

Theorems & Definitions (29)

  • Definition 2.1
  • Definition 2.2
  • Definition 2.3
  • Lemma 2.1
  • Remark 3.1
  • Theorem 3.1
  • proof
  • Theorem 3.2
  • proof
  • Theorem 3.3
  • ...and 19 more