lrAA: Low-Rank Anderson Acceleration

TL;DR

The authors address solving nonlinear matrix equations $G(X)=X$ that arise from spatial discretizations of PDEs by developing lrAA, a low-rank adaptation of Anderson acceleration that operates on $X$ in its factorized form and applies rank truncation with adaptive tolerance. A key innovation is Cross-DEIM, an iterative, warm-started cross-approximation method that efficiently approximates nonlinear terms, enabling sublinear cost per iteration when $X$ remains low-rank. The combination of truncation scheduling, Cross-DEIM, and low-rank operations yields faster convergence and reduced memory usage across linear and nonlinear tests, including Laplace, Bratu, elliptic Monge–Ampère, and Allen–Cahn equations, with preconditioning (ES) further enhancing performance on large grids. The work demonstrates a viable pathway to scalable, low-rank solvers for nonlinear PDE discretizations, with potential extensions to tensor formats and time-dependent problems.

Abstract

This paper proposes a new framework for computing low-rank solutions to nonlinear matrix equations arising from spatial discretization of nonlinear partial differential equations: low-rank Anderson acceleration (lrAA). lrAA is an adaptation of Anderson acceleration (AA), a well-known approach for solving nonlinear fixed point problems, to the low-rank format. In particular, lrAA carries out all linear and nonlinear operations in low-rank form with rank truncation using an adaptive truncation tolerance. We propose a simple scheduling strategy to update the truncation tolerance throughout the iteration according to a residual indicator. This controls the intermediate rank and iteration number effectively. To perform rank truncation for nonlinear functions, we propose a new cross approximation, which we call Cross-DEIM, with adaptive error control that is based on the discrete empirical interpolation method (DEIM). Cross-DEIM employs an iterative update between the approximate singular value decomposition (SVD) and cross approximation. It naturally incorporates a warm-start strategy for each lrAA iterate. We demonstrate the superior performance of lrAA applied to a range of linear and nonlinear problems, including those arising from finite difference discretizations of Laplace's equation, the Bratu problem, the elliptic Monge-Ampére equation and the Allen-Cahn equation.

Figures (16)

• Figure 1: Results for approximation of $G_1$. In the figures, 'CD' denotes Cross-DEIM.
• Figure 2: Results for approximation of $G_2$. In the figures, 'CD' denotes Cross-DEIM.
• Figure 3: Results for parametric matrix approximation: $H_1$ in terms of iteration number (left figure) and max intermediate rank (right figure). Blue: cold-start, red: warm-start. Black line: numerical rank (the axis is to the right of the figures).
• Figure 4: Results for parametric matrix approximation: $H_2$ in terms of iteration number (left figure) and max intermediate rank (right figure). Blue: cold-start, red: warm-start. Black line: numerical rank (the axis is to the right of the figures).
• Figure 5: Laplace's equation solved by lrAA. In this figure we display the results obtained by the scheduling $\epsilon_{k+1} = \theta \rho_k$. From left to right $\theta$ takes the values 1, 0.5, 0.2, and 0.1. Here all results are obtained with window size size 5 and $m = n = 31$. Results for other grid sizes are qualitatively similar.