Conjugate Gradient Methods are Not Efficient: Experimental Study of the Locality Limitation

TL;DR

Conjugate Gradient convergence on sparse SPD systems is shown to be fundamentally constrained by locality: information can propagate at most one graph edge per iteration, so accurately capturing local quantities requires many iterations proportional to the graph diameter $D$ (e.g., $D\sim n$ in 1D). The authors perform an extensive experimental study on 1D and 2D discretizations, demonstrating stagnation of local quantities like $u(0)$ prior to the final iteration and showing that global residual improvement does not guarantee early local accuracy. They propose hierarchical, multilevel preconditioning to speed up information transport, achieving substantial iteration reductions (e.g., from 64 to 33 iterations with one level, and much larger improvements with multiple levels) that cannot be explained by changes in the condition number alone. The work highlights the practical importance of multilevel strategies for maintaining local accuracy and guiding future graph-based theoretical analyses of information propagation in iterative solvers.

Abstract

The convergence of the Conjugate Gradient method is subject to a locality limitation which imposes a lower bound on the number of iterations required before a qualitatively accurate approximation can be obtained. This limitation originates from the restricted transport of information in the graph induced by the sparsity pattern of the system matrix. In each iteration, information from the right-hand side can propagate only across directly connected graph nodes. The diameter of this graph therefore determines a minimum number of iterations that is necessary to achieve an acceptable level of accuracy.

Figures (20)

• Figure 1: The normalized residual when solving (\ref{['eq:example1']}) is displayed depending on the progress of the CG iterations. The orange dotted line is the upper bound of the convergence from eq. (\ref{['eq:conv_bound']}) with $C=1$.
• Figure 1: The reduction of normalized residual for example (\ref{['eq:example1']}) with $f(x)=10$ solved with CG iterations. Additionally the error in the left endpoint is shown. Note again the sudden drop of the residual norm and also the error when $k=n$. \newlabelfig:exa2-convergence0
• Figure 1: Exact solution of eq. (\ref{['eq:poisson']}) displayed on a grid with $32 \times 8$ mesh nodes. \newlabelfig:poisson-exact0
• Figure 1: CG with and without hierarchical 1-level preconditioner for the solution of (\ref{['eq:tridiag']}) with $\gamma=2$ ($d$ accordingly). The iterates are displayed in comparison, showing that the preconditioned iteration delivers almost the same intermediate results in about half the number of iterations. A solution visually coinciding with the exact solution is reached in 33 iterations, rather than 64 for the unpreconditioned system.
• Figure 2: Same as Fig. \ref{['fig:convergence_cg_plain']} showing the reduction of the normalized residual for example (\ref{['eq:example1']}) with $f(x)=0$, now in semilogarithmic scale. Note the sudden drop to $|| {\bf r}_n || \leq 10^{-13}$ occurring in the last iteration. Additionally, the error at the left endpoint is displayed (marked by green + symbols).

Theorems & Definitions (19)

• Remark 2.1
• Remark 2.2
• Remark 2.3
• Remark 2.4
• Remark 2.5
• Remark 3.1
• Remark 3.2
• Remark 3.3
• Remark 3.4
• Remark 3.5