Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Random Walks, Faber Polynomials and Accelerated Power Methods

Peter Cowal, Nicholas F. Marshall, Sara Pollock

TL;DR

The paper develops a family of polynomials defined by mean-zero random-walk recurrences that bound $P_n$ on a radially convex stability region and grow rapidly beyond the region, enabling $z^n$ to be well-approximated by a degree-$\sim$ $\sqrt{n}$ polynomial. It connects these polynomials to Faber polynomials and analyzes the geometry of the stability curve, with cusps playing a key role in enabling rapid growth. Leveraging these polynomials, the authors design static and dynamic momentum-accelerated power iterations that extend momentum methods to a wide class of nonsymmetric matrices and provide convergence guarantees under spectral containment assumptions. They complement the theory with extensive numerical experiments on toy, barbell, and large sparse matrices, showing improved convergence over classical Chebyshev- or standard power-method approaches, especially when the subdominant spectrum lies inside the stability region. The work offers a principled framework for adaptive acceleration in iterative linear algebra and suggests broader applicability to other nonsymmetric iterative schemes through the lens of complex-plane polynomial approximation.

Abstract

In this paper, we construct families of polynomials defined by recurrence relations related to mean-zero random walks. We show these families of polynomials can be used to approximate $z^n$ by a polynomial of degree $\sim \sqrt{n}$ in associated radially convex domains in the complex plane. Moreover, we show that the constructed families of polynomials have a useful rapid growth property and a connection to Faber polynomials. Applications to iterative linear algebra are presented, including the development of arbitrary-order dynamic momentum power iteration methods suitable for classes of non-symmetric matrices.

Random Walks, Faber Polynomials and Accelerated Power Methods

TL;DR

The paper develops a family of polynomials defined by mean-zero random-walk recurrences that bound on a radially convex stability region and grow rapidly beyond the region, enabling to be well-approximated by a degree- polynomial. It connects these polynomials to Faber polynomials and analyzes the geometry of the stability curve, with cusps playing a key role in enabling rapid growth. Leveraging these polynomials, the authors design static and dynamic momentum-accelerated power iterations that extend momentum methods to a wide class of nonsymmetric matrices and provide convergence guarantees under spectral containment assumptions. They complement the theory with extensive numerical experiments on toy, barbell, and large sparse matrices, showing improved convergence over classical Chebyshev- or standard power-method approaches, especially when the subdominant spectrum lies inside the stability region. The work offers a principled framework for adaptive acceleration in iterative linear algebra and suggests broader applicability to other nonsymmetric iterative schemes through the lens of complex-plane polynomial approximation.

Abstract

In this paper, we construct families of polynomials defined by recurrence relations related to mean-zero random walks. We show these families of polynomials can be used to approximate by a polynomial of degree in associated radially convex domains in the complex plane. Moreover, we show that the constructed families of polynomials have a useful rapid growth property and a connection to Faber polynomials. Applications to iterative linear algebra are presented, including the development of arbitrary-order dynamic momentum power iteration methods suitable for classes of non-symmetric matrices.

Paper Structure

This paper contains 25 sections, 21 theorems, 132 equations, 8 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Related work
  4. Main contributions
  5. Preliminaries
  6. Stability regions in the complex plane
  7. Hypocycloids
  8. Main analytic results
  9. Momentum accelerated power iterations
  10. Notation and assumptions
  11. The Generalized Momentum Power Method
  12. The dynamic generalized momentum power method
  13. Numerical Results
  14. Toy example
  15. Barbell example
  16. ...and 10 more sections

Key Result

Theorem 1.1

Let $\gamma$ be the closed curve parameterized by eq:curveformula, for a probability vector $p = (p_0,\ldots,p_m)$ satisfying meanzero with $p_1 = 0$. Then, $\gamma$ has no self-intersections and is continuously differentiable except at $n$ cusps located at $e^{2\pi i k/n}$ for $k \in \{0,\ldots,n-1 Moreover, the region enclosed by $\gamma$ is radially convex.

Figures (8)

  • Figure 1: The curve \ref{['eq:curveformula']} for $p = (\frac{7}{12},0,\frac{3}{12},\frac{2}{12})$, $(\frac{2}{3},0,0,\frac{1}{3})$, and $(\frac{5}{8},0,\frac{2}{8},0,\frac{1}{8})$ (ordered left to right).
  • Figure 2: The curve \ref{['eq:curveformula']} for $p = (\frac{2}{3},0,\frac{1}{3})$, $(\frac{3}{4},0,0,\frac{1}{4})$, and $(\frac{4}{5},0,0,0,\frac{1}{5})$ (ordered left to right). These curves are called a $3$-cusped hypocycloid (deltoid), a $4$-cusped hypocycloid (astroid), and a $5$-cusped hypocycloid, respectively.
  • Figure 3: $P_n^{(m)}(1+\varepsilon)$ for $\varepsilon = 10^{-5}$ for $n\in\{1,\ldots,1000\}$ and $m \in \{2,3,4,5\}$, along with the rates predicted by Theorem \ref{['thm:boundedrapid']}, where $P_n^{(m)}$ is the hypocycloid polynomial defined in \ref{['eqn:genrec']} and associated with the hypocycloid curves illustrated in Figure \ref{['fig:hypocycloids']}.
  • Figure 4: As a consequence of Theorem \ref{['ellipseslow']}, if a polynomial is bounded on the deltoid region, then rapid growth in the sense of \ref{['eq:defrapidgrowth']} is possible at $1$ but impossible at $-1/3$, since there is an ellipse contained in the deltoid region tangent to this point.
  • Figure 5: Left: convergence of hypocycloid momentum methods for varying degrees of hypocycloid for the toy example. Right: convergence of the dynamic momentum method with various probability distributions.
  • ...and 3 more figures

Theorems & Definitions (41)

  • Definition 1.1
  • Remark 1.1: Connection to Faber polynomials
  • Remark 1.2
  • Theorem 1.1: Description of stability region
  • Theorem 1.2: Boundedness and rapid growth
  • Theorem 1.3: Approximation of $z^n$ by polynomial of degree $\sim \sqrt{n}$
  • Theorem 1.4: Bound on growth of polynomial bounded on an ellipse
  • Remark 2.1
  • Lemma 2.1
  • proof : Proof of Lemma \ref{['lem:general-momentum-works']}
  • ...and 31 more