Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Rapid Grassmannian Averaging with Chebyshev Polynomials

Brighton Ancelin, Alex Saad-Falcon, Kason Ancelin, Justin Romberg

TL;DR

These proposed algorithms, Rapid Grassmannian Averaging (RGrAv) and Decentralized Rapid Grassmannian Averaging (DRGrAv), overcome the spectral structure of the problem to rapidly compute an average using only small matrix multiplications and QR factorizations.

Abstract

We propose new algorithms to efficiently average a collection of points on a Grassmannian manifold in both the centralized and decentralized settings. Grassmannian points are used ubiquitously in machine learning, computer vision, and signal processing to represent data through (often low-dimensional) subspaces. While averaging these points is crucial to many tasks (especially in the decentralized setting), existing methods unfortunately remain computationally expensive due to the non-Euclidean geometry of the manifold. Our proposed algorithms, Rapid Grassmannian Averaging (RGrAv) and Decentralized Rapid Grassmannian Averaging (DRGrAv), overcome this challenge by leveraging the spectral structure of the problem to rapidly compute an average using only small matrix multiplications and QR factorizations. We provide a theoretical guarantee of optimality and present numerical experiments which demonstrate that our algorithms outperform state-of-the-art methods in providing high accuracy solutions in minimal time. Additional experiments showcase the versatility of our algorithms to tasks such as K-means clustering on video motion data, establishing RGrAv and DRGrAv as powerful tools for generic Grassmannian averaging.

Rapid Grassmannian Averaging with Chebyshev Polynomials

TL;DR

These proposed algorithms, Rapid Grassmannian Averaging (RGrAv) and Decentralized Rapid Grassmannian Averaging (DRGrAv), overcome the spectral structure of the problem to rapidly compute an average using only small matrix multiplications and QR factorizations.

Abstract

We propose new algorithms to efficiently average a collection of points on a Grassmannian manifold in both the centralized and decentralized settings. Grassmannian points are used ubiquitously in machine learning, computer vision, and signal processing to represent data through (often low-dimensional) subspaces. While averaging these points is crucial to many tasks (especially in the decentralized setting), existing methods unfortunately remain computationally expensive due to the non-Euclidean geometry of the manifold. Our proposed algorithms, Rapid Grassmannian Averaging (RGrAv) and Decentralized Rapid Grassmannian Averaging (DRGrAv), overcome this challenge by leveraging the spectral structure of the problem to rapidly compute an average using only small matrix multiplications and QR factorizations. We provide a theoretical guarantee of optimality and present numerical experiments which demonstrate that our algorithms outperform state-of-the-art methods in providing high accuracy solutions in minimal time. Additional experiments showcase the versatility of our algorithms to tasks such as K-means clustering on video motion data, establishing RGrAv and DRGrAv as powerful tools for generic Grassmannian averaging.

Paper Structure

This paper contains 16 sections, 4 theorems, 36 equations, 3 figures, 1 table, 8 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Averaging Subspaces
  5. Averaging Subspaces in a Decentralized Network
  6. The Power Method
  7. Methods
  8. Motivation
  9. RGrAv Algorithms
  10. Experiments
  11. Decentralized Grassmannian Averaging
  12. K-Means for Video Motion Clustering
  13. Reproducibility Statement
  14. Additional Algorithms
  15. Auxiliary Theorems
  16. ...and 1 more sections

Key Result

Theorem 1

For $t \geq 1$, the minimization problem where $\mathcal{P}_t'$ is the set of $t$th order polynomials such that $f_t(0) = 0$ and $f_t(1) = 1$, is solved by which is a modification of a Chebyshev polynomial of the first kind.A proof may be found in apx:thm_main

Figures (3)

  • Figure 1: A visual comparison between the power method and our method. Each method's corresponding $t$th-order polynomial is applied to the eigenvalues $\lambda$ in the domain $\PDs*{0,1}$. The Chebyshev recursion with threshold parameter $\alpha=0.5$ results in the polynomial oscillations being reduced and flattened in the range $[0, \alpha]$.
  • Figure 2: Plots of Mean Squared Error/Disagreement for the example decentralized Grassmannian averaging problem. DRGrAv is our proposed algorithm, DeEPCA is from deepca, DPRGD and DPRGT are from dprgd_dprgt, COM is from sarlette2009consensus, and Gossip is from gossip. The units of the x axes are communication rounds, not algorithm iterations.
  • Figure 3: A comparison of runtime for K-means with various averaging algorithms and numbers of clusters K. The four colors represent the averaging algorithm as RGrAv (green), flag mean (orange), Fréchet mean (blue), and power method (pink). The four algorithms produce clusters with similar quality (excluded for brevity), but the RGrAv algorithm is significantly faster, showing $2\times$-$10\times$ speedup over the other averaging algorithms.

Theorems & Definitions (8)

  • Theorem 1
  • Lemma 2
  • proof
  • Definition 1
  • Lemma 3
  • proof
  • Lemma 4
  • proof