Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Generalized Mean Approach for Distributed-PCA

Zhi-Yu Jou, Su-Yun Huang, Hung Hung, Shinto Eguchi

TL;DR

This article proposes a novel DPCA method that incorporates eigenvalue information to aggregate local results via the matrix $\beta$-mean, which is called $\beta$-DPCA, and studies the stability of eigenvector ordering under eigenvalue perturbation for $\beta$-DPCA.

Abstract

Principal component analysis (PCA) is a widely used technique for dimension reduction. As datasets continue to grow in size, distributed-PCA (DPCA) has become an active research area. A key challenge in DPCA lies in efficiently aggregating results across multiple machines or computing nodes due to computational overhead. Fan et al. (2019) introduced a pioneering DPCA method to estimate the leading rank-$r$ eigenspace, aggregating local rank-$r$ projection matrices by averaging. However, their method does not utilize eigenvalue information. In this article, we propose a novel DPCA method that incorporates eigenvalue information to aggregate local results via the matrix $β$-mean, which we call $β$-DPCA. The matrix $β$-mean offers a flexible and robust aggregation method through the adjustable choice of $β$ values. Notably, for $β=1$, it corresponds to the arithmetic mean; for $β=-1$, the harmonic mean; and as $β\to 0$, the geometric mean. Moreover, the matrix $β$-mean is shown to associate with the matrix $β$-divergence, a subclass of the Bregman matrix divergence, to support the robustness of $β$-DPCA. We also study the stability of eigenvector ordering under eigenvalue perturbation for $β$-DPCA. The performance of our proposal is evaluated through numerical studies.

A Generalized Mean Approach for Distributed-PCA

TL;DR

This article proposes a novel DPCA method that incorporates eigenvalue information to aggregate local results via the matrix -mean, which is called -DPCA, and studies the stability of eigenvector ordering under eigenvalue perturbation for -DPCA.

Abstract

Principal component analysis (PCA) is a widely used technique for dimension reduction. As datasets continue to grow in size, distributed-PCA (DPCA) has become an active research area. A key challenge in DPCA lies in efficiently aggregating results across multiple machines or computing nodes due to computational overhead. Fan et al. (2019) introduced a pioneering DPCA method to estimate the leading rank- eigenspace, aggregating local rank- projection matrices by averaging. However, their method does not utilize eigenvalue information. In this article, we propose a novel DPCA method that incorporates eigenvalue information to aggregate local results via the matrix -mean, which we call -DPCA. The matrix -mean offers a flexible and robust aggregation method through the adjustable choice of values. Notably, for , it corresponds to the arithmetic mean; for , the harmonic mean; and as , the geometric mean. Moreover, the matrix -mean is shown to associate with the matrix -divergence, a subclass of the Bregman matrix divergence, to support the robustness of -DPCA. We also study the stability of eigenvector ordering under eigenvalue perturbation for -DPCA. The performance of our proposal is evaluated through numerical studies.
Paper Structure (12 sections, 4 theorems, 15 equations, 1 figure, 1 table, 3 algorithms)

This paper contains 12 sections, 4 theorems, 15 equations, 1 figure, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Method
  3. DPCA via $\bm\beta$-mean aggregation
  4. Selection of ${\boldsymbol \beta}$
  5. Theoretical Properties
  6. Connection to matrix $\beta$-mean
  7. Connection to matrix ${\boldsymbol \beta}$-divergence
  8. Stability of eigenvector ordering
  9. Simulation Studies
  10. Simulation settings
  11. Results
  12. Conclusion

Key Result

Proposition 3

The limiting cases of the matrix $\beta$-divergence are summarized below. (i) We have $\lim_{\beta\to 0} \Phi_{\beta}(\mathbf{M})= {\rm tr} \left(\mathbf{M} \ln\left(\mathbf{M} \right) -\mathbf{M} \right) + p$, and which is the von Neuman matrix divergence and is denoted as $D_{\rm vN}(\mathbf{M}_{1},\mathbf{M}_{2})$. (ii) We have $\lim_{\beta\to -1} \Phi_{\beta}(\mathbf{M}) = -\ln {\rm det} (\ma

Figures (1)

  • Figure 1: The mean similarity measure $\rho_{k}$ for $k\in\{ r, r+1, \ldots, 15\}$ under $r=5$, $n=250$, $p\in\{500,1000\}$ and $m\in\{5,10\}$.

Theorems & Definitions (8)

  • Remark 1
  • Definition 1: Matrix $\beta$-mean
  • Remark 2: limiting case of $\beta\to 0$
  • Definition 2: Matrix $\beta$-divergence
  • Proposition 3: Limiting cases
  • Proposition 4: Minimum matrix $\beta$-divergence estimation
  • Corollary 5
  • Proposition 6