Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The $φ$-PCA Framework: A Unified and Efficiency-Preserving Approach with Robust Variants

Hung Hung, Zhi-Yu Jou, Su-Yun Huang, Shinto Eguchi

TL;DR

The paper introduces φ-PCA, a unifying framework that partitions data into $m$ random blocks and aggregates local covariances via a φ-mean to perform PCA. It rigorously shows that φ-PCA preserves the asymptotic efficiency of standard PCA while enabling ordering-robustness, with HM-PCA (where $φ(u)=u^{-1}$) identified as optimally robust for ordering when outliers lie outside the signal subspace. The authors provide detailed perturbation analyses, establish that efficiency is invariant to $(m,φ)$, and derive practical guidelines for choosing $m$ (suggesting $m=\lfloor\sqrt{n}\rfloor$). Numerical simulations and MNIST data demonstrate that HM-PCA and GM-PCA achieve robustness to outliers without sacrificing accuracy on clean data, outperforming conventional PCA and AM-PCA in contaminated scenarios. The work also discusses limitations in high-dimensional regimes and positions the partition-aggregation principle as a broader strategy for robust methodology beyond PCA.

Abstract

Principal component analysis (PCA) is a fundamental tool in multivariate statistics, yet its sensitivity to outliers and limitations in distributed environments restrict its effectiveness in modern large-scale applications. To address these challenges, we introduce the $φ$-PCA framework which provides a unified formulation of robust and distributed PCA. The class of $φ$-PCA methods retains the asymptotic efficiency of standard PCA, while aggregating multiple local estimates using a proper $φ$ function enhances ordering-robustness, leading to more accurate eigensubspace estimation under contamination. Notably, the harmonic mean PCA (HM-PCA), corresponding to the choice $φ(u)=u^{-1}$, achieves optimal ordering-robustness and is recommended for practical use. Theoretical results further show that robustness increases with the number of partitions, a phenomenon seldom explored in the literature on robust or distributed PCA. Altogether, the partition-aggregation principle underlying $φ$-PCA offers a general strategy for developing robust and efficiency-preserving methodologies applicable to both robust and distributed data analysis.

The $φ$-PCA Framework: A Unified and Efficiency-Preserving Approach with Robust Variants

TL;DR

The paper introduces φ-PCA, a unifying framework that partitions data into random blocks and aggregates local covariances via a φ-mean to perform PCA. It rigorously shows that φ-PCA preserves the asymptotic efficiency of standard PCA while enabling ordering-robustness, with HM-PCA (where ) identified as optimally robust for ordering when outliers lie outside the signal subspace. The authors provide detailed perturbation analyses, establish that efficiency is invariant to , and derive practical guidelines for choosing (suggesting ). Numerical simulations and MNIST data demonstrate that HM-PCA and GM-PCA achieve robustness to outliers without sacrificing accuracy on clean data, outperforming conventional PCA and AM-PCA in contaminated scenarios. The work also discusses limitations in high-dimensional regimes and positions the partition-aggregation principle as a broader strategy for robust methodology beyond PCA.

Abstract

Principal component analysis (PCA) is a fundamental tool in multivariate statistics, yet its sensitivity to outliers and limitations in distributed environments restrict its effectiveness in modern large-scale applications. To address these challenges, we introduce the -PCA framework which provides a unified formulation of robust and distributed PCA. The class of -PCA methods retains the asymptotic efficiency of standard PCA, while aggregating multiple local estimates using a proper function enhances ordering-robustness, leading to more accurate eigensubspace estimation under contamination. Notably, the harmonic mean PCA (HM-PCA), corresponding to the choice , achieves optimal ordering-robustness and is recommended for practical use. Theoretical results further show that robustness increases with the number of partitions, a phenomenon seldom explored in the literature on robust or distributed PCA. Altogether, the partition-aggregation principle underlying -PCA offers a general strategy for developing robust and efficiency-preserving methodologies applicable to both robust and distributed data analysis.
Paper Structure (15 sections, 6 theorems, 33 equations, 6 figures)

This paper contains 15 sections, 6 theorems, 33 equations, 6 figures.

Table of Contents

  1. Introduction
  2. The $\bm\phi$-PCA Framework
  3. The Ordering-Robustness of $\bm\phi$-PCA
  4. Perturbation and stability analysis of $\bm\phi$-PCA
  5. The optimal ordering-robustness of HM-PCA
  6. Determination of the partition size $\bm m$
  7. Insights into the Ordering-Robustness of $\bm\phi$-PCA
  8. Effects of random partitioning and $\bm\phi$-aggregation
  9. Comparison of HM-PCA and GM-PCA
  10. Comparison of GM-PCA and PPCA
  11. Numerical Studies
  12. Simulation settings
  13. Simulation results
  14. Data Analysis
  15. Conclusions

Key Result

Theorem 1

Assume that $\phi$ is positive, strictly monotone and has a continuous second derivative. Let $\xi=({\rm diag}(\Lambda)^\top,{\rm vec}(\Gamma)^\top)^\top$ and $\widehat{\xi}_\phi^{(m)}=({\rm diag}(\widehat{\Lambda})^\top,{\rm vec}(\widehat{\Gamma})^\top)^\top$. Then, for fixed $m$ and $p$, as $n\to\ where ${\boldsymbol V}=H^\top{\rm cov}\{{\rm vec}(XX^\top)\}H$ with $H=[\gamma_1\otimes\gamma_1,\ld

Figures (6)

  • Figure 1: The means of similarity measure $s_q$, $q\in\{r,r+1,\ldots, 50\}$ of HM-PCA, GM-PCA, AM-PCA, GM-PCA2, PPCA, PCA, and opt-PCA under $r=10$, $n=400$, $p=200$, and different combinations of $\pi\in\{0.05, 0.1\}$ and $\sigma_{\rm out}\in \{1,1000\}$.
  • Figure 2: The means of similarity measure $s_q$, $q\in\{r,r+1,\ldots, 50\}$ of HM-PCA, GM-PCA, AM-PCA, GM-PCA2, PPCA, PCA, and opt-PCA under $r=10$, $n=400$, $p=1000$, and different combinations of $\pi\in\{0.05, 0.1\}$ and $\sigma_{\rm out}\in \{1,1000\}$.
  • Figure 3: Reconstructed images of digits $1$ and $2$ with $r=50$ contamination setting (i) $\sigma_{\rm{out}}=300$ and $\pi \in \{0, 0.05, \ldots, 0.3 \}$.
  • Figure 4: Reconstructed images of digits $1$ and $2$ with $r=50$ contamination setting (ii) $\pi=0.1$ and $\sigma_{\rm{out}} \in \{100, 200, \ldots, 700 \}$.
  • Figure :
  • ...and 1 more figures

Theorems & Definitions (9)

  • Theorem 1: Efficiency-preserving asymptotic normality of $\phi$-PCA
  • Remark 1
  • Theorem 2: Perturbation of $\phi$-PCA vs. standard PCA
  • Remark 2
  • Remark 3: Geometric mean case
  • Theorem 3: Asymptotic expansion of ordering-robustness for $\phi$-PCA
  • Corollary 4
  • Theorem 5: Ordering-robustness gains of HM-, GM-, and AM-PCA
  • Proposition 1: Hung and Huang, 2025