Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Graph Regularized PCA

Antonio Briola, Marwin Schmidt, Fabio Caccioli, Carlos Ros Perez, James Singleton, Christian Michler, Tomaso Aste

TL;DR

Graph-regularized PCA (GR-PCA) extends classical PCA to structured data by learning a sparse precision graph over features and adding a graph-Laplacian penalty to bias loadings toward low-frequency, graph-consistent patterns. The approach solves a joint objective that balances reconstruction accuracy, sparsity, and smoothness across the learned graph, enabling structure-aware dimensionality reduction that improves interpretability and robustness to anisotropic noise. Through a synthetic data generator and systematic experiments across ER/BA/WS topologies, GR-PCA demonstrates higher selectivity and alignment to ground-truth low-frequency components while maintaining competitive out-of-sample reconstruction, especially when high-frequency signals align with graph structure. The method is simple, scalable, and adaptable to learned or oracle precision graphs, offering a practical route to structure-aware PCA in domains like neuroscience, climate science, and genomics.

Abstract

High-dimensional data often exhibit dependencies among variables that violate the isotropic-noise assumption under which principal component analysis (PCA) is optimal. For cases where the noise is not independent and identically distributed across features (i.e., the covariance is not spherical) we introduce Graph Regularized PCA (GR-PCA). It is a graph-based regularization of PCA that incorporates the dependency structure of the data features by learning a sparse precision graph and biasing loadings toward the low-frequency Fourier modes of the corresponding graph Laplacian. Consequently, high-frequency signals are suppressed, while graph-coherent low-frequency ones are preserved, yielding interpretable principal components aligned with conditional relationships. We evaluate GR-PCA on synthetic data spanning diverse graph topologies, signal-to-noise ratios, and sparsity levels. Compared to mainstream alternatives, it concentrates variance on the intended support, produces loadings with lower graph-Laplacian energy, and remains competitive in out-of-sample reconstruction. When high-frequency signals are present, the graph Laplacian penalty prevents overfitting, reducing the reconstruction accuracy but improving structural fidelity. The advantage over PCA is most pronounced when high-frequency signals are graph-correlated, whereas PCA remains competitive when such signals are nearly rotationally invariant. The procedure is simple to implement, modular with respect to the precision estimator, and scalable, providing a practical route to structure-aware dimensionality reduction that improves structural fidelity without sacrificing predictive performance.

Graph Regularized PCA

TL;DR

Graph-regularized PCA (GR-PCA) extends classical PCA to structured data by learning a sparse precision graph over features and adding a graph-Laplacian penalty to bias loadings toward low-frequency, graph-consistent patterns. The approach solves a joint objective that balances reconstruction accuracy, sparsity, and smoothness across the learned graph, enabling structure-aware dimensionality reduction that improves interpretability and robustness to anisotropic noise. Through a synthetic data generator and systematic experiments across ER/BA/WS topologies, GR-PCA demonstrates higher selectivity and alignment to ground-truth low-frequency components while maintaining competitive out-of-sample reconstruction, especially when high-frequency signals align with graph structure. The method is simple, scalable, and adaptable to learned or oracle precision graphs, offering a practical route to structure-aware PCA in domains like neuroscience, climate science, and genomics.

Abstract

High-dimensional data often exhibit dependencies among variables that violate the isotropic-noise assumption under which principal component analysis (PCA) is optimal. For cases where the noise is not independent and identically distributed across features (i.e., the covariance is not spherical) we introduce Graph Regularized PCA (GR-PCA). It is a graph-based regularization of PCA that incorporates the dependency structure of the data features by learning a sparse precision graph and biasing loadings toward the low-frequency Fourier modes of the corresponding graph Laplacian. Consequently, high-frequency signals are suppressed, while graph-coherent low-frequency ones are preserved, yielding interpretable principal components aligned with conditional relationships. We evaluate GR-PCA on synthetic data spanning diverse graph topologies, signal-to-noise ratios, and sparsity levels. Compared to mainstream alternatives, it concentrates variance on the intended support, produces loadings with lower graph-Laplacian energy, and remains competitive in out-of-sample reconstruction. When high-frequency signals are present, the graph Laplacian penalty prevents overfitting, reducing the reconstruction accuracy but improving structural fidelity. The advantage over PCA is most pronounced when high-frequency signals are graph-correlated, whereas PCA remains competitive when such signals are nearly rotationally invariant. The procedure is simple to implement, modular with respect to the precision estimator, and scalable, providing a practical route to structure-aware dimensionality reduction that improves structural fidelity without sacrificing predictive performance.
Paper Structure (15 sections, 12 equations, 2 figures, 4 tables)

This paper contains 15 sections, 12 equations, 2 figures, 4 tables.

Table of Contents

  1. Abstract
  2. Introduction
  3. Methodology
  4. Data
  5. Step 1: Definition of the features' structure
  6. Step 2: Generation of true loadings
  7. Step 3: Introduction of nuisance loadings
  8. Step 4: Sampling of scores
  9. Step 5: Injection of graph-correlated noise
  10. Step 6: Generation of data samples
  11. Evaluation Metrics
  12. Selectivity score
  13. Alignment score
  14. Global reconstruction score
  15. Results

Figures (2)

  • Figure 1: Illustration of the the first three steps of the data generation pipeline. The feature structure is given by an Erdős--Rényi graph with $|V| = 20$ and edge probability $0.12$. The colormap represents node degree, shifting from purple (lowest) to yellow (highest). (Top Histogram Panels) Two smooth signal components ($r = 2$) generated via Tikhonov smoothing ($\gamma = 3$) and soft-thresholding ($S_\omega = 0.4$). Observe that the loadings are sign-coherent and their magnitude decays smoothly with distance from the central seed node, respecting the graph topology. (Bottom Histogram Panels) Three nuisance components ($q = 3$), each consisting of $s = 6$ randomly signed spikes on sub-maximal degree nodes. In contrast to the signal, these loadings are spatially incoherent and visually disjoint from the underlying community structure.
  • Figure 2: Selectivity as a function of edge density for the four PCA variants, evaluated across the three feature structures (ER in blue, BA in red, WS in green). Solid lines correspond to the isotropic noise regime, while dashed lines indicate the anisotropic regime.