$O(k)$-Equivariant Dimensionality Reduction on Stiefel Manifolds

TL;DR

This work introduces Principal Stiefel Coordinates (PSC), a framework for $O(k)$-equivariant dimensionality reduction of data on Stiefel and Grassmannian manifolds by embedding $V_k(\mathbb{R}^n)$ into $V_k(\mathbb{R}^N)$ via $\alpha \in V_n(\mathbb{R}^N)$ and projecting with a continuous map $\pi_\alpha$. The authors provide two methods to obtain the embedding: $\alpha_{PCA}$, a PCA-based warm start, and $\alpha_{GD}$, a gradient-descent refinement that minimizes a nuclear-norm objective to reduce distortion, with $\pi_\alpha$ proven to be continuous and $O(k)$-equivariant. The paper proves that $\alpha_{PCA}$ is a global optimizer in noiseless, low-dimensional settings, while $\alpha_{GD}$ improves results when data deviates from linear embeddings; this is demonstrated across synthetic and real datasets, including stimulus space models, brain connectivity matrices, and video data. The PSC approach extends naturally to Grassmannians and is competitive with, and often superior to, existing methods like PGA, especially under nonlinearity and noise, highlighting its practical value for manifold-valued data in neuroscience, computer vision, and beyond.

Abstract

Many real-world datasets live on high-dimensional Stiefel and Grassmannian manifolds, $V_k(\mathbb{R}^N)$ and $Gr(k, \mathbb{R}^N)$ respectively, and benefit from projection onto lower-dimensional Stiefel and Grassmannian manifolds. In this work, we propose an algorithm called \textit{Principal Stiefel Coordinates (PSC)} to reduce data dimensionality from $ V_k(\mathbb{R}^N)$ to $V_k(\mathbb{R}^n)$ in an \textit{$O(k)$-equivariant} manner ($k \leq n \ll N$). We begin by observing that each element $α\in V_n(\mathbb{R}^N)$ defines an isometric embedding of $V_k(\mathbb{R}^n)$ into $V_k(\mathbb{R}^N)$. Next, we describe two ways of finding a suitable embedding map $α$: one via an extension of principal component analysis ($α_{PCA}$), and one that further minimizes data fit error using gradient descent ($α_{GD}$). Then, we define a continuous and $O(k)$-equivariant map $π_α$ that acts as a "closest point operator" to project the data onto the image of $V_k(\mathbb{R}^n)$ in $V_k(\mathbb{R}^N)$ under the embedding determined by $α$, while minimizing distortion. Because this dimensionality reduction is $O(k)$-equivariant, these results extend to Grassmannian manifolds as well. Lastly, we show that $π_{α_{PCA}}$ globally minimizes projection error in a noiseless setting, while $π_{α_{GD}}$ achieves a meaningfully different and improved outcome when the data does not lie exactly on the image of a linearly embedded lower-dimensional Stiefel manifold as above. Multiple numerical experiments using synthetic and real-world data are performed.

Figures (15)

• Figure 1: Principal Stiefel Coordinates (PSC) illustrated with $k=1, n=2, N=3$. $y, \pi_\alpha(y) \in V_k(\mathbb{R}^N), \alpha \in V_n(\mathbb{R}^N), \hat{y}_\alpha \in V_k(\mathbb{R}^n)$.
• Figure 1: (a) 100 data samples $\mathcal{Y} \subset V_1(\mathbb{R}^3)$ generated with noise level $\varepsilon=0.5$. Without noise, all samples lie on the great circle, whose orientation and embedding in the sphere are determined by $\alpha$ as shown in Figure \ref{['fig:pipeline_alpha']}. (b) Optimization loss landscape \ref{['eq:nuclear_norm_opt']} over the projections of $\alpha \in V_{2}(\mathbb{R}^3)$ to the Grassmannian $Gr(1,3) = \mathbb{R}\textbf{P}^2$. Input data are 100 noisy samples $\mathcal{Y}$ with $\varepsilon=0.8$. As described below, the loss function \ref{['eq:nuclear_norm_opt']} descends to $Gr(1,3)$ which we plot as the upper hemisphere of $S^2$. The azimuthal angle is $\theta \in [0, 2 \pi]$ and the inclination angle is $\phi \in [0, \frac{\pi}{2}]$ in this polar coordinate plot. Higher function values correspond to lower projection error. "True" is the $\alpha$ used to generate the noisy samples.
• Figure 1: Stimulus space model experiment results from multi-dimensional scaling, Persistent cohomology, $\alpha_{PCA}$ and $\alpha_{random}$ without any post-processing or smoothing.
• Figure 1: Runtime of checking rank condition. Mean and standard deviation calculated over 5 runs.
• Figure 2: Variance ratio for PSC compared to principal geodesic analysis. Here the $x$-axis is the dimension of the submanifold which is the target of the projection. That is, in dimension 1, PGA projects to a geodesic 1-manifold, while PSC projects to a circle.

Theorems & Definitions (39)

• Remark 1.1
• Definition 2.1: Stiefel manifold
• Definition 2.2
• Definition 2.3: Singular value decomposition (SVD)
• Theorem 2.4
• Definition 3.1
• Definition 3.2
• Remark 3.3
• Theorem 3.4
• Proof 1