Curse of Dimensionality on Persistence Diagrams

TL;DR

The paper addresses the reliability of persistence diagrams as topological descriptors in high-dimension low-sample-size data. It derives that in the HDLSS regime, observed diagrams drift away from the originals with distinct behaviors for Rips and Čech filtrations, constituting a curse of dimensionality. The authors propose a normalization-based PCA approach to compress high-dimensional data, showing that it yields bounded bottleneck and Hausdorff distances between original and compressed diagrams in the Rips case, thus mitigating the curse to some extent. Overall, the work exposes fundamental limitations of persistence diagrams under HDLSS and offers a principled mitigation path with normalized PCA, guiding future theoretical and methodological developments in topological data analysis for high-dimensional data.

Abstract

The stability of persistent homology has led to wide applications of the persistence diagram as a trusted topological descriptor in the presence of noise. However, with the increasing demand for high-dimension and low-sample-size data processing in modern science, it is questionable whether persistence diagrams retain their reliability in the presence of high-dimensional noise. This work aims to study the reliability of persistence diagrams in the high-dimension low-sample-size data setting. By analyzing the asymptotic behavior of persistence diagrams for high-dimensional random data, we show that persistence diagrams are no longer reliable descriptors of low-sample-size data under high-dimensional noise perturbations. We refer to this loss of reliability of persistence diagrams in such data settings as the curse of dimensionality on persistence diagrams. Next, we investigate the possibility of using normalized principal component analysis as a method for reducing the dimensionality of the high-dimensional observed data to resolve the curse of dimensionality. We show that this method can mitigate the curse of dimensionality on persistence diagrams. Our results shed some new light on the challenges of processing high-dimension low-sample-size data by persistence diagrams and provide a starting point for future research in this area.

Figures (4)

• Figure 1: Numerical results for comparison between the $1$st original persistence diagram (upper left) and the $1$st observed persistence diagrams in different dimensions (upper right: $d=1000$; lower left: $d=5000$; lower right: $d=10{,}000$). Different colors indicate different multiplicities of generators in persistence diagrams.
• Figure 2: Numerical results for comparison between the $1$st original persistence diagram (upper left) and the $1$st compressed persistence diagrams in different dimensions (upper right: $d=1000$; lower left: $d=5000$; lower right: $d=10{,}000$). Different colors indicate different multiplicities of generators in persistence diagrams.
• Figure 3: The minimum eigengap of the real Wishart matrix $W\sim \mathcal{W}(I_{100},d)$ with $d$ ranges in $[150,2000]$.
• Figure 4: The curve fitting result. The blue dots represent the minimum eigengap averaged over 20,000 randomly generated Wishart matrices and the red curve is the fitting curve obtained by the fitting model $x\sqrt{d-y}+z$. Here $x= 0.0353324372$, $y=100.690024$, and $z=-0.000162906231$, respectively.

Theorems & Definitions (98)

• Definition 2.1
• Definition 2.2
• Proposition 2.3
• proof : Proof.
• Definition 2.4
• Definition 2.5
• Definition 2.6
• Theorem 2.7: chazal2014persistence
• Remark 2.8
• Definition 2.9