Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TopoPrune: Robust Data Pruning via Unified Latent Space Topology

Arjun Roy, Prajna G. Malettira, Manish Nagaraj, Kaushik Roy

TL;DR

This work tackles the instability of geometry-based data pruning by introducing TopoPrune, a dual-scale topology framework that captures the data's intrinsic structure. It combines a global topology-aware manifold embedding with a local differentiable persistent-homology optimization to rank samples by structural complexity and form a high-quality coreset, augmented by a training-free mislabels proxy NLPS. The approach yields higher accuracy and markedly greater stability across architectures, with strong robustness to noisy embeddings and considerable transferability between diverse proxy and target models. The results underscore the potential of topology-based data selection to enable reliable, model-agnostic data-efficient learning in real-world pipelines.

Abstract

Geometric data pruning methods, while practical for leveraging pretrained models, are fundamentally unstable. Their reliance on extrinsic geometry renders them highly sensitive to latent space perturbations, causing performance to degrade during cross-architecture transfer or in the presence of feature noise. We introduce TopoPrune, a framework which resolves this challenge by leveraging topology to capture the stable, intrinsic structure of data. TopoPrune operates at two scales, (1) utilizing a topology-aware manifold approximation to establish a global low-dimensional embedding of the dataset. Subsequently, (2) it employs differentiable persistent homology to perform a local topological optimization on the manifold embeddings, ranking samples by their structural complexity. We demonstrate that our unified dual-scale topological approach ensures high accuracy and precision, particularly at significant dataset pruning rates (e.g., 90%). Furthermore, through the inherent stability properties of topology, TopoPrune is (a) exceptionally robust to noise perturbations of latent feature embeddings and (b) demonstrates superior transferability across diverse network architectures. This study demonstrates a promising avenue towards stable and principled topology-based frameworks for robust data-efficient learning.

TopoPrune: Robust Data Pruning via Unified Latent Space Topology

TL;DR

This work tackles the instability of geometry-based data pruning by introducing TopoPrune, a dual-scale topology framework that captures the data's intrinsic structure. It combines a global topology-aware manifold embedding with a local differentiable persistent-homology optimization to rank samples by structural complexity and form a high-quality coreset, augmented by a training-free mislabels proxy NLPS. The approach yields higher accuracy and markedly greater stability across architectures, with strong robustness to noisy embeddings and considerable transferability between diverse proxy and target models. The results underscore the potential of topology-based data selection to enable reliable, model-agnostic data-efficient learning in real-world pipelines.

Abstract

Geometric data pruning methods, while practical for leveraging pretrained models, are fundamentally unstable. Their reliance on extrinsic geometry renders them highly sensitive to latent space perturbations, causing performance to degrade during cross-architecture transfer or in the presence of feature noise. We introduce TopoPrune, a framework which resolves this challenge by leveraging topology to capture the stable, intrinsic structure of data. TopoPrune operates at two scales, (1) utilizing a topology-aware manifold approximation to establish a global low-dimensional embedding of the dataset. Subsequently, (2) it employs differentiable persistent homology to perform a local topological optimization on the manifold embeddings, ranking samples by their structural complexity. We demonstrate that our unified dual-scale topological approach ensures high accuracy and precision, particularly at significant dataset pruning rates (e.g., 90%). Furthermore, through the inherent stability properties of topology, TopoPrune is (a) exceptionally robust to noise perturbations of latent feature embeddings and (b) demonstrates superior transferability across diverse network architectures. This study demonstrates a promising avenue towards stable and principled topology-based frameworks for robust data-efficient learning.
Paper Structure (46 sections, 2 theorems, 8 equations, 12 figures, 13 tables, 1 algorithm)

This paper contains 46 sections, 2 theorems, 8 equations, 12 figures, 13 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background and Related Work
  3. Topology at Two Scales
  4. Low-Dimensional Manifold Approximations.
  5. Interactions of Samples and their Nearest Neighbors.
  6. Methodology
  7. Global Structure: Dataset Representation with Topological Manifold Embedding
  8. Local Structure: Sample Neighborhoods with Persistence-Based Optimizer
  9. Interpreting this notion of local dataset structure.
  10. Comprehensive Score with Global and Local Dataset Structures
  11. A Training-Free Proxy for Mislabeled Samples
  12. Topological Stratification and Coreset Construction
  13. Results
  14. Experimental Setup
  15. Performant and Stable Coresets with TopoPrune
  16. ...and 31 more sections

Key Result

Proposition 1.6

Let $f_A$ be a network embedding. Consider a new embedding $f_B$ defined by a simple isotropic scaling transformation, $f_B(x) = \alpha f_A(x)$ for some scalar $\alpha > 0, \alpha \neq 1$. Then the distribution of distances to the prototype is scaled accordingly: $P(S_k(f_B)) = \alpha P(S_k(f_A))$.

Figures (12)

  • Figure 1: Topological data selection yields higher-performing and stable coresets. Coreset performance across multiple runs reveals the limitations of common methods. (a) Euclidean-based selection is stable but achieves lower accuracy. (b) Graph-based methods achieve higher accuracy but are highly variable. (c) Our topological approach achieves both high accuracy and stability.
  • Figure 2: An overview of TopoPrune. (Left) A topology-aware projection visualizes the global data manifold. (Middle) Within each class, a density-preserving persistent homology optimization derives a local persistence score per sample. The color map indicates high (yellow) to low (blue) density. (Right) The final coreset is constructed via stratified sampling on a unified score combining global density and local persistence. This not only prioritizes the most topologically informative samples but also faithfully represents the density distribution of the original dataset.
  • Figure 3: Topological metrics are more consistent across networks. Which translates directly to better coreset performance. Metric distributions become progressively more uniform as we move from (a) unstable Euclidean distances, to (b) density estimation from global topological projection, and finally to (c) local persistence. This enhanced metric stability allows TopoPrune to consistently outperform geometry-based baselines (d), achieving both higher mean accuracy and lower standard deviation across 10 diverse architectures at a high pruning rate of 90% for CIFAR-100, where top left (low standard deviation and high accuracy) is best.
  • Figure 4: Impact of noisy feature embeddings for Moderate, D2, and TopoPrune on CIFAR-100. (a) Moderate has decent precision (shown by the tightness of the shaded region) but lower accuracy on average. (b) D2 has higher accuracy but significantly looser precision, especially at high pruning rate and high $\epsilon$-noise. (c) TopoPrune achieves both high accuracy and tighter standard deviation across all pruning rates and $\epsilon$-noise.
  • Figure 5: Overview of Simplicial Complexes and Persistent Homology
  • ...and 7 more figures

Theorems & Definitions (9)

  • Definition 1.1: Vietoris-Rips Filtration
  • Definition 1.2: Persistence Diagram
  • Definition 1.3: Bottleneck Distance
  • Definition 1.4: Gromov-Hausdorff Distance
  • Definition 1.5: Class Prototype and Distance Distribution
  • Proposition 1.6: Sensitivity to Scaling
  • proof
  • Proposition 1.7: Invariance and Stability of Persistent Homology
  • Remark