Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Predict Training Data Quality via Its Geometry in Metric Space

Yang Ba, Mohammad Sadeq Abolhasani, Rong Pan

TL;DR

The paper addresses how training data geometry in a metric space affects AI model generalization. It introduces PH-based diversity measures derived from lifetimes in Vietoris–Rips filtrations, including $p_i = l_i/L$, Rényi persistence entropy $PE_k^{(q)}$, and PH-based Hill numbers $ ext{PEH}_k^{q}$, focusing on $H_0$ and $H_1$. It demonstrates that PH-based diversity satisfies key axioms and correlates with higher accuracy and lower variance in transfer-learning tasks with BERT across several text datasets, while entropy-based metrics like the Vendi Score fail to predict data quality. The results offer practical guidance for data selection and augmentation, showing that well-structured geometric diversity enables near-full-data performance with substantially smaller training sets.

Abstract

High-quality training data is the foundation of machine learning and artificial intelligence, shaping how models learn and perform. Although much is known about what types of data are effective for training, the impact of the data's geometric structure on model performance remains largely underexplored. We propose that both the richness of representation and the elimination of redundancy within training data critically influence learning outcomes. To investigate this, we employ persistent homology to extract topological features from data within a metric space, thereby offering a principled way to quantify diversity beyond entropy-based measures. Our findings highlight persistent homology as a powerful tool for analyzing and enhancing the training data that drives AI systems.

Predict Training Data Quality via Its Geometry in Metric Space

TL;DR

The paper addresses how training data geometry in a metric space affects AI model generalization. It introduces PH-based diversity measures derived from lifetimes in Vietoris–Rips filtrations, including , Rényi persistence entropy , and PH-based Hill numbers , focusing on and . It demonstrates that PH-based diversity satisfies key axioms and correlates with higher accuracy and lower variance in transfer-learning tasks with BERT across several text datasets, while entropy-based metrics like the Vendi Score fail to predict data quality. The results offer practical guidance for data selection and augmentation, showing that well-structured geometric diversity enables near-full-data performance with substantially smaller training sets.

Abstract

High-quality training data is the foundation of machine learning and artificial intelligence, shaping how models learn and perform. Although much is known about what types of data are effective for training, the impact of the data's geometric structure on model performance remains largely underexplored. We propose that both the richness of representation and the elimination of redundancy within training data critically influence learning outcomes. To investigate this, we employ persistent homology to extract topological features from data within a metric space, thereby offering a principled way to quantify diversity beyond entropy-based measures. Our findings highlight persistent homology as a powerful tool for analyzing and enhancing the training data that drives AI systems.
Paper Structure (15 sections, 7 equations, 4 figures, 1 table)

This paper contains 15 sections, 7 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Persistent Homology in Metric Space
  4. Diversity Measurement
  5. Methodology
  6. PH-Based Diversity Measure
  7. Axiomatic Definition of Diversity
  8. Relationship between Structural Diversity and Model Performance
  9. Experiment & Analysis
  10. Experiment Setup
  11. Experiment Result
  12. Conclusion
  13. Technical Appendices and Supplementary Material
  14. Proof of Diversity Axioms for PH-Based Measures
  15. Correlation Between PH-based Diversity Measure and the Standard Deviation of Model Accuracy

Figures (4)

  • Figure 1: There are four scenarios for adding new data points to augment the current dataset (blue). However, without prior knowledge of the unseen data, clear guidance on which approach is most effective is lacking.
  • Figure 2: A demonstration of three representative subsets construction for "Medical" dataset
  • Figure 3: Model Accuracy vs. PH-based Diversity Measures
  • Figure 4: Minimum $H_0$ values vs. the standard deviation of model accuracy. A negative correlation between the minimum $H_0$ values and the standard deviation of model accuracy indicates that greater separation between clusters improves the stability of model training.