Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GICDM: Mitigating Hubness for Reliable Distance-Based Generative Model Evaluation

Nicolas Salvy, Hugues Talbot, Bertrand Thirion

TL;DR

This work tackles the instability of distance-based evaluation in high-dimensional embedding spaces caused by hubness. It introduces GICDM, a hubness-aware extension of ICDM that first uniformizes the real data density via ICDM, then computes per-generated-point scaling factors using only real data, and finally applies multi-scale filtering to guard against overcorrection. The approach yields more reliable fidelity and coverage metrics and better aligns with human judgments across synthetic and real benchmarks. By mitigating hubness while preserving the relative positioning of generated samples, GICDM enables more trustworthy evaluation of high-dimensional generative models.

Abstract

Generative model evaluation commonly relies on high-dimensional embedding spaces to compute distances between samples. We show that dataset representations in these spaces are affected by the hubness phenomenon, which distorts nearest neighbor relationships and biases distance-based metrics. Building on the classical Iterative Contextual Dissimilarity Measure (ICDM), we introduce Generative ICDM (GICDM), a method to correct neighborhood estimation for both real and generated data. We introduce a multi-scale extension to improve empirical behavior. Extensive experiments on synthetic and real benchmarks demonstrate that GICDM resolves hubness-induced failures, restores reliable metric behavior, and improves alignment with human judgment.

GICDM: Mitigating Hubness for Reliable Distance-Based Generative Model Evaluation

TL;DR

This work tackles the instability of distance-based evaluation in high-dimensional embedding spaces caused by hubness. It introduces GICDM, a hubness-aware extension of ICDM that first uniformizes the real data density via ICDM, then computes per-generated-point scaling factors using only real data, and finally applies multi-scale filtering to guard against overcorrection. The approach yields more reliable fidelity and coverage metrics and better aligns with human judgments across synthetic and real benchmarks. By mitigating hubness while preserving the relative positioning of generated samples, GICDM enables more trustworthy evaluation of high-dimensional generative models.

Abstract

Generative model evaluation commonly relies on high-dimensional embedding spaces to compute distances between samples. We show that dataset representations in these spaces are affected by the hubness phenomenon, which distorts nearest neighbor relationships and biases distance-based metrics. Building on the classical Iterative Contextual Dissimilarity Measure (ICDM), we introduce Generative ICDM (GICDM), a method to correct neighborhood estimation for both real and generated data. We introduce a multi-scale extension to improve empirical behavior. Extensive experiments on synthetic and real benchmarks demonstrate that GICDM resolves hubness-induced failures, restores reliable metric behavior, and improves alignment with human judgment.
Paper Structure (29 sections, 3 theorems, 15 equations, 16 figures, 11 tables, 1 algorithm)

This paper contains 29 sections, 3 theorems, 15 equations, 16 figures, 11 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Background: Hubness in High-Dimensional Distributions
  3. Gaussian case
  4. Origin of Hubness: Density Gradient
  5. Overview of In-sample Hubness Reduction
  6. GICDM
  7. Density Estimation from Nearest Neighbors Distances
  8. Extending ICDM to Generated Points
  9. Preventing Overcorrections
  10. Multi-scale filtering
  11. Related Work: Generative Model Metrics
  12. Experiments
  13. Conclusion
  14. Hubness Measures
  15. Hubness in Common Embedding Spaces
  16. ...and 14 more sections

Key Result

Proposition 4.1

Let where $V_d$ is the volume of the unit ball in dimension $d$. Then, $\hat{p}_{\mu, K}(x_i)$ is a local density estimator.

Figures (16)

  • Figure 1: High dimension challenges distance-based metrics: The scenario in (a) consists of real samples from a 60/40 mixture of two hyperspheres, and generated samples from a mixture with swapped radii and proportions. Standard metrics are plotted in (b) as the dimension increases. The real and generated sets are disjoint, so all metrics should score 0; however, in high dimensions (top), none does due to hubness. After applying GICDM (bottom), their scores correctly remain at 0. See \ref{['sec:hypersphere_metric_wise_results']} for individual metric results.
  • Figure 2: Distribution of $k$-occurrences ($k=5$) for $20000$ samples from a standard Gaussian in (a)$d=4$ and (b)$d=32$. The y-axis shows the count (log scale) for each $k$-occurrence value (x-axis). As dimension increases, the distribution skews right and hubs emerge: points that frequently appear as nearest neighbors.
  • Figure 3: $5$-Occurrence vs Centrality ($N=20000$, $d=32$): (a) (adapted from radovanovic2010hubs, Fig. 3) Standard Gaussian: hubness aligns with centrality; central points have higher occurrences. (b) Uniform sphere: removing centrality eliminates hubness. So, high dimensionality alone does not cause hubness.
  • Figure 4: Density Gradient: (a) (adapted from hara2016flattening, Fig. 2c): Illustrative Gaussian points (blue density) with $1$-occurrence counts; arrows indicate nearest neighbors. Occurrences accumulate in denser regions, as arrows follow the density gradient. (b): $5000$ samples from a uniform square. Color indicates $200$-occurrence count. Boundaries force neighbors inward, creating density gradients and hubness; both effects intensify with increasing dimension. After applying (c) NICDM and (d) ICDM: Left, arrows show updated neighbors. The maximum $O_1$ decreases from $4$ to $3$ (NICDM) and $2$ (ICDM); the number of antihubs drops from $6$ to $5$ and $2$. Right, the occurrence distribution becomes more uniform after NICDM, and even more so after ICDM.
  • Figure 5: Crossover dimension: For a standard Gaussian, the crossover dimension $d^*$ is where the median $K$-NN squared distance equals the expected squared distance to the center, marking the transition from points being closer to each other (low $d$) to being closer to the center (high $d$). The plot shows $d^*$, from \ref{['proposition:crossover_dimension']}, as a function of $K$ for various sample sizes $N$. As $N$ increases, points are closer so $d^*$ increases; as $K$ increases, $K$-NN distances increase so $d^*$ decreases. Using two sufficiently different $K$ values ensures that at least one is outside the crossover regime.
  • ...and 11 more figures

Theorems & Definitions (5)

  • Proposition 4.1
  • proof
  • Corollary 4.2
  • Proposition 4.3
  • proof