Linking Robustness and Generalization: A k* Distribution Analysis of Concept Clustering in Latent Space for Vision Models

TL;DR

The paper addresses the challenge that latent-space quality in vision models is often assessed indirectly via projections, hindering interpretability and cross-model comparisons. It introduces the k* distribution, a local-neighborhood analysis that evaluates concept-level structure in latent spaces, and defines true and approximate skewness metrics $\Gamma_{k^*}$ and $\Gamma'_{k^*}$ to quantify latent-space quality. Empirical results across RobustBench robust models and OpenCLIP encoders show that current models tend to fracture concept distributions, and that improvements in generalization and robustness generally reduce fracturing, leading to better concept clustering. This framework provides a direct, interpretable means to compare latent spaces across models and datasets, linking latent-space organization to model robustness and generalization with practical implications for model analysis and design.

Abstract

Most evaluations of vision models use indirect methods to assess latent space quality. These methods often involve adding extra layers to project the latent space into a new one. This projection makes it difficult to analyze and compare the original latent space. This article uses the k* Distribution, a local neighborhood analysis method, to examine the learned latent space at the level of individual concepts, which can be extended to examine the entire latent space. We introduce skewness-based true and approximate metrics for interpreting individual concepts to assess the overall quality of vision models' latent space. Our findings indicate that current vision models frequently fracture the distributions of individual concepts within the latent space. Nevertheless, as these models improve in generalization across multiple datasets, the degree of fracturing diminishes. A similar trend is observed in robust vision models, where increased robustness correlates with reduced fracturing. Ultimately, this approach enables a direct interpretation and comparison of the latent spaces of different vision models and reveals a relationship between a model's generalizability and robustness. Results show that as a model becomes more general and robust, it tends to learn features that result in better clustering of concepts. Project Website is available online at https://shashankkotyan.github.io/k-Distribution/

Figures (4)

• Figure 1: Overview of the framework to create and analyze k* Distribution. We use the learned features of a vision model to compute k* values of individual evaluated samples and then compute the k* distribution for a particular concept (class). We then evaluate the entire learned latent space by quantifying the quality of latent spaced using either True Skewness Coefficient $\Gamma_{k^*}$ based on Overall k* distribution or an Approximate Skewness Coefficient $\Gamma^{'}_{k^*}$ based on averaging the skew of individual k* distributions.
• Figure 2: Comparison of two variants of robust Wang2023Better_WRN-70-16 models: a) $L_2$ Robust (Left) and b) $L_\infty$ Robust (Right) Models using k* Distribution, t-SNE and UMAP. Note that information between the k* Distribution, UMAP, and t-SNE remains the same. However, it is easier to compare the models by visually inspecting k* Distribution compared to t-SNE and UMAP. This problem of comparing using t-SNE and UMAP becomes more prevalent as a number of concepts (classes) increases or the number of models increases.
• Figure 3: Comparison of different robust models using the Approximate Skewness Coefficient $\Gamma^{'}_{k^*}$. Note that $\gamma_{k^*} > 0.5$ means the latent space is mostly fractured (Blue Region), $-0.5 < \gamma_{k^*} < 0.5$ means the latent space is mostly overlapped (Red Region), while $\gamma_{k^*}< -0.5$ means the latent space is mostly clustered (Green Region).
• Figure 4: Comparison of different Open-CLIP models using the Approximate Skewness Coefficient $\Gamma^{'}_{k^*}$ for multiple datasets. Note that $\gamma_{k^*} > 0.5$ means the latent space is mostly fractured (Blue Region), $-0.5 < \gamma_{k^*} < 0.5$ means the latent space is mostly overlapped (Red Region), while $\gamma_{k^*}< -0.5$ means the latent space is mostly clustered (Green Region).