Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

This Looks Distinctly Like That: Grounding Interpretable Recognition in Stiefel Geometry against Neural Collapse

Junhao Jia, Jiaqi Wang, Yunyou Liu, Haodong Jing, Yueyi Wu, Xian Wu, Yefeng Zheng

TL;DR

Adaptive Manifold Prototypes (AMP) is proposed, a framework that leverages Riemannian optimization on the Stiefel manifold to represent class prototypes as orthonormal bases and make rank one prototype collapse infeasible by construction.

Abstract

Prototype networks provide an intrinsic case based explanation mechanism, but their interpretability is often undermined by prototype collapse, where multiple prototypes degenerate to highly redundant evidence. We attribute this failure mode to the terminal dynamics of Neural Collapse, where cross entropy optimization suppresses intra class variance and drives class conditional features toward a low dimensional limit. To mitigate this, we propose Adaptive Manifold Prototypes (AMP), a framework that leverages Riemannian optimization on the Stiefel manifold to represent class prototypes as orthonormal bases and make rank one prototype collapse infeasible by construction. AMP further learns class specific effective rank via a proximal gradient update on a nonnegative capacity vector, and introduces spatial regularizers that reduce rotational ambiguity and encourage localized, non overlapping part evidence. Extensive experiments on fine-grained benchmarks demonstrate that AMP achieves state-of-the-art classification accuracy while significantly improving causal faithfulness over prior interpretable models.

This Looks Distinctly Like That: Grounding Interpretable Recognition in Stiefel Geometry against Neural Collapse

TL;DR

Adaptive Manifold Prototypes (AMP) is proposed, a framework that leverages Riemannian optimization on the Stiefel manifold to represent class prototypes as orthonormal bases and make rank one prototype collapse infeasible by construction.

Abstract

Prototype networks provide an intrinsic case based explanation mechanism, but their interpretability is often undermined by prototype collapse, where multiple prototypes degenerate to highly redundant evidence. We attribute this failure mode to the terminal dynamics of Neural Collapse, where cross entropy optimization suppresses intra class variance and drives class conditional features toward a low dimensional limit. To mitigate this, we propose Adaptive Manifold Prototypes (AMP), a framework that leverages Riemannian optimization on the Stiefel manifold to represent class prototypes as orthonormal bases and make rank one prototype collapse infeasible by construction. AMP further learns class specific effective rank via a proximal gradient update on a nonnegative capacity vector, and introduces spatial regularizers that reduce rotational ambiguity and encourage localized, non overlapping part evidence. Extensive experiments on fine-grained benchmarks demonstrate that AMP achieves state-of-the-art classification accuracy while significantly improving causal faithfulness over prior interpretable models.
Paper Structure (25 sections, 16 equations, 5 figures, 3 tables)

This paper contains 25 sections, 16 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Post-hoc Interpretability Methods
  4. Prototype-based Interpretability Methods
  5. Optimization on Orthogonal Manifolds
  6. Preliminaries on Prototypical Networks
  7. Methodology
  8. Orthogonal Subspaces on the Stiefel Manifold
  9. Dynamic Rank Calibration via Proximal Gradients
  10. Semantic Gauge Fixing and Unsupervised Part Discovery
  11. Spatial Entropy Minimization.
  12. Spatial Overlap Penalty.
  13. Decoupled Optimization and Interpretable Inference
  14. Interpretable inference.
  15. Experiments
  16. ...and 10 more sections

Figures (5)

  • Figure 1: Motivation of Adaptive Manifold Prototypes (AMP). (a) Contrast between diverse and redundant prototypes. (b) Loss regularizations yield soft constraints. (c) AMP imposes a geometric constraint on the Stiefel manifold, ensuring part diversity.
  • Figure 2: The overview of our proposed AMP framework. (a) Training: learn Stiefel bases and capacity matrices via Riemannian gradients. (b) Inference: project features onto class manifolds, then aggregate activations to class scores for interpretable predictions..
  • Figure 3: Qualitative visualization of the AMP reasoning process on the CUB-200-2011 (left) and Stanford Cars (right) datasets. For a given test image, AMP explicitly decomposes the final Class Evidence Score into a sparse sum of localized visual evidence.
  • Figure 4: Hyperparameter sensitivity on CUB-200-2011 and Standford Cars.
  • Figure 5: Human evaluation results and dynamic rank distribution.