Insight: Interpretable Semantic Hierarchies in Vision-Language Encoders

TL;DR

Vision-language foundations are powerful but opaque; Insight addresses this by learning a hierarchical, spatially grounded concept space using a CLIP-DINOiser backbone and Matryoshka Sparse Autoencoders. It discovers explicit parent-child relationships from patch co-occurrences and assigns expressive names through a hierarchy-aware labeling strategy that leverages a large, multi-source vocabulary. The approach yields rich, interpretable explanations across classification, open-vocabulary segmentation, and captioning without sacrificing performance, achieving competitive results and superior spatial grounding compared to prior concept-based methods. This work enhances trust and controllability of vision-language systems, enabling safer and more transparent deployment in real-world tasks.

Abstract

Language-aligned vision foundation models perform strongly across diverse downstream tasks. Yet, their learned representations remain opaque, making interpreting their decision-making hard. Recent works decompose these representations into human-interpretable concepts, but provide poor spatial grounding and are limited to image classification tasks. In this work, we propose Insight, a language-aligned concept foundation model that provides fine-grained concepts, which are human-interpretable and spatially grounded in the input image. We leverage a hierarchical sparse autoencoder and a foundation model with strong semantic representations to automatically extract concepts at various granularities. Examining local co-occurrence dependencies of concepts allows us to define concept relationships. Through these relations we further improve concept naming and obtain richer explanations. On benchmark data, we show that Insight provides performance on classification and segmentation that is competitive with opaque foundation models while providing fine-grained, high quality concept-based explanations. Code is available at https://github.com/kawi19/Insight.

Figures (24)

• Figure 1: Insight provides rich conceptual explanations for vision foundation model tasks. Our model provides a hierarchical concept representation (e.g.'Sailboat' concept is a child of 'Ship Boat') with local, spatially grounded concepts that are automatically named. These concepts enable Insight to provide concept-based explanations for decision-making across vision tasks.
• Figure 2: Overview of Insight. We visualize our approach to represent human-interpretable, well localized, and consistent concepts of different granularities in a language-aligned vision foundation model. As backbone, we use pretrained CLIP-DINOiser wysoczanska2024clipdinoiser and extract concepts patch-wise by training a Matryoshka Sparse Autoencoder (SAE) without any supervision. From these learned concept representations, we discover concept relations through examining the co-occurrences of concepts within patches that allow more fine-grained interpretation of decision-making on downstream tasks. We further name the concepts with a Matryoshka-aware matching between concept representations and a large vocabulary of labels, taking the discovered concept relations into account.
• Figure 3: A concept relationship subgraph. The parent concept, 'motor car', is given as box in the center with children concepts arranged around it. For each concept, we provide the assigned label (top) and the top-4 most activating images (first row) along with their spatial localization (second row) based on concept activation strength per patch.
• Figure 4: Spatial grounding of concept-based explanations. For an image of the "Swimming hole" Class of Places365 we show explanations of Insight against existing method. For Insight, we provide the parent concept (dark green) and its contributing children (light green) with percentage of contribution and the spatial grounding of the top-3 most contributing concepts. In contrast to our work, existing methods (bottom) show spurious concepts.
• Figure 5: Human User Study. Results on Task 1 -- Consistency evaluation (left) and Task 2 -- Relationship evaluation (right) for 1̃000 neurons each covered with 5 users in Amazon MTurk. We provide the scale description given to the user at the bottom.