Exploring Interpretability for Visual Prompt Tuning with Hierarchical Concepts

TL;DR

This work tackles the challenge of interpreting visual prompt tuning by introducing Interpretable Visual Prompt Tuning (IVPT), which links prompts to human-understandable concepts via a hierarchy of category-agnostic prototypes. IVPT grounds each concept prototype to image regions through Concept Region Discovery and aggregates region features with Intra-region Feature Aggregation to form interpretable prompts, enabling fine-to-coarse explanations across ViT layers. A hierarchical fusion mechanism aligns fine-grained prompts with coarse-grained concepts, guided by a region-consistency loss and a multi-term training objective that balances accuracy and interpretability. Experiments on fine-grained and pathology datasets demonstrate that IVPT improves interpretability (consistency and stability) while maintaining or enhancing classification accuracy, outperforming conventional part-prototype methods and prior interpretable approaches. The approach offers practical potential for transparent AI in safety-critical domains, though it relies on in-domain concept prototypes, signaling avenues for expanding to broader domains.

Abstract

Visual prompt tuning offers significant advantages for adapting pre-trained visual foundation models to specific tasks. However, current research provides limited insight into the interpretability of this approach, which is essential for enhancing AI reliability and enabling AI-driven knowledge discovery. In this paper, rather than learning abstract prompt embeddings, we propose the first framework, named Interpretable Visual Prompt Tuning (IVPT), to explore interpretability for visual prompts, by introducing hierarchical concept prototypes. Specifically, visual prompts are linked to human-understandable semantic concepts, represented as a set of category-agnostic prototypes, each corresponding to a specific region of the image. Then, IVPT aggregates features from these regions to generate interpretable prompts, which are structured hierarchically to explain visual prompts at different granularities. Comprehensive qualitative and quantitative evaluations on fine-grained classification benchmarks show its superior interpretability and performance over conventional visual prompt tuning methods and existing interpretable methods.

Figures (8)

• Figure 1: IVPT generates prompts based on learnable concept prototypes associated with specific visual concepts. Each prompt aggregates features from an automatically learned region guided by the specific prototype, representing a corresponding concept.
• Figure 2: IVPT introduces category-agnostic concept prototypes to generate explainable visual prompt embeddings. At each layer, Concept Region Discovery (CRD) module captures specific visual concepts as concept-level image regions, while Intra-region Feature Aggregation (IFA) module aggregates features grouped by these region maps to obtain prompt embeddings. To capture concepts from multi-level granularities across layers, a hierarchical structure of concept prototypes is used, with the prototype number varying across layers. A fusion layer, guided by the coarse-grained region, fuses the prompts of the same layer in a fine-to-coarse manner.
• Figure 3: Illustration of fine-to-coarse hierarchical prompt fusion.
• Figure 4: Illustration of coarse-grained region maps with four prototypes across images of various categories, highlighting the importance score of each concept's features in the final classification.
• Figure 5: Qualitative results of explainable region maps generated by various approaches, with the number of concepts set to 10. We highlight the areas that significantly distinguish IVPT from other methods with red circles.