ECOR: Explainable CLIP for Object Recognition

TL;DR

This work tackles explainability in open-vocabulary object recognition with CLIP by defining explainability as the joint distribution $P(c,\{r_i\}_{i=1}^{m}|I)$ and proposing an autoregressive two-stage model that first predicts $P(\{r_i\}_{i=1}^{m}|I)$ and then $P(c|\{r_i\}_{i=1}^{m},I)$. It implements this via prompt-tuned CLIP with Prompt_R for rationales and Prompt_{c|R} for categories, optimizing a two-term loss $L_{train} = L_{CLIP}(...Prompt_R) + L_{CLIP}(...Prompt_{c|R})$ and evaluating on six DROR datasets. The results show state-of-the-art explainable classification and strong zero-shot transfer, with ablations confirming that conditioning category on rationales and jointly modeling $P(R|I)$ and $P(c|R,I)$ are crucial, while inverted conditioning or independence assumptions harm performance, especially on large datasets. The work advances trustworthy, interpretable object recognition and provides code for reproducibility.

Abstract

Large Vision Language Models (VLMs), such as CLIP, have significantly contributed to various computer vision tasks, including object recognition and object detection. Their open vocabulary feature enhances their value. However, their black-box nature and lack of explainability in predictions make them less trustworthy in critical domains. Recently, some work has been done to force VLMs to provide reasonable rationales for object recognition, but this often comes at the expense of classification accuracy. In this paper, we first propose a mathematical definition of explainability in the object recognition task based on the joint probability distribution of categories and rationales, then leverage this definition to fine-tune CLIP in an explainable manner. Through evaluations of different datasets, our method demonstrates state-of-the-art performance in explainable classification. Notably, it excels in zero-shot settings, showcasing its adaptability. This advancement improves explainable object recognition, enhancing trust across diverse applications. The code will be made available online upon publication.

Figures (4)

• Figure 1: Our definition of explainability involves the effective utilization of true and relevant visual attributes, represented as text rationales, for object recognition and class prediction.
• Figure 2: In the first step (a), the model utilizes Prompt$_{R}$to calculate the distribution of rationales. Then in Stage 2 (b), it identifies the distribution of categories conditioned on rationales using Prompt$_{c|R}$. Notably, the model's ability to detect categories is informed by the identified rationales, providing an explanation for its choices.
• Figure 3: Examples of the generated rationales and their corresponding images in the DROR ImageNet dataset mao2023doubly. The central words represent categories, while the surrounding words depict the corresponding rationales. Additionally, we provide one example image, retrieved through Google search, for each category and rationale.
• Figure 4: Visualization of saliency maps. Images are sampled from DROR datasets mao2023doubly. In columns 2, 4, 7, and 9, we show the rationales produced by the model to explain the prediction. In columns 3, 5, 8, and 10, we show the saliency map Chen_2022_ACCV that the models used to produce the prediction and rationales. Our method produces the correct category and rationales by attending to the appropriate parts of the images. Moreover, our model is more robust against being deceived by objects that are similar to the correct rationales but incorrect. This is clear in the ocean example (row 4, column 1), as our model correctly detected the waves, in contrast to the baseline model that focused on the clouds.

Theorems & Definitions (1)

• Definition 1