Automated Detection of Visual Attribute Reliance with a Self-Reflective Agent

TL;DR

SAIA reframes interpretability as a scientific discovery process by deploying a self-reflective agent that autonomously hypothesizes, tests, and reevaluates visual attribute dependencies in pretrained vision models. Built on a multimodal LLM, SAIA iteratively generates targeted interventions and uses unsupervised self-evaluation to refine explanations, achieving improved predictiveness over non-reflective baselines. The authors validate SAIA on a controlled benchmark of 130 models across 18 reliance categories and demonstrate real-world attribute dependencies in pretrained systems like CLIP-ViT and YOLOv8, including counterfactual cases. The work offers a scalable, automated approach to auditing model behavior, with practical implications for robustness, fairness, and trust in deployed vision systems. Limitations include dependence on image-generation quality and potential priors in the agent, guiding future work toward richer hypothesis spaces and higher-fidelity stimuli.

Abstract

When a vision model performs image recognition, which visual attributes drive its predictions? Detecting unintended reliance on specific visual features is critical for ensuring model robustness, preventing overfitting, and avoiding spurious correlations. We introduce an automated framework for detecting such dependencies in trained vision models. At the core of our method is a self-reflective agent that systematically generates and tests hypotheses about visual attributes that a model may rely on. This process is iterative: the agent refines its hypotheses based on experimental outcomes and uses a self-evaluation protocol to assess whether its findings accurately explain model behavior. When inconsistencies arise, the agent self-reflects over its findings and triggers a new cycle of experimentation. We evaluate our approach on a novel benchmark of 130 models designed to exhibit diverse visual attribute dependencies across 18 categories. Our results show that the agent's performance consistently improves with self-reflection, with a significant performance increase over non-reflective baselines. We further demonstrate that the agent identifies real-world visual attribute dependencies in state-of-the-art models, including CLIP's vision encoder and the YOLOv8 object detector.

Figures (10)

• Figure 1: Attribute Reliance Detection. We use a self-reflective agent to produce natural-language descriptions of the visual attributes that a model relies on to recognize or detect a given concept. For each target concept (e.g., vase, teacher, or pedestrians), the agent first conducts hypothesis testing to reach a candidate description and then validates the description's predictiveness of actual model behavior through a self-evaluation protocol. The top row shows the agent’s generated explanations. The bottom rows show images predicted to elicit high (green) or low (red) model responses, along with their actual model scores. Results are shown for different target concepts across an object recognition model with a controlled attribute reliance (left), CLIP (middle), and YOLOv8 (right).
• Figure 2: Attribute reliance detection through hypothesis testing and self-reflection. To discover features that drive model prediction, SAIA starts by formulating and testing a range of hypotheses. After reaching a conclusion (e.g., the model favors suit in corporate settings), it performs self-evaluation by testing the model responses on images with and without this feature. When inconsistencies between the conclusion and model behavior are observed (e.g., some non-corporate images yield high scores, while some corporate images yield lower scores), SAIA updates its prior beliefs according to these discrepancies and test alternative hypothesized explanations.
• Figure 3: Self-reflection stage. SAIA initiates a hypothesis-testing experiment. After testing multiple candidate hypotheses (see Fig. \ref{['fig:agent']}), SAIA draws an inital conclusion (e.g., teddy bear are detected based on appearing in educational settings). SAIA then uses a self-evaluation protocol that generates synthetic images via a text-to-image model and computes the subject model’s scores on these images. The self-evaluation protocol compares the predicted and actual model scores, triggering another round of hypothesis testing if results deviate from expectations. In this example, SAIA observes that the highest scores correlate with direct child interaction rather than generic educational settings, leading to refined future hypotheses.
• Figure 4: Simulating visual feature reliance. We simulate feature reliance by modulating object recognition scores based on the presence of specific visual attributes (e.g., a bird detector that relies on the presence of beach background). Given an input image and object category $t$, $\mathcal{O}_t$ produces a object recognition score for object presence. If the object is not detected, a low random score is assigned as the final model score of the image. If the object is detected, we simulate an attribute dependency (e.g., presence of a “beach background” for bird detection) through the procedure described in Sec. \ref{['sec:simulation']}. If the condition is satisfied, the final model score equals $\mathcal{O}_t(\texttt{img})$ recognition score. Otherwise, the score is discounted by a factor $\alpha$ to represent the model's weaker response in the case that the attribute condition is not met.
• Figure 5: Predictiveness Score over Self-Reflection Rounds.(a) We plot the average predictiveness score over all models in the benchmark for a harsh reliance magnitude of $\alpha=0.9$, as well as a softer reliance magnitude of $\alpha=0.5$. We see a steady increase in the predictiveness scores of SAIA's conclusions over rounds for both discount factors, approaching their respective theoretical upper bounds as given by the ground truth baseline. As expected, the scores for softer reliance models ($\alpha=0.5$) are consistently lower than those of the stronger reliance models ($\alpha=0.9$), illustrating that subtler attribute reliances are more challenging to detect. SAIA outperforms all nonreflective baselines (MAIA, Milan, and Multi-Agent) by a significant margin for both reliance magnitude values. (b) We compare the predictiveness scores of SAIA's reliance descriptions over factual models with more intuitive demographic attribute reliances against counterfactual reliances on object-demographic associations that are not commonly observed. Although SAIA's descriptions of the counterfactual models achieve lower predictiveness scores, the performance still reliably improves over increased rounds of self-reflection for both $\alpha$ settings.