Learning Interpretable Logic Rules from Deep Vision Models

TL;DR

VisionLogic addresses the interpretability gap in deep vision models by converting the final fully connected layer into interpretable predicates and grounding them to human-understandable visual concepts via causal validation. It delivers both local explanations for individual predictions and global explanations for classes in the form of logic rules, while preserving substantial discriminative power on ImageNet. The framework reveals rich predicate-concept mappings, supports many-to-many associations, and demonstrates robustness to appearance changes and perturbations, with CNNs and Transformers showing complementary behaviors. Empirically, VisionLogic improves localization over saliency baselines and provides a faithful, human-aligned explanation mechanism that complements traditional neural explanations. Overall, it offers a trustworthy, scalable bridge between complex model behavior and human-understandable reasoning for real-world vision tasks.

Abstract

We propose a general framework called VisionLogic to extract interpretable logic rules from deep vision models, with a focus on image classification tasks. Given any deep vision model that uses a fully connected layer as the output head, VisionLogic transforms neurons in the last layer into predicates and grounds them into vision concepts using causal validation. In this way, VisionLogic can provide local explanations for single images and global explanations for specific classes in the form of logic rules. Compared to existing interpretable visualization tools such as saliency maps, VisionLogic addresses several key challenges, including the lack of causal explanations, overconfidence in visualizations, and ambiguity in interpretation. VisionLogic also facilitates the study of visual concepts encoded by predicates, particularly how they behave under perturbation -- an area that remains underexplored in the field of hidden semantics. Apart from providing better visual explanations and insights into the visual concepts learned by the model, we show that VisionLogic retains most of the neural network's discriminative power in an interpretable and transparent manner. We envision it as a bridge between complex model behavior and human-understandable explanations, providing trustworthy and actionable insights for real-world applications.

Figures (6)

• Figure 1: Comparison of VisionLogic and GradCAM: VisionLogic is capable of identifying individual visual concepts with bounding boxes; human annotations can further refine and highlight the regions activating the predicates.
• Figure 2: Grounding hidden predicates to high-level visual concepts. We iteratively refine the bounding box to localize potential regions containing visual concepts. These regions are validated to: 1) causally affect a specific predicate $p_j$, and 2) be solely responsible for computing $p_j$. Human annotators further refine these regions and recognize consistent visual concepts across multiple such regions.
• Figure 3: Hidden predicates grounded in visual concepts. Each predicate is supported by three images. In each image, the predicate is labeled above the image frame, and the colored region within the image highlights the concept identified by the human annotator.
• Figure 4: Sensitivity to location changes.
• Figure 5: Predicates ranked first encode the global structure. Red regions correspond to ResNet, while green correspond to ViT.