Factor Graph-based Interpretable Neural Networks

TL;DR

This work tackles the vulnerability of concept-level explanations to unknown perturbations by introducing AGAIN, a factor graph–based interpretable neural network. Instead of retraining to defend against unseen attacks, AGAIN encodes real-world logical rules as a factor graph, detects explanatory violations during inference, and rectifies explanations via an interactive intervention switch that does not rely on perturbation learning. The authors provide theoretical arguments linking explanation comprehensibility to the presence and configuration of the factor graph, and validate their approach on three datasets (CUB, MIMIC-III EWS, Synthetic-MNIST), showing superior E-ACC and LSM scores under unknown perturbations while preserving predictive performance. They also discuss limitations and future work, including dependence on correct category predictions and the need to revise the graph if domain knowledge changes, highlighting a promising direction for robust, trustable explanations with reduced retraining costs.

Abstract

Comprehensible neural network explanations are foundations for a better understanding of decisions, especially when the input data are infused with malicious perturbations. Existing solutions generally mitigate the impact of perturbations through adversarial training, yet they fail to generate comprehensible explanations under unknown perturbations. To address this challenge, we propose AGAIN, a fActor GrAph-based Interpretable neural Network, which is capable of generating comprehensible explanations under unknown perturbations. Instead of retraining like previous solutions, the proposed AGAIN directly integrates logical rules by which logical errors in explanations are identified and rectified during inference. Specifically, we construct the factor graph to express logical rules between explanations and categories. By treating logical rules as exogenous knowledge, AGAIN can identify incomprehensible explanations that violate real-world logic. Furthermore, we propose an interactive intervention switch strategy rectifying explanations based on the logical guidance from the factor graph without learning perturbations, which overcomes the inherent limitation of adversarial training-based methods in defending only against known perturbations. Additionally, we theoretically demonstrate the effectiveness of employing factor graph by proving that the comprehensibility of explanations is strongly correlated with factor graph. Extensive experiments are conducted on three datasets and experimental results illustrate the superior performance of AGAIN compared to state-of-the-art baselines.

Figures (11)

• Figure 1: Interpretable neural networks suffer from perturbations that generate incomprehensible explanations. For instance, the model predicts the input as "Dog" but explains it with "Wings" and "Plume".
• Figure 2: An example of the factor graph. It consists of 4 factors and 8 variables.
• Figure 3: Overall structure of AGAIN.
• Figure 4: Factor graph construction.
• Figure 5: The impact of the factor graph size on P-ACC and E-ACC across 4 perturbation magnitudes on two real-world datasets.