xDNN(ASP): Explanation Generation System for Deep Neural Networks powered by Answer Set Programming
Ly Ly Trieu, Tran Cao Son
TL;DR
This work tackles the explainability challenge of deep neural networks by translating a trained DNN and its training data into an answer-set program (ASP), enabling global explanations. The xDNN(ASP) system constructs a logic program Pi(M) with top-, intermediate-, and bottom-level rules that reflect input-to-output behavior and interactions among hidden layers, using C5.0 to derive rule-sets at each level. Experiments on synthetic XOR-type datasets show Pi(M) preserves high predictive fidelity (approximately 92–95%) and reveals feature importance and hidden-node impacts, providing guidance for network redesign. Compared with prior decompositional methods, xDNN(ASP) offers more stable fidelity and interpretable rule-based explanations, with a linear-size program whose structure supports actionable insights for model simplification and optimization.
Abstract
Explainable artificial intelligence (xAI) has gained significant attention in recent years. Among other things, explainablility for deep neural networks has been a topic of intensive research due to the meteoric rise in prominence of deep neural networks and their "black-box" nature. xAI approaches can be characterized along different dimensions such as their scope (global versus local explanations) or underlying methodologies (statistic-based versus rule-based strategies). Methods generating global explanations aim to provide reasoning process applicable to all possible output classes while local explanation methods focus only on a single, specific class. SHAP (SHapley Additive exPlanations), a well-known statistical technique, identifies important features of a network. Deep neural network rule extraction method constructs IF-THEN rules that link input conditions to a class. Another approach focuses on generating counterfactuals which help explain how small changes to an input can affect the model's predictions. However, these techniques primarily focus on the input-output relationship and thus neglect the structure of the network in explanation generation. In this work, we propose xDNN(ASP), an explanation generation system for deep neural networks that provides global explanations. Given a neural network model and its training data, xDNN(ASP) extracts a logic program under answer set semantics that-in the ideal case-represents the trained model, i.e., answer sets of the extracted program correspond one-to-one to input-output pairs of the network. We demonstrate experimentally, using two synthetic datasets, that not only the extracted logic program maintains a high-level of accuracy in the prediction task, but it also provides valuable information for the understanding of the model such as the importance of features as well as the impact of hidden nodes on the prediction. The latter can be used as a guide for reducing the number of nodes used in hidden layers, i.e., providing a means for optimizing the network.