Interpretable Neural-Symbolic Concept Reasoning

TL;DR

The paper tackles the interpretability gap in deep learning by introducing the Deep Concept Reasoner (DCR), a novel interpretable concept-based model that uses concept embeddings to generate differentiable fuzzy rules and executes them on semantically meaningful concept truth degrees. DCR delivers semantically meaningful, differentiable rule-based predictions, enabling global explanations and counterfactuals, while achieving competitive accuracy on a diverse set of tasks. The key contributions include a formal rule syntax, differentiable rule generation and execution via neural modules for role and relevance, a parsimony mechanism with a Gödel-based fuzzy semantics, and comprehensive experiments showing superior generalization, interpretable rule discovery aligned with ground truth, and robust counterfactual explanations. The approach bridges concept-based and neural-symbolic paradigms, offering practical impact for trustworthy AI by combining accuracy, interpretability, and actionable explanations across tabular, image, and graph domains.

Abstract

Deep learning methods are highly accurate, yet their opaque decision process prevents them from earning full human trust. Concept-based models aim to address this issue by learning tasks based on a set of human-understandable concepts. However, state-of-the-art concept-based models rely on high-dimensional concept embedding representations which lack a clear semantic meaning, thus questioning the interpretability of their decision process. To overcome this limitation, we propose the Deep Concept Reasoner (DCR), the first interpretable concept-based model that builds upon concept embeddings. In DCR, neural networks do not make task predictions directly, but they build syntactic rule structures using concept embeddings. DCR then executes these rules on meaningful concept truth degrees to provide a final interpretable and semantically-consistent prediction in a differentiable manner. Our experiments show that DCR: (i) improves up to +25% w.r.t. state-of-the-art interpretable concept-based models on challenging benchmarks (ii) discovers meaningful logic rules matching known ground truths even in the absence of concept supervision during training, and (iii), facilitates the generation of counterfactual examples providing the learnt rules as guidance.

Figures (12)

• Figure 1: (a) An interpretable concept-based model $f$ maps concepts $\hat{C}$ to tasks $\hat{Y}$ generating an interpretable rule. When input features are not semantically meaningful, a concept encoder $g$ can map raw features to a concept space. (b) The proposed approach (DCR) outperforms interpretable concept-based models in the Dot dataset. CE stands for concept embeddings and CT for concept truth values.
• Figure 2: (left) Deep Concept Reasoner (DCR) generates fuzzy logic rules using neural models on concept embeddings. Then DCR executes the rule using the concept truth degrees to evaluate the rule symbolically. (right) Schema of DCR modules: first neural models $\phi$ and $\psi$ generate the rule, and then the rule is executed symbolically.
• Figure 3: Mean ROC AUC for task predictions for all baselines across all tasks (the higher the better). DCR often outperforms interpretable concept-based models. CE stands for concept embeddings, while CT for concept truth degrees. Models trained on concept embeddings are not interpretable as concept embeddings lack a clear semantic for individual embedding dimensions.
• Figure 4: Sensitivity of model explanation when changing the radius of the input perturbation. The lower, the better. DCR explanations engender trust as they are stable under small perturbations of the input. The same does not hold generally for LIME explanations of XGBoost or Relu Net decision rules.
• Figure 5: Model confidence as a function of the number of perturbed features on counterfactual examples. The lower, the better. Similarly to interpretable methods, DCR prediction confidence quickly drops after inverting the truth degree of a small set of relevant concepts, facilitating the discovery of counterfactual examples.

Theorems & Definitions (4)

• Example 3.1
• Example 3.2
• Example 3.3
• Example 3.4