Logical Distillation of Graph Neural Networks
Alexander Pluska, Pascal Welke, Thomas Gärtner, Sagar Malhotra
TL;DR
This work introduces Iterated Decision Trees (IDTs) as a symbolic, interpretable distillation of Graph Neural Networks (GNNs) that captures the two-variable counting fragment $C^2$ and extends it to handle mean-like aggregations common in GNNs. By learning successive IDT layers from intermediate GNN representations, the authors show that the distilled models are concise, maintain competitive accuracy, and can surpass the GNN when the ground truth is expressible in $C^2$. The approach bridges logic and deep learning by making the reasoning of GNNs explicit through logical formulas and leaf-set constructions, enabling human-understandable explanations. The work demonstrates practical benefits on both synthetic and real datasets (e.g., AIDS), and outlines a path toward broader applicability, including extensions to more complex graph types and richer modal operators.
Abstract
We present a logic based interpretable model for learning on graphs and an algorithm to distill this model from a Graph Neural Network (GNN). Recent results have shown connections between the expressivity of GNNs and the two-variable fragment of first-order logic with counting quantifiers (C2). We introduce a decision-tree based model which leverages an extension of C2 to distill interpretable logical classifiers from GNNs. We test our approach on multiple GNN architectures. The distilled models are interpretable, succinct, and attain similar accuracy to the underlying GNN. Furthermore, when the ground truth is expressible in C2, our approach outperforms the GNN.