GraphNarrator: Generating Textual Explanations for Graph Neural Networks

TL;DR

GraphNarrator tackles the challenge of explaining Graph Neural Networks when graphs carry semantic text by generating natural language explanations. It introduces a pseudo-labeling pipeline where saliency-based graph explanations are verbalized into prompts for a Pseudo-Label Generator LLM, which is iteratively improved with information-theoretic objectives via Expert Iteration and then distilled into an End-to-End Explainer LLM using knowledge distillation with LoRA. Across three TAG datasets, GraphNarrator yields faithful, concise explanations and favorable human judgments, outperforming zero-shot LLM baselines in key metrics like PMI and Simulatability while maintaining readability. This work advances trustworthy, human-friendly explanations for GNNs in text-rich graph domains, enabling better interpretability in applications with semantic feature graphs.

Abstract

Graph representation learning has garnered significant attention due to its broad applications in various domains, such as recommendation systems and social network analysis. Despite advancements in graph learning methods, challenges still remain in explainability when graphs are associated with semantic features. In this paper, we present GraphNarrator, the first method designed to generate natural language explanations for Graph Neural Networks. GraphNarrator employs a generative language model that maps input-output pairs to explanations reflecting the model's decision-making process. To address the lack of ground truth explanations to train the model, we propose first generating pseudo-labels that capture the model's decisions from saliency-based explanations, then using Expert Iteration to iteratively train the pseudo-label generator based on training objectives on explanation quality. The high-quality pseudo-labels are finally utilized to train an end-to-end explanation generator model. Extensive experiments are conducted to demonstrate the effectiveness of GraphNarrator in producing faithful, concise, and human-preferred natural language explanations.

Figures (10)

• Figure 1: Illustration of saliency-based graph explanation and natural language graph explanation.
• Figure 2: An illustration of GraphNarrator. A pseudo-label generator model is first trained to provide pseudo-labels, which are used for knowledge distillation to an LLM as an end-to-end explainer. (a) GraphNarrator first generates saliency-based graph explanations, then verbalizes them into a documented form (Saliency Paragraph) for easier understanding of LLMs, and feeds them to LLMs to generate initial natural language explanation pseudo-labels. (b) We propose the graph explanation expert iteration procedure to iteratively improve the pseudo-label generator LLM with three objectives related to faithfulness and brevity.
• Figure 3: Illustration of graph explanation verbalization. The blue edge denotes a cross-edge.
• Figure 4: The change of three pseudo-label quality scores in the TAG explanation expert iteration process, w.r.t number of training iteration.
• Figure 5: Visualization of a saliency-based explanation and a corresponding natural language explanation generated by GraphNarrator. In (a), red words indicate important terms, with darker red showing higher importance. We only visualized first 15 words in each paper due to the space limitation. In (b), yellow highlights reference high-saliency areas and emphasize that the explanation summarized key information.