Global Concept Explanations for Graphs by Contrastive Learning

TL;DR

This work develops Megan2, an extension of the MEGAN framework, to generate global concept explanations for graph property prediction tasks by learning a subgraph-centric latent space and clustering it into interpretable concepts. A contrastive learning objective, channel-wise projection networks, and a prototype-optimization pipeline (via a genetic algorithm) enable compact, representative prototypes and concept-level interpretations, with optional GPT-4 hypothesis generation for causal insights. Across synthetic datasets (e.g., BA2Motifs, RbMotifs) and real-world molecular datasets (Mutagenicity, AqSolDB), Megan2 both faithfully recovers ground-truth motifs and uncovers diverse, chemistry-aligned concepts, providing finer-grained explanations than prior global explainers. The results demonstrate the potential of global concept explanations to reveal underlying structure–property relationships in graph domains, while highlighting limitations related to Megan’s assumptions and the reliability of language-model hypotheses for certain tasks.

Abstract

Beyond improving trust and validating model fairness, xAI practices also have the potential to recover valuable scientific insights in application domains where little to no prior human intuition exists. To that end, we propose a method to extract global concept explanations from the predictions of graph neural networks to develop a deeper understanding of the tasks underlying structure-property relationships. We identify concept explanations as dense clusters in the self-explaining Megan models subgraph latent space. For each concept, we optimize a representative prototype graph and optionally use GPT-4 to provide hypotheses about why each structure has a certain effect on the prediction. We conduct computational experiments on synthetic and real-world graph property prediction tasks. For the synthetic tasks we find that our method correctly reproduces the structural rules by which they were created. For real-world molecular property regression and classification tasks, we find that our method rediscovers established rules of thumb. More specifically, our results for molecular mutagenicity prediction indicate more fine-grained resolution of structural details than existing explainability methods, consistent with previous results from chemistry literature. Overall, our results show promising capability to extract the underlying structure-property relationships for complex graph property prediction tasks.

Figures (9)

• Figure 1: Visual overview of the proposed global explanation method. ⓐ Simplified MEGAN model architecture. The message-passing encoder creates explanation masks along multiple channels (pink/blue), which then result in individual subgraph latent representations for each channel. ⓑ In each channel's subgraph latent space, a clustering algorithm is used to find elements in the dataset that exhibit structurally similar explanation masks. ⓒ Each cluster is analyzed regarding its members and results are presented to the user in the form of an automatically generated report.
• Figure 2: Simplified overview of the multi-explanation graph attention network (MEGAN) architecture. Modified illustration based on the original work of Teufel et al. teufelMEGANMultiexplanationGraph2023a. Rounded boxes represent input/output tensor structures of the network. Square boxes represent network layers and arrows indicate layer interconnections.
• Figure 3: Visualization of the contrastive learning method. ⓐ The contrastive learning is applied for each explanation channel's latent space individually by maximizing an embedding's similarity to the positive sample and minimizing similarity to negative samples. Other elements in the training batch are used for the negative samples and the positive samples are derived by data augmentation. ⓑ The augmentation is applied in the final stage of the message-passing encoder, during the global graph pooling operation. The augmented view removes all graph structures outside the explained areas and applies additive gaussian noise to the explanation masks.
• Figure 4: ⓐ Illustration of the prototype optimization process. Starting from a population consisting of a concept's member graphs, node and edge deletion mutations are applied in each epoch to find a suitable prototype graph. ⓑ Example for the GPT-based hypothesis generation. The text representation of the concept prototype graph is included in a query that prompts the model to generate a causal explanation for the concept's structure-property relationship.
• Figure 5: Selected results of the concept clustering for a Megan2 model trained on the BA2Motifs dataset. Each row represents one concept cluster for either the house class (green) or the cycle class (orange). Columns from left to right show the result of the prototype optimization, aggregated statistics for the cluster members, and the 4 members closest to the cluster centroid. Prototype graphs are automatically optimized during the report generation, but have been re-drawn for the clarity of the visualization.