From Graph Diffusion to Graph Classification

TL;DR

It is shown how graph diffusion models can be applied for graph classification and found that to achieve competitive classification accuracy, score-based graph diffusion models should be trained with a novel training objective that is tailored to graph classification.

Abstract

Generative models such as diffusion models have achieved remarkable success in state-of-the-art image and text tasks. Recently, score-based diffusion models have extended their success beyond image generation, showing competitive performance with discriminative methods in image {\em classification} tasks~\cite{zimmermann2021score}. However, their application to classification in the {\em graph} domain, which presents unique challenges such as complex topologies, remains underexplored. We show how graph diffusion models can be applied for graph classification. We find that to achieve competitive classification accuracy, score-based graph diffusion models should be trained with a novel training objective that is tailored to graph classification. In experiments with a sampling-based inference method, our discriminative training objective achieves state-of-the-art graph classification accuracy.

Figures (7)

• Figure 1: The overall architecture of our model, adapted from SwinGNN yan2023swingnn, processes graph inputs by first transforming them into noisy adjacency matrices. These matrices are then concatenated with label embeddings and passed through multiple layers of the model. After processing through these blocks, the model outputs a denoised, clean adjacency matrix. This clean matrix is subsequently transformed back into graph form, completing the cycle from input to output.
• Figure 2: Overview of training and testing procedure in our method. Train Time: The input adjacency matrix is first randomly permuted, then input to a GNN along with all possible class labels (separately, but in the same model). The loss $L_\text{DEN}$ is then computed based on the output of the ground-truth class, the $L_\text{CLF}$ is computed based on the output of all the labels, and $L_\text{SUM}$ is the sum of two losses. Test Time: The adjacency matrix is permuted and input to the GNN along with all possible labels, similar to train time. Then, we compute the likelihood of the graph given each label via exact or approximate likelihood computation. We obtain the posterior probability for each label and pick the label with the highest probability. At inference time, the procedure is repeated for $P$ permutations, and the final label is decided based on majority voting (not shown for brevity).
• Figure 3: The mean accuracy curve during training using $L_\text{CLF}$ on IMDB-BINARY dataset when training with or without random permutation of the adjacency matrix. Training with permutations offers significant gains in accuracy.
• Figure 4: The accuracy curve of varying the number of permutations across three training objectives ($L_\text{CLF}$, $L_\text{DEN}$, and $L_\text{SUM}$) with approximate inference. In general, increasing the number of inference-time permutations improves performance, but the gains saturate around 100 permutations.
• Figure 5: The mean validation accuracy curve during training using $L_\text{DEN}$ on IMDB-BINARY dataset when taining with or without random permutation of the adjacency matrix. The blue line is training without permutation, the red line is training with permutation