Graph Generative Models Evaluation with Masked Autoencoder
Chengen Wang, Murat Kantarcioglu
TL;DR
This work addresses the challenge of evaluating graph generative models by learning expressive graph representations with a Graph Masked Autoencoder (GMAE). By extracting faithful representations, the method enables evaluation via distributional metrics such as $FD$ and $MMD$, and diversity metrics like $P&R$ and $D&C$, while leveraging a perturbation-based setup and Spearman correlation to assess representation quality. Empirical results across REDDIT-MULTI-5K, DBLP_v1, and Proteins show that GMAE-based evaluation often outperforms two deep-learning baselines, though no method uniformly dominates across all metrics and datasets. The study highlights the value of DL-based graph representations for GGM evaluation and argues for using multiple complementary techniques to capture fidelity and diversity in graph generation tasks.
Abstract
In recent years, numerous graph generative models (GGMs) have been proposed. However, evaluating these models remains a considerable challenge, primarily due to the difficulty in extracting meaningful graph features that accurately represent real-world graphs. The traditional evaluation techniques, which rely on graph statistical properties like node degree distribution, clustering coefficients, or Laplacian spectrum, overlook node features and lack scalability. There are newly proposed deep learning-based methods employing graph random neural networks or contrastive learning to extract graph features, demonstrating superior performance compared to traditional statistical methods, but their experimental results also demonstrate that these methods do not always working well across different metrics. Although there are overlaps among these metrics, they are generally not interchangeable, each evaluating generative models from a different perspective. In this paper, we propose a novel method that leverages graph masked autoencoders to effectively extract graph features for GGM evaluations. We conduct extensive experiments on graphs and empirically demonstrate that our method can be more reliable and effective than previously proposed methods across a number of GGM evaluation metrics, such as "Fréchet Distance (FD)" and "MMD Linear". However, no single method stands out consistently across all metrics and datasets. Therefore, this study also aims to raise awareness of the significance and challenges associated with GGM evaluation techniques, especially in light of recent advances in generative models.