Unsupervised Graph Neural Architecture Search with Disentangled Self-supervision

TL;DR

A novel Disentangled Self-supervised Graph Neural Architecture Search (DSGAS) model is proposed, which is able to discover the optimal architectures capturing various latent graph factors in a self-supervised fashion based on unlabeled graph data.

Abstract

The existing graph neural architecture search (GNAS) methods heavily rely on supervised labels during the search process, failing to handle ubiquitous scenarios where supervisions are not available. In this paper, we study the problem of unsupervised graph neural architecture search, which remains unexplored in the literature. The key problem is to discover the latent graph factors that drive the formation of graph data as well as the underlying relations between the factors and the optimal neural architectures. Handling this problem is challenging given that the latent graph factors together with architectures are highly entangled due to the nature of the graph and the complexity of the neural architecture search process. To address the challenge, we propose a novel Disentangled Self-supervised Graph Neural Architecture Search (DSGAS) model, which is able to discover the optimal architectures capturing various latent graph factors in a self-supervised fashion based on unlabeled graph data. Specifically, we first design a disentangled graph super-network capable of incorporating multiple architectures with factor-wise disentanglement, which are optimized simultaneously. Then, we estimate the performance of architectures under different factors by our proposed self-supervised training with joint architecture-graph disentanglement. Finally, we propose a contrastive search with architecture augmentations to discover architectures with factor-specific expertise. Extensive experiments on 11 real-world datasets demonstrate that the proposed model is able to achieve state-of-the-art performance against several baseline methods in an unsupervised manner.

Figures (4)

• Figure 1: The framework of Disentangled Self-supervised Graph Neural Architecture Search (DSGAS), including the following three key components: 1) Disentangled graph architecture super-network enables multiple architectures to be disentangled and optimized simultaneously in an end-to-end manner. 2) Self-supervised training with joint architecture-graph disentanglement estimates the performance of architectures under various latent factors by considering the relationship between architectures, graphs and factors. 3) Contrastive search with architecture augmentations encourages and discovers architectures with distinct capabilities of capturing factors. (Best viewed in color)
• Figure 2: The performance of GNAS methods on real-world datasets under semi-supervised settings, where DSGAS-P denotes DSGAS without pretraining. The results are averaged by five random runs. (Best viewed in color)
• Figure 3: (a) Comparisons of different ablated variants of DSGAS on real-world datasets under unsupervised settings. The horizontal dashed line refers to the results of the best-performed GNAS baseline. (b) Comparisons of different architecture augmentations of DSGAS on real-world datasets under unsupervised settings, where 'Alpha', 'Weight' and 'Embed' denote the augmentations from perspective of operation choices, weight and embeddings. 'Compose' denotes uniformly choosing one of the three augmentations. The horizontal dashed line refers to the results of the best-performed GNAS baseline. (Best viewed in color)
• Figure 4: Visualizations of the search architectures on different datasets. Nodes denote GNN operations except that 'INPUT' denotes the input graphs with structures and features. Directed edges denote calculation flows, where different colors denote the architecture operation choices under different factors. (Best viewed in color)