Graph Structure Learning with Bi-level Optimization

TL;DR

The proposed GSEBO is instantiate on classical GNNs and compared with the state-of-the-art GSL methods and extensive experiments validate the effectiveness of the proposed GSEBO on four real-world datasets.

Abstract

Currently, most Graph Structure Learning (GSL) methods, as a means of learning graph structure, improve the robustness of GNN merely from a local view by considering the local information related to each edge and indiscriminately applying the mechanism across edges, which may suffer from the local structure heterogeneity of the graph (\ie the uneven distribution of inter-class connections over nodes). To overcome the cons, we extract the graph structure as a learnable parameter and jointly learn the structure and common parameters of GNN from the global view. Excitingly, the common parameters contain the global information for nodes features mapping, which is also crucial for structure optimization (\ie optimizing the structure relies on global mapping information). Mathematically, we apply a generic structure extractor to abstract the graph structure and transform GNNs in the form of learning structure and common parameters. Then, we model the learning process as a novel bi-level optimization, \ie \textit{Generic Structure Extraction with Bi-level Optimization for Graph Structure Learning (GSEBO)}, which optimizes GNN parameters in the upper level to obtain the global mapping information and graph structure is optimized in the lower level with the global information learned from the upper level. We instantiate the proposed GSEBO on classical GNNs and compare it with the state-of-the-art GSL methods. Extensive experiments validate the effectiveness of the proposed GSEBO on four real-world datasets.

Figures (5)

• Figure 1: Examples for (a) the impact of local structure heterogeneity (node 1 and 9, the weights between nodes do not provide enough information for classes differentiation), and (b) the global view of messaging (the edge between node 3 and node 5, the deleted edge prevents the intra-class information transfer).
• Figure 2: The framework of GSEBO. The generic structure extractor (GSE) decouples the graph structure from GNNs to account for the edge-specific modeling, the inner and outer optimization steps are adopted for common parameters and graph structure optimization.
• Figure 3: Optimize inner and outer steps on the training set of Cora.
• Figure 4: The training loss and test accuracy of classical GNN and GSEBO on Cora.
• Figure 5: Node classification performance of GSEBO on Cora poisoned at different inter-class levels.