Graph in Graph Neural Network

TL;DR

The paper tackles the limitation of conventional GNNs that require vertex representations as vectors by introducing Graph in Graph (GIG) Networks, where each vertex can itself be a graph. It presents a complete architecture including a GIG Sample Generation (GSG) layer, GVU and GGU modules in stacked GIG hidden layers, and a GIG output layer, enabling local graph cues and cross-graph relationships to inform learning. Empirical results across 14 datasets show state-of-the-art performance on 13/14 tasks, with strong results on real-world multi-graph data such as skeleton-based action recognition; ablations confirm the critical roles of GVU/GGU and robust behavior to proxy configurations. The work broadens graph representation learning by enabling multi-graph inputs and vertex-graph representations, offering a flexible framework that can incorporate existing GNN updating mechanisms and enhance performance in diverse graph analytics domains.

Abstract

Existing Graph Neural Networks (GNNs) are limited to process graphs each of whose vertices is represented by a vector or a single value, limited their representing capability to describe complex objects. In this paper, we propose the first GNN (called Graph in Graph Neural (GIG) Network) which can process graph-style data (called GIG sample) whose vertices are further represented by graphs. Given a set of graphs or a data sample whose components can be represented by a set of graphs (called multi-graph data sample), our GIG network starts with a GIG sample generation (GSG) module which encodes the input as a \textbf{GIG sample}, where each GIG vertex includes a graph. Then, a set of GIG hidden layers are stacked, with each consisting of: (1) a GIG vertex-level updating (GVU) module that individually updates the graph in every GIG vertex based on its internal information; and (2) a global-level GIG sample updating (GGU) module that updates graphs in all GIG vertices based on their relationships, making the updated GIG vertices become global context-aware. This way, both internal cues within the graph contained in each GIG vertex and the relationships among GIG vertices could be utilized for down-stream tasks. Experimental results demonstrate that our GIG network generalizes well for not only various generic graph analysis tasks but also real-world multi-graph data analysis (e.g., human skeleton video-based action recognition), which achieved the new state-of-the-art results on 13 out of 14 evaluated datasets. Our code is publicly available at https://github.com/wangjs96/Graph-in-Graph-Neural-Network.

Figures (7)

• Figure 1: Comparison between existing GNNs and our GIG network based on the example of human skeleton video-based action recognition task. While existing GNNs can only process graphs whose vertices carrying vectors or single values, they require to reduce each graph-style human skeleton frame as a vector. Our approach can directly include every graph-style frame in a GIG vertex in the proposed GIG sample, which is then processed by our GIG network that can process GIG samples whose vertices carrying more comprehensive graphs.
• Figure 2: Illustration of the propagation of the GIG network. Given a real-world multi-graph sample or a set of graph samples as the input, the GSG layer first transforms the input as a GIG sample, where each graph/graph sample is represented by a GIG vertex and each GIG vertex is represented by a proxy vertex. Then, a set of GIG layers are stacked, where each consists of (i) a GVU module which individually updates each GIG vertex (i.e., its graph edges, graph vertices, directed proxy edges starting from graph vertices, and local proxy vertex) based on its internal cues; and (ii) a GGU module which further updates each GIG vertex based on its relationship with other GIG vertices (i.e., updating global-edges, global proxy vertices, directed proxy edges starting from global proxy vertices, and graph vertices).
• Figure 3: Illustration of the propagation of the GVU and GGU module.
• Figure 4: (a) Comparison between original GNNs and GIG-GNNs with the same vertex/edge updating functions;(b) t-SNE visualisation of features learned on CLUSTER dataset (a six classes vertex classification dataset), where each dot represents a vertex and each colour represents a category label.
• Figure 5: Visualization of features learned by GIG-ST-GCN and ST-GCN, where sizes of white circles represent their importance for action recognition defined by networks. GIG-ST-GCN paid similar attentions to all joints throughout all frames with more attentions to crucial joints than ST-GCN.