Learning Network Representations with Disentangled Graph Auto-Encoder

TL;DR

The paper addresses the challenge of learning disentangled representations in graph-structured data, where real-world graphs arise from multiple latent factors. It introduces two models, DGA and DVGA, built on a Dynamic Disentangled Graph Encoder with $K$ channels, component-wise flows, and a factor-wise decoder to factorize information across latent factors. Key contributions include the dynamic multi-channel encoder, per-channel normalizing flows for richer posteriors, an explicit factor-wise decoder, and an independence regularizer to enforce factor separation, yielding improved link prediction, node clustering, and semi-supervised classification on synthetic and real graphs. The work enables interpretable, factor-aware graph representations with practical gains, while analyzing computational trade-offs and offering ablations that underline the importance of each component.

Abstract

The (variational) graph auto-encoder is widely used to learn representations for graph-structured data. However, the formation of real-world graphs is a complicated and heterogeneous process influenced by latent factors. Existing encoders are fundamentally holistic, neglecting the entanglement of latent factors. This reduces the effectiveness of graph analysis tasks, while also making it more difficult to explain the learned representations. As a result, learning disentangled graph representations with the (variational) graph auto-encoder poses significant challenges and remains largely unexplored in the current research. In this paper, we introduce the Disentangled Graph Auto-Encoder (DGA) and the Disentangled Variational Graph Auto-Encoder (DVGA) to learn disentangled representations. Specifically, we first design a disentangled graph convolutional network with multi-channel message-passing layers to serve as the encoder. This allows each channel to aggregate information about each latent factor. The disentangled variational graph auto-encoder's expressive capability is then enhanced by applying a component-wise flow to each channel. In addition, we construct a factor-wise decoder that takes into account the characteristics of disentangled representations. We improve the independence of representations by imposing independence constraints on the mapping channels for distinct latent factors. Empirical experiments on both synthetic and real-world datasets demonstrate the superiority of our proposed method compared to several state-of-the-art baselines.

Figures (9)

• Figure 1: An illustrative social network example that inspired our work. We consider using lines of different colors to represent different underlying factors that contribute to the relationship between two persons. The thickness of the lines is directly proportional to the frequency of interaction due to these factors.
• Figure 2: The whole framework of the proposed DVGA. It takes in nodes and their neighbors along with their feature vectors. The dynamic disentangled encoder utilizes DGCN, which employs a dynamic assignment mechanism that considers neighborhoods induced by $K$ distinct factors and neighborhoods in the $k$-th component space. After channel-wise convolution, the resulting component representations are merged and fed into flows for learning expressive disentangled node representations, which are then sent into a factor-wise decoder to reconstruct the adjacency matrix. A classifier $g_\phi$ is employed to promote the independence of component representations among several latent factors. The joint optimization of the factor-wise auto-encoder and independence regularization enhances the disentanglement. In this example, we assume the existence of three latent factors, each of which corresponds to one of the three channels.
• Figure 3: The absolute correlation values between the elements of the 64-dimensional representations learned by VGAE and DVGA on a synthetic graph with four latent factors. DVGA shows clearly four diagonal blocks (marked by the red dashed blocks), but VGAE does not show this correlation.
• Figure 4: Visualization of node embeddings on Cora.
• Figure 5: Results for link prediction of synthetic graphs with different number of channels $K$.

Theorems & Definitions (1)

• proof