Representation Learning on Graphs: Methods and Applications

TL;DR

This survey synthesizes the landscape of representation learning on graphs, detailing node and subgraph embedding methods from shallow factorization and random-walk approaches to deep, neighborhood-aggregation and graph neural networks. It presents a unified encoder–decoder framework that clarifies how embeddings are learned and what graph information they capture, including extensions to heterogeneous and multi-layer graphs. The authors discuss supervised and unsupervised strategies, highlight practical applications such as visualization, classification, and link prediction, and outline crucial open problems, particularly around scalability, temporal dynamics, higher-order motifs, and interpretability. Overall, the work clarifies core design choices, connects diverse methods, and maps a path for theoretical and practical advancements in graph representation learning.

Abstract

Machine learning on graphs is an important and ubiquitous task with applications ranging from drug design to friendship recommendation in social networks. The primary challenge in this domain is finding a way to represent, or encode, graph structure so that it can be easily exploited by machine learning models. Traditionally, machine learning approaches relied on user-defined heuristics to extract features encoding structural information about a graph (e.g., degree statistics or kernel functions). However, recent years have seen a surge in approaches that automatically learn to encode graph structure into low-dimensional embeddings, using techniques based on deep learning and nonlinear dimensionality reduction. Here we provide a conceptual review of key advancements in this area of representation learning on graphs, including matrix factorization-based methods, random-walk based algorithms, and graph neural networks. We review methods to embed individual nodes as well as approaches to embed entire (sub)graphs. In doing so, we develop a unified framework to describe these recent approaches, and we highlight a number of important applications and directions for future work.

Figures (9)

• Figure 1: Two different views of a character-character interaction graph derived from the Les Misérables novel, where two nodes are connected if the corresponding characters interact. The coloring in the left figure emphasizes differences in the nodes' global positions in the graph: nodes have the same color if they belong to the same community, at a global level. In contrast, the coloring in the right figure denotes structural equivalence between nodes, or the fact that two nodes play similar roles in their local neighborhoods (e.g., "bridging nodes" are colored blue). The colorings for both figures were generated using different settings of the node2vec node embedding method grover2016node2vec, described in Section \ref{['sec:nodes']}. Reprinted from grover2016node2vec with permission.
• Figure 2: A, Graph structure of the Zachary Karate Club social network, where nodes are connected if the corresponding individuals are friends. The nodes are colored according to the different communities that exist in the network. B, Two-dimensional visualization of node embeddings generated from this graph using the DeepWalk method (Section \ref{['sec:randwalk']}) perozzi2014deepwalk. The distances between nodes in the embedding space reflect similarity in the original graph, and the node embeddings are spatially clustered according to the different color-coded communities. Reprinted with permission from perozzi2014deepwalkperozzithesis.
• Figure 3: Overview of the encoder-decoder approach. First the encoder maps the node, $v_i$, to a low-dimensional vector embedding, $\mathbf{z}_i$, based on the node's position in the graph, its local neighborhood structure, and/or its attributes. Next, the decoder extracts user-specified information from the low-dimensional embedding; this might be information about $v_i$'s local graph neighborhood (e.g., the identity of its neighbors) or a classification label associated with $v_i$ (e.g., a community label). By jointly optimizing the encoder and decoder, the system learns to compress information about graph structure into the low-dimensional embedding space.
• Figure 4: The random-walk based methods sample a large number of fixed-length random walks starting from each node, $v_i$. The embedding vectors are then optimized so that the dot-product, or angle, between two embeddings, $\mathbf{z}_i$ and $\mathbf{z}_j$, is (roughly) proportional to the probability of visiting $v_j$ on a fixed-length random walk starting from $v_i$.
• Figure 5: A, Illustration of how node2vec biases the random walk using the $p$ and $q$ parameters. Assuming that the walk just transitioned from $v_s$ to $v_*$, the edge labels, $\alpha$, are proportional to the probability of the walk taking that edge at next time-step. B, Difference between random-walks that are based on breadth-first search (BFS) and depth-first search (DFS). BFS-like random walks are mainly limited to exploring a node's immediate (i.e., one-hop) neighborhood and are generally more effective for capturing structural roles. DFS-like walks explore further away from the node and are more effective for capturing community structures. Adapted from grover2016node2vec.