The Expressive Power of Graph Neural Networks: A Survey

TL;DR

The paper addresses the incomplete theoretical understanding of GNN expressiveness and proposes a unified framework distinguishing feature embedding and topology representation as core components. It surveys three broad strategies to boost expressiveness—feature enhancement, topology enhancement, and architecture enhancement—and maps a wide range of models to these categories, including WL-based analyses, subgraph counting, and higher-order equivariant networks. The work highlights evaluation gaps, such as metrics that jointly assess feature and topology expressiveness, and discusses practical trade-offs in scalability and complexity. It argues that advancing both theoretical tools (e.g., beyond WL tests) and practical architectures (including graph transformers and physics-inspired designs) is essential for robust, scalable, and broadly applicable GNNs.

Abstract

Graph neural networks (GNNs) are effective machine learning models for many graph-related applications. Despite their empirical success, many research efforts focus on the theoretical limitations of GNNs, i.e., the GNNs expressive power. Early works in this domain mainly focus on studying the graph isomorphism recognition ability of GNNs, and recent works try to leverage the properties such as subgraph counting and connectivity learning to characterize the expressive power of GNNs, which are more practical and closer to real-world. However, no survey papers and open-source repositories comprehensively summarize and discuss models in this important direction. To fill the gap, we conduct a first survey for models for enhancing expressive power under different forms of definition. Concretely, the models are reviewed based on three categories, i.e., Graph feature enhancement, Graph topology enhancement, and GNNs architecture enhancement.

Figures (6)

• Figure 1: The specific logical structure of the article.
• Figure 2: The illustration of the aggregation and update process of WL test. (a) Given two graphs without features, and add color labels to all nodes. (b) In the first iteration, the different information aggregated by the nodes is mapped into new color labels, and then these new labels are redistributed to nodes, and the number of labels is counted after the distribution. After 1st iteration, $G1$ and $G2$ have the same color distributions to determine whether they are isomorphic, and the next iteration is performed. (c) Perform the node neighbor aggregation and redistribution of color labels steps again, and get the different color distributions of $G1$ and $G2$, at this time, it can be determined that the two are non-isomorphic.
• Figure 3: An illustration of the expressive power of NNs and their effects on the performance of learned models. (a) Machine learning problems aim to learn the mapping from the feature space to the target space based on several observed examples. (b) The expressive power of NNs refers to the gap between the two spaces $\mathcal{F}$ and $\mathcal{F'}$. Although NNs are expressive ($\mathcal{F'}$ is dense in $\mathcal{F}$), the learned model $f'$ based on NNs may differ signifcantly from $f^{\ast}$ due to their overfit of the limited observed data. This figure is with reference to Figure. 5.5 in wu2022graph.
• Figure 4: An illustration of the expressive power of GNNs. a) The feature embedding ability of GNNs is the same as that of NNs in that both maps the observed examples in the feature space $\mathcal{X}$ to the target space $Y$ via $f$. The strength of the feature embedding ability is measured by the size of the value domain space $\mathcal{F}$ of $f$. b) The topology representation capability of GNNs is achieved by mapping the observed examples in the feature space to the target space via $f$ and preserving the original topology between the examples. The strength of the capability is measured by the size of the value domain space $\mathcal{F'}$ with ${\bf X}={\bf 1}$. c) The expressive power of GNNs consists of a combination of feature embedding ability and topology representation ability, measured by the size of the intersection of $\mathcal{F}$ and $\mathcal{F'}$ with ${\bf X}={\bf random}$.
• Figure 5: Input and output of the GNNs model under different expressive power representations. When approximate ability is used to characterise the expressive power, the input of the model is a set of graphs and the output is a graph embeddings. When separation ability, the input is a pair of graphs and the output is graph embeddings. When the subgraph counting ability, the input is a single graph and the output is a node (set) embeddings.