The Underappreciated Power of Vision Models for Graph Structural Understanding

TL;DR

This work reveals that vision-based encoders, when fed graph layouts as images, can match traditional GNNs on graph-level tasks while exhibiting distinct, global-pattern-based learning. The authors propose GraphAbstract to systematically evaluate human-like graph perception along four core tasks—topology archetypes, symmetry, spectral gap, and bridges—with scale-generalization tests. Vision models show superior scale-invariant understanding and robustness to distribution shifts, especially in symmetry and global structure tasks, while GNNs gain little from increased capacity without global priors. The study advocates a global-first mindset for graph foundation models, suggesting future work to integrate visual perception with structural priors to achieve robust, interpretable, and scalable graph reasoning. The findings imply practical avenues for leveraging vision-centric representations in tasks dominated by holistic pattern recognition and scale-invariant graph understanding.

Abstract

Graph Neural Networks operate through bottom-up message-passing, fundamentally differing from human visual perception, which intuitively captures global structures first. We investigate the underappreciated potential of vision models for graph understanding, finding they achieve performance comparable to GNNs on established benchmarks while exhibiting distinctly different learning patterns. These divergent behaviors, combined with limitations of existing benchmarks that conflate domain features with topological understanding, motivate our introduction of GraphAbstract. This benchmark evaluates models' ability to perceive global graph properties as humans do: recognizing organizational archetypes, detecting symmetry, sensing connectivity strength, and identifying critical elements. Our results reveal that vision models significantly outperform GNNs on tasks requiring holistic structural understanding and maintain generalizability across varying graph scales, while GNNs struggle with global pattern abstraction and degrade with increasing graph size. This work demonstrates that vision models possess remarkable yet underutilized capabilities for graph structural understanding, particularly for problems requiring global topological awareness and scale-invariant reasoning. These findings open new avenues to leverage this underappreciated potential for developing more effective graph foundation models for tasks dominated by holistic pattern recognition.

Figures (24)

• Figure 1: Prediction overlap analysis across different datasets. Top row: correct prediction overlap; Bottom row: error pattern overlap. GNN variants show high internal consistency, suggesting homogeneous learning behavior, while GNN and Vision models exhibit distinct prediction patterns.
• Figure 2: Case Studies for PROTEINS dataset.
• Figure 3: Training dynamics across different architectures on NCI1 dataset. For each model, we plot the training loss (blue), training accuracy (red), and validation accuracy (green) over 100 epochs. The shaded areas represent the standard deviation across multiple runs.
• Figure 4: Distribution of bridge counts across different graph types under various settings (Train, ID, Near-OOD, and Far-OOD). The plots reveal distinct bridge count patterns for each graph structure (Geometric, Community, Hierarchical, Bottleneck, and Multicore). Notably, the distributions exhibit shifts as graph sizes increase, particularly visible in the OOD scenarios.
• Figure 5: Distribution of spectral gaps across different graph types under various settings (Train, ID, Near-OOD, and Far-OOD). The plots reveal distinct spectral gap patterns for each graph structure. Notably, the distributions exhibit shifts as graph sizes increase.

Theorems & Definitions (18)

• Definition 1: Graph Automorphism
• Definition 2: Symmetric and Asymmetric Graphs
• Definition 3: Cayley Graph
• Definition 4: Bipartite Double Cover
• Definition 5: Cartesian Product
• Definition 1: Automorphism
• Definition 2: Automorphism Group
• Definition 3: Symmetry
• Definition 4: Bipartite Double Cover
• Definition 5: $k$-fold Cyclic Cover