From Sequence to Structure: Uncovering Substructure Reasoning in Transformers
Xinnan Dai, Kai Yang, Jay Revolinsky, Kai Guo, Aoran Wang, Bohang Zhang, Jiliang Tang
TL;DR
This work investigates how decoder-only Transformers infer graph structure from text by introducing Induced Substructure Filtration (ISF), a layer-wise process of progressively identifying subgraphs. It provides a rigorous framework including a Subgraph Isomorphism Indicator Tensor $ ext{T}(G,G')$ and a filtration-based theory that explains how substructures emerge across layers, supported by visualization and experiments on synthetic graphs. It further extends the framework with Thinking-in-Substructure (Tins) to decompose complex patterns and to attributed graphs, such as molecular graphs, demonstrating robust substructure extraction and relevance to graph reasoning in LLMs. The findings suggest a unified perspective on graph understanding in Transformers, with practical implications for building graph-aware LLMs and graph foundation models that can handle composite and labeled graphs. The work highlights the potential to improve efficiency and scalability by decomposing reasoning into substructures and by leveraging prompt design and input representations to align model behavior with graph topology, while outlining limitations and avenues for future theory and experimentation.
Abstract
Recent studies suggest that large language models (LLMs) possess the capability to solve graph reasoning tasks. Notably, even when graph structures are embedded within textual descriptions, LLMs can still effectively answer related questions. This raises a fundamental question: How can a decoder-only Transformer architecture understand underlying graph structures? To address this, we start with the substructure extraction task, interpreting the inner mechanisms inside the transformers and analyzing the impact of the input queries. Specifically, through both empirical results and theoretical analysis, we present Induced Substructure Filtration (ISF), a perspective that captures the substructure identification in the multi-layer transformers. We further validate the ISF process in LLMs, revealing consistent internal dynamics across layers. Building on these insights, we explore the broader capabilities of Transformers in handling diverse graph types. Specifically, we introduce the concept of thinking in substructures to efficiently extract complex composite patterns, and demonstrate that decoder-only Transformers can successfully extract substructures from attributed graphs, such as molecular graphs. Together, our findings offer a new insight on how sequence-based Transformers perform the substructure extraction task over graph data.