Can LLM Graph Reasoning Generalize beyond Pattern Memorization?

TL;DR

The NLGift benchmark is proposed, an evaluation suite of LLM graph reasoning generalization, and it is found that while post-training alignment is most promising for real-world tasks, empowering LLM graph reasoning to go beyond pattern memorization remains an open research question.

Abstract

Large language models (LLMs) demonstrate great potential for problems with implicit graphical structures, while recent works seek to enhance the graph reasoning capabilities of LLMs through specialized instruction tuning. The resulting 'graph LLMs' are evaluated with in-distribution settings only, thus it remains underexplored whether LLMs are learning generalizable graph reasoning skills or merely memorizing patterns in the synthetic training data. To this end, we propose the NLGift benchmark, an evaluation suite of LLM graph reasoning generalization: whether LLMs could go beyond semantic, numeric, structural, reasoning patterns in the synthetic training data and improve utility on real-world graph-based tasks. Extensive experiments with two LLMs across four graph reasoning tasks demonstrate that while generalization on simple patterns (semantic, numeric) is somewhat satisfactory, LLMs struggle to generalize across reasoning and real-world patterns, casting doubt on the benefit of synthetic graph tuning for real-world tasks with underlying network structures. We explore three strategies to improve LLM graph reasoning generalization, and we find that while post-training alignment is most promising for real-world tasks, empowering LLM graph reasoning to go beyond pattern memorization remains an open research question.

Figures (6)

• Figure 1: Overview of the NLGift Benchmark, featuring five types of graph reasoning patterns that are increasingly challenging in order. We present an example for each pattern to show the transfer from training to test sets.
• Figure 2: Results for real-world patterns, where the LLM is either untuned (zero shot), tuned with graph tasks related to the real-world problem (related tasks), or on the mixture of all synthetic tasks (all). We find no obvious benefits or even negative transfers of synthetic graph tuning for real-world graphical problems.
• Figure 3: Results for mixture of graph tasks. $a+b\times3$ indicates that the majority task (shortest path) is $a\%$ of training data while the other three tasks are $b\%$. The two yellow lines show performance upper bound (in-distribution training) and lower bound (zero-shot).
• Figure 4: Average frequency of five keywords for four semantic patterns and the corresponding in-distribution performance. Frequency and in-distribution performance are positively related.
• Figure 5: Different choice of PGR threshold and patterns' Strong Recovery ratio. The generalization gap between different distributions doesn't depend on the choice of the threshold $\lambda$, because there is not a single $\lambda$ shows perfect Strong Recovery ratio.