Can LLMs perform structured graph reasoning?

TL;DR

This work investigates whether pretrained LLMs can perform structured graph reasoning beyond plain text by using semi-structured graph representations. It introduces a 10-task graph traversal benchmark spanning tree-based and grid-based graphs, with variations including weights, directed edges, and node-label jumbling, and evaluates five LLMs under zero/one/three-shot prompts while introducing a novel PathCompare prompting approach. The results reveal that tree-based traversals are generally easier than grid-based ones, that increased degrees of freedom per node and certain prompt choices diminish performance, and that a positive response bias leads models to sometimes output valid-looking but incorrect solutions; PathCompare mitigates some of these issues and improves accuracy across models. The findings expose limitations in current LLMs' ability to reason over structured graphs and offer practical prompting strategies and a dataset to spur further advances in structured, multi-hop reasoning. Overall, the PathCompare method provides a concrete, scalable way to boost graph reasoning in LLMs and highlights biases that future models must address for reliable structured reasoning in graphs.

Abstract

Pretrained Large Language Models (LLMs) have demonstrated various reasoning capabilities through language-based prompts alone, particularly in unstructured task settings (tasks purely based on language semantics). However, LLMs often struggle with structured tasks, because of the inherent incompatibility of input representation. Reducing structured tasks to uni-dimensional language semantics often renders the problem trivial. Keeping the trade-off between LLM compatibility and structure complexity in mind, we design various graph reasoning tasks as a proxy to semi-structured tasks in this paper, in order to test the ability to navigate through representations beyond plain text in various LLMs. Particularly, we design 10 distinct problems of graph traversal, each representing increasing levels of complexity, and benchmark 5 different instruct-finetuned LLMs (GPT-4, GPT-3.5, Claude-2, Llama-2 and Palm-2) on the aforementioned tasks. Further, we analyse the performance of models across various settings such as varying sizes of graphs as well as different forms of k-shot prompting. We highlight various limitations, biases and properties of LLMs through this benchmarking process, such as an inverse relation to the average degrees of freedom of traversal per node in graphs, the overall negative impact of k-shot prompting on graph reasoning tasks, and a positive response bias which prevents LLMs from identifying the absence of a valid solution. Finally, we introduce a new prompting technique specially designed for graph traversal tasks (PathCompare), which demonstrates a notable increase in the performance of LLMs in comparison to standard prompting techniques such as Chain-of-Thought (CoT).

Figures (5)

• Figure 1: Visualization of all problem categories considered for evaluating LLMs. For each problem, we create 3 variations -- O(10), O(20) and O(20) jumbled, representing increasing levels of difficulty.
• Figure 2: Baseline comparison of all model families on O(10) graph problems in 0-shot settings using partial accuracy. Partial accuracy gives a more granular insight into the performance of models, versus binary accuracy.
• Figure 3: Comparison of the effect of k-shot prompting on various LLMs. We observe that in more than half of all tasks, few-shot prompting leads to either a drop or an insignificant ($\leq$ 5%) improvement in accuracy in comparison to 0-shot prompting (depicted in blue). This is observed in 6/8 tasks for GPT3.5, 6/8 tasks for GPT4 and 5/8 tasks for Claude-2.
• Figure 4: Comparison of graph order performance in various LLMs. LLM performance accuracy consistently drops as the order of the graph is increased. Also, while keeping the order of graph magnitude at 20, a drop is observed in performance when the nodes are jumbled, depicting a bias in models to expect node order to be alphabetically arranged.
• Figure 5: Comparison of different prompting techniques. Our proposed prompting technique (PathCompare) demonstrates an improvement of accuracy in the majority of tasks across all models, i.e. 5/8 tasks for GPT3.5, 6/8 tasks for GPT4 and 5/8 tasks for Claude-2. However, one limitation of PathCompare is that it enhances positive response bias, as observed in the case of problem 2.4.