Step-by-Step Reasoning to Solve Grid Puzzles: Where do LLMs Falter?

TL;DR

GridPuzzle provides 274 rule-based grid puzzles to probe LLMs' reasoning beyond final answers. The authors build a manual reasoning-chain error taxonomy and an Auto-evaluator, plus PuzzleEval to score chain correctness without gold reasoning. Across multiple closed- and open-source LLMs, final accuracy is very low, but many reasoning steps are error-free; the dominant errors are wrong premises and incorrect eliminations, and open-source models lag behind proprietary ones. The work highlights the inadequacy of prompting-based improvements alone and offers datasets and metrics to drive deeper improvements in machine reasoning.

Abstract

Solving grid puzzles involves a significant amount of logical reasoning. Hence, it is a good domain to evaluate the reasoning capability of a model which can then guide us to improve the reasoning ability of models. However, most existing works evaluate only the final predicted answer of a puzzle, without delving into an in-depth analysis of the LLMs' reasoning chains (such as where they falter) or providing any finer metrics to evaluate them. Since LLMs may rely on simple heuristics or artifacts to predict the final answer, it is crucial to evaluate the generated reasoning chain beyond overall correctness measures, for accurately evaluating the reasoning abilities of LLMs. To this end, we first develop GridPuzzle, an evaluation dataset comprising 274 grid-based puzzles with different complexities. Second, we propose a new error taxonomy derived from manual analysis of reasoning chains from LLMs including GPT-4, Claude-3, Gemini, Mistral, and Llama-2. Then, we develop an LLM-based framework for large-scale subjective evaluation (i.e., identifying errors) and an objective metric, PuzzleEval, to evaluate the correctness of reasoning chains. Evaluating reasoning chains from LLMs leads to several interesting findings. We further show that existing prompting methods used for enhancing models' reasoning abilities do not improve performance on GridPuzzle. This highlights the importance of understanding fine-grained errors and presents a challenge for future research to enhance LLMs' puzzle-solving abilities by developing methods that address these errors. Data and source code are available at https://github.com/Mihir3009/GridPuzzle.

Figures (15)

• Figure 1: Schematic representation of proposed pipeline. Begins with the data collection of GridPuzzle dataset (top left) and evaluating various LLMs in zero-shot CoT setting (bottom left), then analyzing reasoning chains of LLMs manually to find various error types (top right) and automate this analysis process using LLM to check the correctness of reasoning chain by finding errors (bottom right).
• Figure 2: The process of calculating PuzzleEval metrics is described above. The reasoning chains are produced by our five LLMs and the gold solution is taken from our GridPuzzle dataset.
• Figure 3: Performance of five different LLMs in terms of accuracy on the GridPuzzle dataset.
• Figure 4: The percentage distribution of the broad error categories across the combined reasoning steps of all five LLMs. The total number of steps generated by each model is provided inside the round brackets below the model names.
• Figure 5: The first five sub-figures in the above section show the error Sub-category distribution over five LLMS. The last sub-figure denotes the top 10 error Sub category distribution across all model reasoning steps.