Evaluating Large Language Models on Solved and Unsolved Problems in Graph Theory: Implications for Computing Education

TL;DR

This paper investigates how an LLM handles two graph theory problems, one solved and one open, using an eight-stage protocol that models authentic mathematical inquiry, focusing on Problem 1: If $G$ is nongraceful, is $L(G)$ graceful? and Problem 2: If $L(G)$ is graceful, is $G$ graceful? Findings show reliable interpretation and proof for the solved problem, with expert verification, while the open problem yields coherent exploratory reasoning without generating new insights. The approach demonstrates the model’s strength in organizing established theory and identifying relevant definitions and strategies, but highlights limitations in original mathematical discovery and abstract reasoning required for open problems. The work informs computing education by clarifying how LLMs can support conceptual exploration and the necessity of explicit verification and rigorous argumentation in advanced coursework and research.

Abstract

Large Language Models are increasingly used by students to explore advanced material in computer science, including graph theory. As these tools become integrated into undergraduate and graduate coursework, it is important to understand how reliably they support mathematically rigorous thinking. This study examines the performance of a LLM on two related graph theoretic problems: a solved problem concerning the gracefulness of line graphs and an open problem for which no solution is currently known. We use an eight stage evaluation protocol that reflects authentic mathematical inquiry, including interpretation, exploration, strategy formation, and proof construction. The model performed strongly on the solved problem, producing correct definitions, identifying relevant structures, recalling appropriate results without hallucination, and constructing a valid proof confirmed by a graph theory expert. For the open problem, the model generated coherent interpretations and plausible exploratory strategies but did not advance toward a solution. It did not fabricate results and instead acknowledged uncertainty, which is consistent with the explicit prompting instructions that directed the model to avoid inventing theorems or unsupported claims. These findings indicate that LLMs can support exploration of established material but remain limited in tasks requiring novel mathematical insight or critical structural reasoning. For computing education, this distinction highlights the importance of guiding students to use LLMs for conceptual exploration while relying on independent verification and rigorous argumentation for formal problem solving.