Gödel Test: Can Large Language Models Solve Easy Conjectures?

TL;DR

The paper introduces the Gödel Test to evaluate whether large language models can prove new, simple conjectures in advanced mathematics and reports on GPT-5’s performance across five submodular-optimization conjectures. It demonstrates that GPT-5 does well on single-path reasoning and can even derive valid alternative results, but struggles with cross-paper synthesis required for more complex proofs. The work highlights meaningful progress in routine mathematical reasoning and occasional originality while also exposing clear limitations in integrative, multi-technique proofs. It further surveys and formalizes several conjectures across submodular optimization, presenting a range of algorithmic frameworks (MGFW, bicriteria greedy, (weakly) DR-submodular maximization, and matroid-intersection) with conjectured guarantees, thereby outlining a roadmap for future frontier-model capabilities. Overall, the study provides an early, cautious optimism that future models may increasingly connect disparate proof techniques, provided prompts are carefully designed and external verification remains essential.

Abstract

Recent announcements from frontier AI model labs have highlighted strong results on high-school and undergraduate math competitions. Yet it remains unclear whether large language models can solve new, simple conjectures in more advanced areas of mathematics. We propose the Gödel Test: evaluating whether a model can produce correct proofs for very simple, previously unsolved conjectures. To this end, we study the performance of GPT-5 on five conjectures in combinatorial optimization. For each problem, we provided one or two source papers from which the conjecture arose, withheld our own conjecture, and then assessed the model's reasoning in detail. On the three easier problems, GPT-5 produced nearly correct solutions; for Problem 2 it even derived a different approximation guarantee that, upon checking, refuted our conjecture while providing a valid solution. The model failed on Problem 4, which required combining results from two papers. On Problem 5, a harder case without a validated conjecture, GPT-5 proposed the same algorithm we had in mind but failed in the analysis, suggesting the proof is more challenging than expected. Although our sample is small, the results point to meaningful progress on routine reasoning, occasional flashes of originality, and clear limitations when cross-paper synthesis is required. GPT-5 may represent an early step toward frontier models eventually passing the Gödel Test.