Large Language Model for Science: A Study on P vs. NP

TL;DR

This work proposes LLM4Science, a framework in which large language models act as collaborative peers to accelerate scientific inquiry, using the P vs NP problem as a focal test. It introduces Socratic Reasoning, a modular prompting approach with deduction, transformation, decomposition, verification, and integration to guide LLMs through complex reasoning with self-reflection. A pilot study with GPT-4 demonstrates the approach can produce a reasoning pathway toward a P ≠ NP conclusion, illustrating the potential for LLMs to explore vast solution spaces and generate novel scientific insights. The discussion elaborates on the broader implications, including the role of AI as an innovation navigator, the balance between general-purpose and task-specific systems, and the cognitive impact of treating mathematics as a native language for AI-assisted discovery.

Abstract

In this work, we use large language models (LLMs) to augment and accelerate research on the P versus NP problem, one of the most important open problems in theoretical computer science and mathematics. Specifically, we propose Socratic reasoning, a general framework that promotes in-depth thinking with LLMs for complex problem-solving. Socratic reasoning encourages LLMs to recursively discover, solve, and integrate problems while facilitating self-evaluation and refinement. Our pilot study on the P vs. NP problem shows that GPT-4 successfully produces a proof schema and engages in rigorous reasoning throughout 97 dialogue turns, concluding "P $\neq$ NP", which is in alignment with (Xu and Zhou, 2023). The investigation uncovers novel insights within the extensive solution space of LLMs, shedding light on LLM for Science.

Figures (1)

• Figure 2.1: Example of a dialogue turn in Socratic Reasoning.