From Reasoning to Learning: A Survey on Hypothesis Discovery and Rule Learning with Large Language Models

TL;DR

This survey tackles the challenge of enabling Large Language Models to autonomously discover and refine knowledge through hypotheses and rules, guided by Peirce’s abduction, deduction, and induction. It proposes a structured taxonomy—generation, application, validation, and integrated discovery—and reviews methods across natural-language and formal-language representations, highlighting prompting, retrieval-augmented generation, and human-in-the-loop approaches. The work critically assesses evaluation strategies, benchmarks, and gaps, emphasizing the mismatch between open-ended natural language reasoning and rigorous formal-language evaluation, and advocates for integrated, dynamic environments to approximate real-world scientific discovery. The findings underscore the potential of LLMs as engines of genuine innovation while outlining concrete directions for benchmarks, evaluation, and end-to-end hypothesis-discovery systems with proactive evidence gathering and robust validation. This framework aims to advance LLM-driven scientific inquiry by moving beyond instruction-following toward autonomous hypothesis generation, testing, and refinement with real-world impact.

Abstract

Since the advent of Large Language Models (LLMs), efforts have largely focused on improving their instruction-following and deductive reasoning abilities, leaving open the question of whether these models can truly discover new knowledge. In pursuit of artificial general intelligence (AGI), there is a growing need for models that not only execute commands or retrieve information but also learn, reason, and generate new knowledge by formulating novel hypotheses and theories that deepen our understanding of the world. Guided by Peirce's framework of abduction, deduction, and induction, this survey offers a structured lens to examine LLM-based hypothesis discovery. We synthesize existing work in hypothesis generation, application, and validation, identifying both key achievements and critical gaps. By unifying these threads, we illuminate how LLMs might evolve from mere ``information executors'' into engines of genuine innovation, potentially transforming research, science, and real-world problem solving.

Figures (3)

• Figure 1: Taxonomy for Hypothesis Discovery with LLMs. Our survey categorizes work into four topics based on Peirce’s definition of hypothesis discovery: Generation (creating hypotheses that explain given observations with abduction), Application (deducing new observations from established hypotheses with deduction), Validation (verifying and refining hypotheses against new evidence with induction), and Integrated Hypothesis Discovery (examining the dynamic interdependencies among these components in a continuous, iterative process).
• Figure 2: On the left-hand side, a) illustrates Peirce's framework for hypothesis discovery through abduction, deduction, and induction. The process begins with abduction, which generates explanatory hypotheses based on an initial set of observations. Deduction is then used to apply these hypotheses and derive predictions. Induction evaluates how well the predicted observations align with actual outcomes, updating the confidence of the hypotheses or rejecting those that are no longer valid. This process is iterative: validated hypotheses may be refined through further rounds of abduction using updated observations, gradually leading to more robust theories. On the right-hand side, b) provides a simple example that illustrates this process.
• Figure 3: Differences and similarities among different types of hypothesis discovery tasks