Do Language Models Follow Occam's Razor? An Evaluation of Parsimony in Inductive and Abductive Reasoning

Abstract

Non-deductive reasoning, encompassing inductive and abductive reasoning, is essential in addressing complex real-world questions. One key feature of inductive and abductive reasoning is that there are many valid hypotheses; the simplest ones (those that adhere to Occam's Razor) are often most useful. However, this aspect is ignored in recent work that evaluates the non-deductive reasoning capabilities of large language models (LLMs). This work fills this gap, focusing on understanding whether the inductive and abductive reasoning capabilities of LLMs adhere to Occam's Razor, while also examining the correctness of their reasoning. To accomplish this goal, we introduce a framework to synthetically generate reasoning questions that (a) require inductive reasoning and abductive reasoning simultaneously; (b) is readily extended to produce any abductive/inductive reasoning question expressible in first-order logic. The task for the intelligent agent is to produce hypotheses to explain observations under a given world model. We also propose a new automated metric to assess whether hypotheses quantitatively adhere to Occam's Razor; those hypotheses that are correct and simplest are considered high-quality. Our findings on state-of-the-art LLMs suggest that LLMs can perform inductive and abductive reasoning in simple scenarios, but struggle with complex world models and with producing high-quality hypotheses, even with popular reasoning-enhancing techniques such as in-context learning and RLVR.

Figures (10)

• Figure 1: An example of a question requiring inductive and abductive reasoning. Note that the dashed arrow means the subtype relationship All rats are mammals is hidden and needs to be inferred (i.e., a hypothesis to propose). This example uses a real-world model for illustrative purposes. In our dataset, we use fictional world models.
• Figure 2: Pipeline for generating each reasoning question in InAbHyD.
• Figure 3: Weak accuracy, strong accuracy, and quality versus the height of the ontology tree with single hypothesis.
• Figure 4: Weak accuracy, strong accuracy, and quality versus the height of the ontology tree with multiple hypotheses under zero-shot (top), 8-shot in-distribution (middle), and 8-shot out-of-distribution (bottom). We list the absolute results of the 8-shot in-distribution and out-of-distribution in Appendix \ref{['sec:moreresults']}.
• Figure 5: Results for Llama3-70B and two large reasoning models gpt-5.4 and o3 with chain-of-thought prompting (top) and buffer-of-thought prompting (bottom).