Position: On the Methodological Pitfalls of Evaluating Base LLMs for Reasoning
Jason Chan, Zhixue Zhao, Robert Gaizauskas
TL;DR
This paper argues that evaluating base LLMs on reasoning is methodologically problematic due to a mismatch between their linguistic-plausibility training objective and normative reasoning goals. Through a case study, it shows that logically correct or incorrect conclusions can arise as incidental byproducts of surface patterns rather than genuine reasoning. It highlights the instruction-following confound and asserts that findings about base LLMs do not reliably generalize to instruct LLMs. The authors recommend shifting focus to instruct LLMs for reasoning evaluation and reexamining prior base LLM conclusions accordingly.
Abstract
Existing work investigates the reasoning capabilities of large language models (LLMs) to uncover their limitations, human-like biases and underlying processes. Such studies include evaluations of base LLMs (pre-trained on unlabeled corpora only) for this purpose. Our position paper argues that evaluating base LLMs' reasoning capabilities raises inherent methodological concerns that are overlooked in such existing studies. We highlight the fundamental mismatch between base LLMs' pretraining objective and normative qualities, such as correctness, by which reasoning is assessed. In particular, we show how base LLMs generate logically valid or invalid conclusions as coincidental byproducts of conforming to purely linguistic patterns of statistical plausibility. This fundamental mismatch challenges the assumptions that (a) base LLMs' outputs can be assessed as their bona fide attempts at correct answers or conclusions; and (b) conclusions about base LLMs' reasoning can generalize to post-trained LLMs optimized for successful instruction-following. We call for a critical re-examination of existing work that relies implicitly on these assumptions, and for future work to account for these methodological pitfalls.