On Learning Verifiers for Chain-of-Thought Reasoning
Maria-Florina Balcan, Avrim Blum, Zhiyuan Li, Dravyansh Sharma
TL;DR
The paper tackles the challenge of verifying Chain-of-Thought reasoning by framing verifiers as learnable predictors that assess stepwise validity under a PAC framework. It introduces two verifier paradigms: simple verifiers that learn from random, labeled CoT traces and trustable verifiers that leverage a gold-standard set of proofs to achieve soundness across distributions, addressing distribution-shift risks in interactive reasoning. The authors derive formal sample-complexity bounds for both finite and VC-bounded verifier classes under SVPAC and TVPAC settings, establishing upper bounds and fundamental lower bounds that reveal the tradeoffs when gold-standard guidance is limited or unavailable. A key insight is that small numbers of gold proofs enable efficient learning, while without such a constraint, the complexity scales linearly with the verifier class size, though intersection-closed classes offer improved bounds via closure. These results lay theoretical groundwork for building reliable CoT verifiers and potentially guiding LLMs toward correct, verifiable reasoning in practice.
Abstract
Chain-of-Thought reasoning has emerged as a powerful approach for solving complex mathematical and logical problems. However, it can often veer off track through incorrect or unsubstantiated inferences. Formal mathematical reasoning, which can be checked with a formal verifier, is one approach to addressing this issue. However, currently LLMs are simply not good enough to solve complex problems in a formal way, and even just formalizing an informal problem statement can be challenging. Motivated by this fact, in this work we consider the problem of learning reliable verifiers for natural language Chain-of-Thought reasoning. That is, given a problem statement and step-by-step solution in natural language, the aim of the verifier is to output [Yes] if the reasoning steps in the solution are all valid, and [No] otherwise. In this work we give a formal PAC-learning framework for studying this problem. We propose and analyze several natural verification goals, at different levels of strength, in this framework. We provide sample complexity upper-bounds for learning verifiers satisfying these goals, as well as lower-bound and impossibility results for learning other natural verification objectives without additional assumptions.