Assessing LLM Reasoning Steps via Principal Knowledge Grounding

TL;DR

This work tackles the challenge of validating that LLM intermediate reasoning is grounded in essential knowledge. It introduces a Principal Knowledge Collection (112k PK units) derived from the MMLU benchmark, and defines knowledge-grounded metrics—Knowledge Precision (KP), Knowledge Recall (KR), and F1—to quantify how reasoning recalls and applies requisite knowledge. An open-weight evaluator is distilled from a strong teacher model to enable cost-effective assessment of PK usage in reasoning. By integrating these metrics into Direct Preference Optimization and reasoning-focused variants, the approach enables controlled reasoning depth and improved end-task performance, while also reducing token consumption through knowledge-aware guidance. Overall, the framework provides interpretable diagnostics for knowledge grounding and practical mechanisms to steer LLM reasoning toward concise, correct, and knowledge-grounded solutions.

Abstract

Step-by-step reasoning has become a standard approach for large language models (LLMs) to tackle complex tasks. While this paradigm has proven effective, it raises a fundamental question: How can we verify that an LLM's reasoning is accurately grounded in knowledge? To address this question, we introduce a novel evaluation suite that systematically assesses the knowledge grounding of intermediate reasoning. Our framework comprises three key components. (1) Principal Knowledge Collection, a large-scale repository of atomic knowledge essential for reasoning. Based on the collection, we propose (2) knowledge-grounded evaluation metrics designed to measure how well models recall and apply prerequisite knowledge in reasoning. These metrics are computed by our (3) evaluator LLM, a lightweight model optimized for cost-effective and reliable metric computation. Our evaluation suite demonstrates remarkable effectiveness in identifying missing or misapplied knowledge elements, providing crucial insights for uncovering fundamental reasoning deficiencies in LLMs. Beyond evaluation, we demonstrate how these metrics can be integrated into preference optimization, showcasing further applications of knowledge-grounded evaluation.

Figures (6)

• Figure 1: Example of a reasoning failure. The model correctly applies the knowledge of "combining fractions" but omits prerequisite steps such as "simplifying a fraction," which leads to an incorrect answer. This underscores the importance of evaluating reasoning steps through knowledge-grounded assessments.
• Figure 2: Overview of our evaluation suite for assessing LLM reasoning steps via knowledge grounding. We first construct the Principal Knowledge Collection (§ \ref{['sec:kc_construction']}) through a two-step process: Given a task, we (a) collect atomic knowledge crucial for task resolution from multiple top-performing LLMs. Subsequently, we (b) cluster these units into semantically coherent groups to obtain the principal knowledge for each cluster. Grounded on this collection, we (c) evaluate the model’s reasoning steps (§ \ref{['sec:eval_reasoning']}) by measuring whether the model accurately retrieves and applies the principal knowledge using our proposed knowledge recall and precision metrics.
• Figure 3: Accuracy improvement when incorrect questions are retried with additional knowledge. (a) Cases where all applied knowledge was correct but some necessary knowledge was missing (KP=1.0, KR < 1). (b) Cases where some knowledge was misapplied (KP < 1).
• Figure 4: Comparison of accuracy and token length in various data selection settings. Details about length based selection method are described in Appendix \ref{['appendix:length_base']}
• Figure 5: The average number of atomic knowledge generated by each LLM before and after clustering. We define principal knowledge as the closest knowledge to the centroid of each cluster and show the proportions of this knowledge coming from each model.