Key-Point-Driven Mathematical Reasoning Distillation of Large Language Model

TL;DR

KPDD enhances the reasoning performance of SLMs by breaking down the problem-solving process into three stages: Core Question Extraction, Problem-Solving Information Extraction, and Step-by-Step Solution, and KPDD-PoT achieves state-of-the-art performance in mathematical reasoning tasks.

Abstract

Large Language Models (LLMs) have demonstrated exceptional proficiency in mathematical reasoning tasks due to their extensive parameter counts and training on vast datasets. Despite these capabilities, deploying LLMs is hindered by their computational demands. Distilling LLM mathematical reasoning into Smaller Language Models (SLMs) has emerged as a solution to this challenge, although these smaller models often suffer from errors in calculation and semantic understanding. Prior work has proposed Program-of-Thought Distillation (PoTD) to avoid calculation error. To further address semantic understanding errors, we propose Key-Point-Driven Mathematical Reasoning Distillation (KPDD). KPDD enhances the reasoning performance of SLMs by breaking down the problem-solving process into three stages: Core Question Extraction, Problem-Solving Information Extraction, and Step-by-Step Solution. This method is further divided into KPDD-CoT, which generates Chain-of-Thought rationales, and KPDD-PoT, which creates Program-of-Thought rationales. The experiment results show that KPDD-CoT significantly improves reasoning abilities, while KPDD-PoT achieves state-of-the-art performance in mathematical reasoning tasks. Our approach effectively mitigates misunderstanding errors, advancing the deployment of efficient and capable SLMs.

Figures (5)

• Figure 1: Error analysis of 50 GSM8K problems with incorrect answers returned by CoTD using FlanT5-Base. The experimental results indicate that understanding errors and calculation errors are the major factors affecting CoTD's reasoning performance.
• Figure 2: Detailed data generation for KPDD-CoT. Initially, we use few-shot prompting to guide LLMs in producing reasoning processes. We then discard any reasoning process that does not align with the correct answers. In this way, we have constructed a high-quality KPDD-CoT reasoning distillation dataset.
• Figure 3: Effect of Reasoning Paths. We fine-tune FlanT5-Base with different reasoning paths to analyse the effect of reasoning paths. The experiment results shows that diverse reasoning paths can improve SLMs' reasoning performance.
• Figure 4: Error Analysis for SLMs. We conducted an error analysis of four different categories of distillation methods. The experiment results show that integrating multiple key points of the questions can significantly reduce SLMs' understanding errors, enhance the comprehension of the questions and further improve the reasoning performance of SLMs.
• Figure 5: Detailed data generation for KPDD-PoT. Similar to KPDD-CoT, we prompt LLMs to generate the reasoning process in the KPDD-PoT format. We extract the rationale in the PoT format from this process and run it through a Python interpreter. If there are errors or incorrect answers, we discard the reasoning process. Finally, we constructed a high-quality KPDD-PoT reasoning dataset.