Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dual Instruction Tuning with Large Language Models for Mathematical Reasoning

Yongwei Zhou, Tiejun Zhao

TL;DR

This work tackles persistent errors in chain-of-thought reasoning by introducing a dual instruction tuning framework that trains LLMs on forward (IRSP) and reverse (IR) reasoning tasks. By constructing targeted IRSP and IR data from MathInstruct and performing multi-task fine-tuning, the approach enhances instruction understanding and execution across diverse mathematical domains. Empirical results show meaningful improvements in in-domain and out-of-domain tasks, with notable gains on several benchmarks and improved domain generalization, though gaps remain versus the largest closed-source models. The method offers a practical path to more reliable mathematical reasoning in LLMs, especially when full-scale pretraining on broad mathematical knowledge is not feasible.

Abstract

Recent advancements highlight the success of instruction tuning with large language models (LLMs) utilizing Chain-of-Thought (CoT) data for mathematical reasoning tasks. Despite the fine-tuned LLMs, challenges persist, such as incorrect, missing, and redundant steps in CoT generation leading to inaccuracies in answer predictions. To alleviate this problem, we propose a dual instruction tuning strategy to meticulously model mathematical reasoning from both forward and reverse directions. This involves introducing the Intermediate Reasoning State Prediction task (forward reasoning) and the Instruction Reconstruction task (reverse reasoning) to enhance the LLMs' understanding and execution of instructions. Training instances for these tasks are constructed based on existing mathematical instruction tuning datasets. Subsequently, LLMs undergo multi-task fine-tuning using both existing mathematical instructions and the newly created data. Comprehensive experiments validate the effectiveness and domain generalization of the dual instruction tuning strategy across various mathematical reasoning tasks.

Dual Instruction Tuning with Large Language Models for Mathematical Reasoning

TL;DR

This work tackles persistent errors in chain-of-thought reasoning by introducing a dual instruction tuning framework that trains LLMs on forward (IRSP) and reverse (IR) reasoning tasks. By constructing targeted IRSP and IR data from MathInstruct and performing multi-task fine-tuning, the approach enhances instruction understanding and execution across diverse mathematical domains. Empirical results show meaningful improvements in in-domain and out-of-domain tasks, with notable gains on several benchmarks and improved domain generalization, though gaps remain versus the largest closed-source models. The method offers a practical path to more reliable mathematical reasoning in LLMs, especially when full-scale pretraining on broad mathematical knowledge is not feasible.

Abstract

Recent advancements highlight the success of instruction tuning with large language models (LLMs) utilizing Chain-of-Thought (CoT) data for mathematical reasoning tasks. Despite the fine-tuned LLMs, challenges persist, such as incorrect, missing, and redundant steps in CoT generation leading to inaccuracies in answer predictions. To alleviate this problem, we propose a dual instruction tuning strategy to meticulously model mathematical reasoning from both forward and reverse directions. This involves introducing the Intermediate Reasoning State Prediction task (forward reasoning) and the Instruction Reconstruction task (reverse reasoning) to enhance the LLMs' understanding and execution of instructions. Training instances for these tasks are constructed based on existing mathematical instruction tuning datasets. Subsequently, LLMs undergo multi-task fine-tuning using both existing mathematical instructions and the newly created data. Comprehensive experiments validate the effectiveness and domain generalization of the dual instruction tuning strategy across various mathematical reasoning tasks.
Paper Structure (28 sections, 1 equation, 7 figures, 6 tables)

This paper contains 28 sections, 1 equation, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Dual Instruction Tuning Strategy
  4. Intermediate Reasoning State Prediction (IRSP)
  5. Instruction Reconstruction (IR)
  6. Training and Inference
  7. Experiment
  8. Experiment Settings and Evaluation Metrics
  9. Baselines
  10. Results
  11. Ablation Study
  12. Conclusion
  13. Appendix
  14. Implement Details
  15. Training Data and Evaluation Tasks
  16. ...and 13 more sections

Figures (7)

  • Figure 1: The errors, omissions, and redundancies in the generation process of reasoning steps
  • Figure 2: Illustration of instruction tuning process based on the CodeLlama model
  • Figure 3: Two examples for the IRSP and IR tasks
  • Figure 4: The proportion of data for Intermediate Reasoning State Prediction and Instruction Reconstruction tasks, as well as the masking ratio of reasoning states and instruction
  • Figure 5: A case study on GSM8K and SVAMP datasets for comparison of baseline and our method
  • ...and 2 more figures