Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing the Reasoning Capabilities of Small Language Models via Solution Guidance Fine-Tuning

Jing Bi, Yuting Wu, Weiwei Xing, Zhenjie Wei

TL;DR

The paper tackles the challenge of enabling robust reasoning in small language models under data constraints. It introduces Solution Guidance (SG), a reasoning strategy that emphasizes problem understanding over computations, and Solution-Guidance Fine-Tuning (SGFT), a plug-and-play training paradigm that teaches a small model to generate SG data which then prompts another model to produce final answers. By leveraging a two-model collaborative inference and a data-efficient SG generation pipeline (via LISA-based fine-tuning and prompt engineering), the approach yields substantial accuracy gains on mathematical and commonsense reasoning benchmarks while requiring far less data than traditional Chain-of-Thought fine-tuning. The method runs on consumer-grade GPUs and preserves the broad capabilities of small models, offering a practical path to deploying reasoning-enhanced SLMs in resource-constrained settings.

Abstract

Large language models (LLMs) have demonstrated remarkable performance across a wide range of tasks. Advances in prompt engineering and fine-tuning techniques have further enhanced their ability to address complex reasoning challenges. However, these advanced capabilities are often exclusive to models exceeding 100 billion parameters. Although Chain-of-Thought (CoT) fine-tuning methods have been explored for smaller models (under 10 billion parameters), they typically depend on extensive CoT training data, which can introduce inconsistencies and limit effectiveness in low-data settings. To overcome these limitations, this paper introduce a new reasoning strategy Solution Guidance (SG) and a plug-and-play training paradigm Solution-Guidance Fine-Tuning (SGFT) for enhancing the reasoning capabilities of small language models. SG focuses on problem understanding and decomposition at the semantic and logical levels, rather than specific computations, which can effectively improve the SLMs' generalization and reasoning abilities. With only a small amount of SG training data, SGFT can fine-tune a SLM to produce accurate problem-solving guidances, which can then be flexibly fed to any SLM as prompts, enabling it to generate correct answers directly. Experimental results demonstrate that our method significantly improves the performance of SLMs on various reasoning tasks, enhancing both their practicality and efficiency within resource-constrained environments.

Enhancing the Reasoning Capabilities of Small Language Models via Solution Guidance Fine-Tuning

TL;DR

The paper tackles the challenge of enabling robust reasoning in small language models under data constraints. It introduces Solution Guidance (SG), a reasoning strategy that emphasizes problem understanding over computations, and Solution-Guidance Fine-Tuning (SGFT), a plug-and-play training paradigm that teaches a small model to generate SG data which then prompts another model to produce final answers. By leveraging a two-model collaborative inference and a data-efficient SG generation pipeline (via LISA-based fine-tuning and prompt engineering), the approach yields substantial accuracy gains on mathematical and commonsense reasoning benchmarks while requiring far less data than traditional Chain-of-Thought fine-tuning. The method runs on consumer-grade GPUs and preserves the broad capabilities of small models, offering a practical path to deploying reasoning-enhanced SLMs in resource-constrained settings.

Abstract

Large language models (LLMs) have demonstrated remarkable performance across a wide range of tasks. Advances in prompt engineering and fine-tuning techniques have further enhanced their ability to address complex reasoning challenges. However, these advanced capabilities are often exclusive to models exceeding 100 billion parameters. Although Chain-of-Thought (CoT) fine-tuning methods have been explored for smaller models (under 10 billion parameters), they typically depend on extensive CoT training data, which can introduce inconsistencies and limit effectiveness in low-data settings. To overcome these limitations, this paper introduce a new reasoning strategy Solution Guidance (SG) and a plug-and-play training paradigm Solution-Guidance Fine-Tuning (SGFT) for enhancing the reasoning capabilities of small language models. SG focuses on problem understanding and decomposition at the semantic and logical levels, rather than specific computations, which can effectively improve the SLMs' generalization and reasoning abilities. With only a small amount of SG training data, SGFT can fine-tune a SLM to produce accurate problem-solving guidances, which can then be flexibly fed to any SLM as prompts, enabling it to generate correct answers directly. Experimental results demonstrate that our method significantly improves the performance of SLMs on various reasoning tasks, enhancing both their practicality and efficiency within resource-constrained environments.

Paper Structure

This paper contains 24 sections, 2 equations, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Chain-of-Thought Reasoning.
  4. Distillation of Reasoning Ability.
  5. Method
  6. Solution Guidance Generation
  7. Data Paradigm Setup.
  8. Generation SG Data from LLM.
  9. Data Cleaning.
  10. Solution-Guidance Fine-Tuning (SGFT)
  11. Fine-Tuning Method Based on LISA.
  12. Improvements in Fine-Tuning Based on Prompt Engineering.
  13. Collaborative Inference of Small Models
  14. Experiments
  15. Experimental Setup
  16. ...and 9 more sections

Figures (4)

  • Figure 1: A wrong case of existing CoT reasoning.
  • Figure 2: The figure illustrates the complete architecture of our approach. In this diagram, SLM denotes the Small Language Model, LLM represents the Large Language Model, and SG (Solution Guidance) refers to the data schema introduced in our method.
  • Figure 3: Relationship between the amount of training data required by the CoT method and our method and the accuracy on the GSM8K data set, with the x-axis representing the amount of data and the y-axis representing the accuracy.
  • Figure 4: Comparison of the accuracy of four strategies for collaborative reasoning with small models, M represents an original model, M_CoT represents a model fine-tuned with CoT, and M_SG represents a model fine-tuned with SG.Test results on the GSM8K dataset.