Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ControlMath: Controllable Data Generation Promotes Math Generalist Models

Nuo Chen, Ning Wu, Jianhui Chang, Jia Li

TL;DR

ControlMath, an iterative method involving an equation-generator module and two LLM-based agents that enables the generation of diverse math problems, not limited to specific domains or distributions, is proposed.

Abstract

Utilizing large language models (LLMs) for data augmentation has yielded encouraging results in mathematical reasoning. However, these approaches face constraints in problem diversity, potentially restricting them to in-domain/distribution data generation. To this end, we propose ControlMath, an iterative method involving an equation-generator module and two LLM-based agents. The module creates diverse equations, which the Problem-Crafter agent then transforms into math word problems. The Reverse-Agent filters and selects high-quality data, adhering to the "less is more" principle, achieving better results with fewer data points. This approach enables the generation of diverse math problems, not limited to specific domains or distributions. As a result, we collect ControlMathQA, which involves 190k math word problems. Extensive results prove that combining our dataset with in-domain datasets like GSM8K can help improve the model's mathematical ability to generalize, leading to improved performances both within and beyond specific domains.

ControlMath: Controllable Data Generation Promotes Math Generalist Models

TL;DR

ControlMath, an iterative method involving an equation-generator module and two LLM-based agents that enables the generation of diverse math problems, not limited to specific domains or distributions, is proposed.

Abstract

Utilizing large language models (LLMs) for data augmentation has yielded encouraging results in mathematical reasoning. However, these approaches face constraints in problem diversity, potentially restricting them to in-domain/distribution data generation. To this end, we propose ControlMath, an iterative method involving an equation-generator module and two LLM-based agents. The module creates diverse equations, which the Problem-Crafter agent then transforms into math word problems. The Reverse-Agent filters and selects high-quality data, adhering to the "less is more" principle, achieving better results with fewer data points. This approach enables the generation of diverse math problems, not limited to specific domains or distributions. As a result, we collect ControlMathQA, which involves 190k math word problems. Extensive results prove that combining our dataset with in-domain datasets like GSM8K can help improve the model's mathematical ability to generalize, leading to improved performances both within and beyond specific domains.
Paper Structure (31 sections, 4 equations, 7 figures, 8 tables)

This paper contains 31 sections, 4 equations, 7 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Mathematical Reasoning with LLMs.
  4. Data Augmentation with LLMs.
  5. ControlMath
  6. Controllable Equation Generation
  7. Problem-Crafter Agent.
  8. Efficient Data Selection: More data is not always helpful
  9. Problem-Rewriter Agent.
  10. Efficient Data Selection.
  11. Validation.
  12. Iterative Process.
  13. Why we need to rewrite the problem?
  14. Correlation with AdaBoost.
  15. Statistics
  16. ...and 16 more sections

Figures (7)

  • Figure 1: We train LLaMA 2-7B model llama2 with different training corpus and present results in out-of-domain/distribution datasets: GSM8K-Hard gao2023pal, SVAMP-Hard chen2023good, DM-Polynomials and Probability subdatasets saxton2018analysing.
  • Figure 2: The overview of our ControlMath. Here, we present the example of generating multi-calculation math word problems.
  • Figure 3: Here, we present the LLaMA 2-7B performances with different size corpus when we don't apply our efficient data selection strategy. Here, we train the model with ControlMathQA and GSM8K.
  • Figure 4: Here, we present the LLaMA 2-7B performances with different training corpus in GSM8K.
  • Figure 5: Here, we present the LLaMA 2-7B performances with training different corpus.
  • ...and 2 more figures