Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning to Solve Complex Problems via Dataset Decomposition

Wanru Zhao, Lucas Caccia, Zhengyan Shi, Minseon Kim, Weijia Xu, Alessandro Sordoni

TL;DR

This research explores a reverse curriculum generation approach that recursively decomposes complex datasets into simpler, more learnable components, and proposes a teacher-student framework where the teacher is equipped with the ability to reason step-by-step, allowing curriculum construction over decomposed data.

Abstract

Curriculum learning is a class of training strategies that organizes the data being exposed to a model by difficulty, gradually from simpler to more complex examples. This research explores a reverse curriculum generation approach that recursively decomposes complex datasets into simpler, more learnable components. We propose a teacher-student framework where the teacher is equipped with the ability to reason step-by-step, which is used to recursively generate easier versions of examples, enabling the student model to progressively master difficult tasks. We propose a novel scoring system to measure data difficulty based on its structural complexity and conceptual depth, allowing curriculum construction over decomposed data. Experiments on math datasets (MATH and AIME) and code generation datasets demonstrate that models trained with curricula generated by our approach exhibit superior performance compared to standard training on original datasets.

Learning to Solve Complex Problems via Dataset Decomposition

TL;DR

This research explores a reverse curriculum generation approach that recursively decomposes complex datasets into simpler, more learnable components, and proposes a teacher-student framework where the teacher is equipped with the ability to reason step-by-step, allowing curriculum construction over decomposed data.

Abstract

Curriculum learning is a class of training strategies that organizes the data being exposed to a model by difficulty, gradually from simpler to more complex examples. This research explores a reverse curriculum generation approach that recursively decomposes complex datasets into simpler, more learnable components. We propose a teacher-student framework where the teacher is equipped with the ability to reason step-by-step, which is used to recursively generate easier versions of examples, enabling the student model to progressively master difficult tasks. We propose a novel scoring system to measure data difficulty based on its structural complexity and conceptual depth, allowing curriculum construction over decomposed data. Experiments on math datasets (MATH and AIME) and code generation datasets demonstrate that models trained with curricula generated by our approach exhibit superior performance compared to standard training on original datasets.
Paper Structure (27 sections, 8 equations, 4 figures, 9 tables, 1 algorithm)

This paper contains 27 sections, 8 equations, 4 figures, 9 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Data Augmentation
  4. Curriculum Learning
  5. Decomp: Dataset Decomposition
  6. How to generate subproblems? - Recursive Problem Decomposition
  7. How difficult is a certain (sub)problem? -- Concept Dependency Graph
  8. Concept Clustering via Embedding Similarity
  9. Difficulty Measurement via Structural and Conceptual Features
  10. Curriculum Learning via Difficulty Measurement
  11. Experiments and Results
  12. Experimental Setup
  13. Models
  14. Datasets
  15. Training Details
  16. ...and 12 more sections

Figures (4)

  • Figure 1: Left: We recursively decompose a math example $(q, cot, a)$ into a set of smaller problems (depth 2 in the figure). We first split the $cot$ into steps, then create a question for each step and an associated concept tag. We then ask the teacher model to solve the question step-by-step. We verify the final answer by ensuring it is the same as the answer obtained without the ground-truth step in context. We then recursively apply this procedure until a stopping criterion. Right: We create a graph of tags, where dependency relation is given by the hierarchy in the decomposition tree. The graph of tags is used to quantify the difficulty of a generated sub-problem.
  • Figure 2: Left: We show the decomposition of the math problem on the top obtained with our method, along with the associated concept tags. The problems in the solid white boxes are the generated sub-problems. Right: The graph of concept tags, obtained by connecting the tags across the dataset examples.
  • Figure 3: Concept dependency graph constructed during the AIME data decomposition process. Nodes represent mathematical concepts, and edges indicate prerequisite relationships between concepts.
  • Figure 4: Zero‐shot performance of the Qwen3-4B-Base model across difficulty quintiles. Accuracy decreases as problem difficulty increases, validating our proposed difficulty metric.