A self-evolving multi-role collaborative framework with fine-grained difficulty guidance for innovative mathematical problem generation

TL;DR

This work tackles Innovative Math Problem Generation (IMPG) by introducing a self-evolving, multi-role collaborative framework that uses a fine-grained 16-bit difficulty encoding and a data-driven path sampling algorithm (DAPS) to produce semantically coherent problem encodings. A quintet of roles (sampler, generator, evaluator, state machine, memory) enables closed-loop generation with apprentice and expert modes, while post-training via Continual Pre-Training (CPT), Supervised Fine-Tuning (SFT), and Group Relative Policy Optimization (GRPO) fosters improvement and self-evolution through knowledge distillation from an expert evaluator. The authors construct the HSM3K-CN dataset for high-quality high-school problems and demonstrate that their approach significantly boosts problem innovation while maintaining high correctness, outperforming both open- and closed-source baselines across multiple evaluation metrics, including G-eval, MMB, and Elo ratings. The work provides a practical, scalable framework for automatic, controllable math problem generation with potential for large-scale deployment in intelligent education systems. It also highlights the trade-offs between correctness and innovation and shows how explicit difficulty encoding and distillation-based self-improvement can mitigate the so-called Innovation Curse.

Abstract

Mathematical problem generation (MPG) is a significant research direction in the field of intelligent education. In recent years, the rapid development of large language models (LLMs) has enabled new technological approaches to problem-generation tasks. Although existing LLMs can achieve high correctness rates, they generally lack innovation and exhibit poor discrimination. In this paper, we propose the task of innovative math problem generation (IMPG). To solve the IMPG task, this paper proposes a self-evolving, multi-role collaborative framework with fine-grained difficulty guidance. First, a multi-role collaborative mechanism comprising a sampler, generator, evaluator, state machine, and memory is constructed, ensuring the correctness of generated problems through iterative optimization informed by self-assessment and external feedback. Second, we introduce an improved difficulty model to quantify difficulty and provide fine-grained guidance. We adopt the data-driven association-guided path sampling (DAPS) algorithm to enhance the semantic rationality of sampled encodings. Third, we construct the HSM3K-CN dataset, which comprises high-quality high school math problems. A multi-stage training pipeline is adopted, incorporating continual pre-training (CPT), supervised fine-tuning (SFT), and group relative policy optimization (GRPO), to enhance the generation and evaluation capabilities of the base model. Finally, system self-evolution is achieved by transferring evaluation capabilities from the expert model to the apprentice model via distillation. Experiments show that, compared to baseline models, our proposed method significantly improves the innovation of the generated problems while maintaining a high correctness rate.

Figures (15)

• Figure 1: The limitations of the post-training LLMs and general-purpose LLMs in IMPG task. CDM represents the cognitive diagnosis model. MCP denotes the model context protocol for standardized communication between LLMs and external tools. These limitations will lead to futile efforts and low efficiency for users.
• Figure 2: Overall framework for multi-role collaboration.
• Figure 3: The complete post-training pipeline for self-evolving MPG. The pipeline consists of four stages: CPT with knowledge graph, key knowledge, and algebraic computation; SFT using LoRA fine-tuning on HSM3K-CN datasets and implementing knowledge distillation from expert model; GRPO with parallel reward computation; vLLM acceleration and Gradio UI optimization for efficient inference.
• Figure 4: The test results of all baseline models and our models, including apprentice mode and expert mode on the SIMU-90 dataset. (a) Performance of each model across nine evaluation dimensions. These dimensions encompass five basic dimensions and four advanced dimensions, providing a specific evaluation of the quality of generated problems. (b) Comparison of overall quality scores across models. The bar length represents the average score across nine dimensions, reflecting the overall quality of generated problems. The bar color indicates the average score of three core dimensions of the IMPG task, i.e., Correctness-P, Correctness-S, and Innovation.
• Figure 5: Human preference evaluation of problems generated by human experts and our model (Best@3), including apprentice mode and expert mode across each dimension. Preferences are classified into five levels, with the length of each bar segment indicating the number of evaluators for each preference level.