Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillation

TL;DR

PerSyn tackles the distillation bottleneck where the strongest teacher is not always optimal for a small student. It introduces a router-guided Route-then-Generate framework that assigns each prompt to an optimal teacher based on a joint learnability-quality objective, enabling teachers to synthesize only their assigned prompts. A Bradley–Terry–style router learns per-prompt teacher preferences from pairwise comparisons, with a combined reward balancing learnability and data quality, and the final synthetic dataset is used to fine-tune the student. Empirically, PerSyn yields consistent improvements across instruction tuning and math reasoning across model families and scales, and its analysis reveals quality plays a larger role than learnability while maintaining efficiency through selective generation. These findings offer practical pathways for efficient, personalized data synthesis in scalable distillation pipelines and motivate broader exploration of per-prompt routing in downstream tasks.

Abstract

Training student models on synthetic data generated by strong teacher models is a promising way to distilling the capabilities of teachers. However, recent studies show that stronger models are not always optimal teachers, revealing a mismatch between teacher outputs and student learnability. To address this issue, we propose PerSyn (Personalized data Synthesis), a novel synthesis strategy that operates under a new ``Route then Generate'' paradigm to create data tailored to each student model, enabling it to learn more effectively. Specifically, PerSyn first assigns each prompt to its optimal teacher via a query-level router that jointly considers student learnability and teacher response quality. Each teacher then synthesizes data only for its assigned prompts, making the process more efficient than the conventional ``Generate then Select'' paradigm, where all teachers must generate parallel responses for the entire prompt set before constructing the final dataset. Extensive experiments across different model families and scales demonstrate that PerSyn consistently achieves superior or comparable performance to all baselines in instruct tuning and math reasoning settings. Further analysis verifies the effectiveness of PerSyn and offers extra insights to propel future research.

Figures (10)

• Figure 1: Overview of the two paradigm for obtaining a personalized synthetic dataset. The left part illustrates how we select optimal teacher response for each prompt using the proposed criterion. This process follows the conventional "Generate then Select" approach, which requires parallel teacher responses for the entire prompt set (§ \ref{['sec:annotation_metric']}). In contrast, PerSyn adopts a more efficient "Route then Generate" paradigm: it first routes each prompt to an optimal teacher based on learnability and quality via a router-guided mechanism, and teachers generate responses only for their assigned prompts (§ \ref{['sec:transfer_paradigm']}). Details of router training are described in § \ref{['sec:obtain_router']}.
• Figure 2: Average results of the ablation studies on PerSyn across all benchmarks. "w/o" denotes the exclusion of a specific reward term from PerSyn when assigning prompts to teachers.
• Figure 3: The average results of baselines and PerSyn on four larger-scale student models spanning three model families in the instruction tuning setting. Detailed results are provided in Appendix \ref{['appendix:larger_size_result']}.
• Figure 4: The average results across different $\alpha$ values for three student models from distinct model families in the instruction tuning setting.
• Figure 5: The performance of the PerSyn router for the Qwen2.5-3B student model across different backbone model sizes and pairwise training dataset sizes in the instruction tuning setting. Notably, 500K pairwise training samples, which can be constructed from only 2.5K parallel teacher responses, are sufficient to obtain an effective PerSyn router. Similar observations in math reasoning setting are provided in Appendix \ref{['appendix:details_of_router']}.