DataChef: Cooking Up Optimal Data Recipes for LLM Adaptation via Reinforcement Learning

TL;DR

This paper addresses the data bottleneck in adapting LLMs by formalizing end-to-end data recipe generation. It introduces DataChef-32B, which learns to design executable data pipelines and datasets via online reinforcement learning guided by a data verifier across a large, diverse task pool. Empirical results show DataChef-32B achieving performance on par with Gemini-3-Pro and outperforming open baselines on six held-out tasks, including notable math-domain adaptation and climate-domain tasks. The approach demonstrates the potential of automated, self-improving data curation for LLM training and highlights the viability of surrogate rewards to enable scalable online learning in data-centric AI.

Abstract

In the current landscape of Large Language Models (LLMs), the curation of large-scale, high-quality training data is a primary driver of model performance. A key lever is the \emph{data recipe}, which comprises a data processing pipeline to transform raw sources into training corpora. Despite the growing use of LLMs to automate individual data processing steps, such as data synthesis and filtering, the overall design of data recipes remains largely manual and labor-intensive, requiring substantial human expertise and iteration. To bridge this gap, we formulate \emph{end-to-end data recipe generation} for LLM adaptation. Given a target benchmark and a pool of available data sources, a model is required to output a complete data recipe that adapts a base LLM to the target task. We present DataChef-32B, which performs online reinforcement learning using a proxy reward that predicts downstream performance for candidate recipes. Across six held-out tasks, DataChef-32B produces practical recipes that reach comparable downstream performance to those curated by human experts. Notably, the recipe from DataChef-32B adapts Qwen3-1.7B-Base to the math domain, achieving 66.7 on AIME'25 and surpassing Qwen3-1.7B. This work sheds new light on automating LLM training and developing self-evolving AI systems.

Figures (7)

• Figure 1: Illustration of DataChef training framework. Given a task, a policy LLM generates a data recipe, which is executed to produce a training dataset. The Data Verifier then evaluates a sampled subset to provide a scalar reward, guiding the policy update via GRPO to optimize for data quality and executability.
• Figure 2: Correlation analysis of data evaluation metrics. (left) We summarize the Pearson correlation coefficients across all six evaluated tasks. (right) We detail the relationship between metric scores (X-axis) and downstream performance (Y-axis) on Language and Code tasks. The Data Verifier maintains a strong, consistent positive correlation across disparate domains. Please refer to Fig. \ref{['fig:appx-cor-all']} in Appx. \ref{['appx:additional-results-on-correlation-analysis']} for complete results.
• Figure 3: Analysis of RL Effectiveness. (a) RL training dynamics indicate that the policy consistently converges toward high-quality data recipe generation. (b) Evaluation results show that RL yields substantial improvements on out-of-domain tasks.
• Figure 4: Analysis of operation frequency in generated recipes. We compare the average number of function calls per recipe across different models.
• Figure 5: Visualization of data distribution in generated recipes. We project the source datasets and the data recipes generated by different models into a 2D embedding space.