Recognizing that high-quality training data is critical to performance, researchers are tackling the important challenge of optimizing data preparation for large-scale language models (LLMs). Yicheng Chen, Zerun Ma, Xinchen Xie, and yinging Li from Fudan University and Kai Chen from Shanghai AI Laboratory present a new approach to automate the design of “data recipes,” pipelines used to transform raw data into effective training corpora. The study introduces DataChef-32B, a system that employs reinforcement learning to generate a complete data recipe given a target task and available data sources, a collaboration between Fudan University and the Shanghai Institute of AI. Demonstrating its effectiveness across six tasks, DataChef-32B produces recipes that achieve performance comparable to those hand-picked by human experts. In particular, it outperforms Qwen3-1.7B on the AIME’25 benchmark by a score of 66.7, providing a path to self-evolving AI systems and automated LLM training.
The performance of these models increasingly relies on carefully curated datasets assembled using a process known as a “data recipe,” a pipeline that transforms raw information into a usable training corpus. Although progress has been made in automating individual steps within this process using LLM, designing the entire data recipe remains a manual and expertise-based task. The researchers developed DataChef-32B, a system that can generate complete data recipes considering a target benchmark and a selection of available data sources. DataChef-32B employs online reinforcement learning guided by proxy rewards that accurately predicts how well candidate recipes will perform on downstream tasks. To overcome the challenges of limited data and the cost of evaluating full model training, the team developed a “data verifier,” a method to assess the quality of training data without requiring a full model training run. This provides a quick and low-cost reward signal to the reinforcement learning process, enabling scalable and efficient recipe optimization. The study introduces a comprehensive task pool that includes 31 benchmarks and 257 related datasets across 10 domains including mathematics, coding, finance, and medicine. Each task utilized 8 to 15 source datasets to ensure diversity in the training material. Across six independent evaluation tasks, recipes generated by DataChef-32B achieve performance comparable to recipes painstakingly created by human experts. This achievement highlights the system’s ability to not only automate recipe creation but also enhance the functionality of the base LLM. A policy language model underpins the core of the methodology and generates a complete data recipe for adapting the base large-scale language model (LLM) to the target task. This approach focuses on end-to-end recipe creation, rather than simply automating individual data processing steps. The system accepts a task definition consisting of natural language instructions, available data sources, and evaluation metrics, and outputs a data pipeline formulated as a Python script. These scripts detail the exact sequence of operations to transform your raw data into a training dataset. To facilitate automated evaluation, a dedicated Data Verifier was developed to evaluate the generated training data and provide a scalar reward signal reflecting data quality and pipeline viability. Data Verifier works by sampling a subset of the generated data, evaluating it against a rubric-based evaluation scheme, and assigning a score based on criteria such as validity, formal accuracy, and task relevance. The key innovation lies in the integration of a code interpreter that runs the generated Python scripts to ensure that they work as intended and to identify potential errors. Reinforcement learning, specifically the Generalized Reward Augmented Policy Optimization (GRPO) algorithm, drives the learning process. The Policy LLM is trained online and iteratively improves its recipe generation capabilities based on the rewards it receives from Data Verifier. This allows the system to explore a vast space of possible data recipes and identify the one that yields the best downstream performance on the target task. DataChef-32B recipes incorporate several important data processing steps, including outlier filtering, chain-of-thought synthesis, data standardization, blending, and deduplication. Data validators have proven to be reliable predictors of downstream performance, providing low-cost, instant reward signals to reinforcement learning processes and accelerating data recipe optimization. The framework is designed to accommodate open-ended configurations that accept arbitrary tasks and datasets as input, thereby moving beyond evaluations limited to static datasets and carefully selected initial code. The constant pursuit of better large-scale language models (LLMs) has primarily focused on scaling up, increasing the amount of parameters and data. However, fundamental bottlenecks still remain. It’s all about assembling the right training data in the first place. For many years, this process has been painstakingly manual, relying on human expertise to curate and refine datasets. Relying on a proxy reward function introduces potential limitations. In this case, the proxy accurately predicts downstream performance, but generalizability to other domains remains an open question. Additionally, the system currently operates within a defined pool of existing data sources. Discovery and creation of truly new data is not yet functional. In the future, the convergence of these automatic recipe generation techniques and active learning strategies could unleash a virtuous cycle of improvement, accelerating advances in LLM and extending the principles to other areas of artificial intelligence where data curation is a key constraint.
