Towards Execution-Grounded Automated AI Research

TL;DR

This work investigates execution-grounded automated AI research by building a large-scale automated idea executor and using its execution results as feedback to train ideation models. It evaluates two GPU-intensive LLM research problems—pre-training a 124M Transformer (nanoGPT) and post-training math reasoning (GRPO)—and shows that a large fraction of frontier-LLM ideas can be implemented automatically. Execution-guided evolutionary search yields significant performance gains within ten epochs, outperforming baselines and rivaling human solutions on several tasks, while RL from execution reward improves average ideator quality but struggles with maintaining diversity and achieving upper-bound breakthroughs. The study reveals both the potential and the limitations of execution-grounded automated AI research, highlighting the need for better exploration, more capable execution agents, and richer feedback signals to realize scalable scientific discovery.

Abstract

Automated AI research holds great potential to accelerate scientific discovery. However, current LLMs often generate plausible-looking but ineffective ideas. Execution grounding may help, but it is unclear whether automated execution is feasible and whether LLMs can learn from the execution feedback. To investigate these, we first build an automated executor to implement ideas and launch large-scale parallel GPU experiments to verify their effectiveness. We then convert two realistic research problems - LLM pre-training and post-training - into execution environments and demonstrate that our automated executor can implement a large fraction of the ideas sampled from frontier LLMs. We analyze two methods to learn from the execution feedback: evolutionary search and reinforcement learning. Execution-guided evolutionary search is sample-efficient: it finds a method that significantly outperforms the GRPO baseline (69.4% vs 48.0%) on post-training, and finds a pre-training recipe that outperforms the nanoGPT baseline (19.7 minutes vs 35.9 minutes) on pre-training, all within just ten search epochs. Frontier LLMs often generate meaningful algorithmic ideas during search, but they tend to saturate early and only occasionally exhibit scaling trends. Reinforcement learning from execution reward, on the other hand, suffers from mode collapse. It successfully improves the average reward of the ideator model but not the upper-bound, due to models converging on simple ideas. We thoroughly analyze the executed ideas and training dynamics to facilitate future efforts towards execution-grounded automated AI research.

Figures (10)

• Figure 1: We build an automated idea executor involving Implementer, Scheduler, and Worker. We then use this automated executor as a reward function to teach LLMs to generate more effective ideas through evolutionary search and RL. We only update the ideator in the learning process.
• Figure 2: Model performance comparison with self-execution (top row) vs GPT-5 execution (bottom row) on the GRPO and nanoGPT environments. The baseline accuracy for GRPO is 0.480, and the baseline loss for nanoGPT is 3.255. The completion rate is high for most models, especially under self-execution.
• Figure 3: Best performance at each epoch when performing execution-guided search with different models. For the nanoGPT environment (left), we use the reciprocal of the validation loss as the metric; for the GRPO environment (right), we use validation accuracy as the metric. Claude-4.5-Opus exhibits a scaling trend on both environments and achieves the best performance on nanoGPT. Claude-4.5-Sonnet achieves the best performance on GRPO due to effective hyper-parameter tuning, but saturates early.
• Figure 4: Comparison between best-of-N (blue) and our execution-guided search (red) under the same sampling budget where we sample N=80 ideas at each epoch.
• Figure 5: Training curves of RL from execution reward. We plot the average reward per epoch in the upper row, and the max reward per epoch in the lower row. For the GRPO environment, the reward is the accuracy; for the nanoGPT environment, the reward is the reciprocal of the loss. The average reward increases, but not the max reward.