AceGRPO: Adaptive Curriculum Enhanced Group Relative Policy Optimization for Autonomous Machine Learning Engineering

TL;DR

AceGRPO addresses the challenge of long-horizon autonomous Machine Learning Engineering by enabling an LLM to learn from its own trial-and-error through an evolving data buffer and a curriculum-guided adaptive sampling strategy. It reframes optimization as step-wise learning over a dynamically expanding task distribution and prioritizes informative states with Learnability Potential to maximize gradient signals under limited compute. The approach yields a 100% valid submission rate on MLE-Bench-Lite for Ace-30B, with medal and HumanRank performance approaching or surpassing larger frontier models, while maintaining strong open-source efficiency. This demonstrates sustained self-evolution and practical potential for deploying autonomous ML agents in iterative engineering tasks.

Abstract

Autonomous Machine Learning Engineering (MLE) requires agents to perform sustained, iterative optimization over long horizons. While recent LLM-based agents show promise, current prompt-based agents for MLE suffer from behavioral stagnation due to frozen parameters. Although Reinforcement Learning (RL) offers a remedy, applying it to MLE is hindered by prohibitive execution latency and inefficient data selection. Recognizing these challenges, we propose AceGRPO with two core components: (1) Evolving Data Buffer that continuously repurposes execution traces into reusable training tasks, and (2) Adaptive Sampling guided by a Learnability Potential function, which dynamically prioritizes tasks at the agent's learning frontier to maximize learning efficiency. Leveraging AceGRPO, our trained Ace-30B model achieves a 100% valid submission rate on MLE-Bench-Lite, approaches the performance of proprietary frontier models, and outperforms larger open-source baselines (e.g., DeepSeek-V3.2), demonstrating robust capability for sustained iterative optimization. Code is available at https://github.com/yuzhu-cai/AceGRPO.

Figures (5)

• Figure 1: The Agent Optimization Loop for MLE. It is a continuous iteration over three distinct phases: Draft (initial generation), Debug (error correction), and Improve (metric optimization). The transitions are deterministically governed by the execution feedback, allowing the agent to refine its code solution and accumulate context through repeated interactions.
• Figure 2: The AceGRPO framework. MLE is step-wise optimization over a dynamic task distribution. The method features an Evolving Data Buffer that accumulates intermediate states (Draft, Debug, Improve) via Streaming Expansion. An Adaptive Sampler selects high-potential tasks to maximize the gradient signal for GRPO training, enabling efficient self-improvement on long-horizon challenges.
• Figure 3: Performance comparison on first valid submissions across different models. We report three key metrics: (left) Medal Rate measuring the percentage of tasks achieving medals on first submission, (middle) Average HumanRank score indicating the mean HumanRank score obtained on first valid submission, and (right) Average Step showing the mean number of iterations required to produce the first valid submission. Lower steps indicate faster generation of valid solutions. The bars show performance for each model's first valid submission, with lower steps indicating faster solution generation
• Figure 4: Medal rate evolution over time. Ace-30B shows significant improvement over the base model Qwen3-30B-A3B-Thinking-2507, approaching the performance of larger closed-source model GPT-5.2.
• Figure 5: Training dynamics of rewards curve and performance of valid submission