Youtu-Agent: Scaling Agent Productivity with Automated Generation and Hybrid Policy Optimization

TL;DR

Youtu-Agent tackles the dual challenges of high configuration costs and static capabilities in LLM-based agents by delivering a modular, YAML-based framework that decouples Environment, Toolkits, and Agent. It introduces two automated generation paradigms, Workflow and Meta-Agent, to synthesize tools and configurations, and couples this with a hybrid policy optimization stack: a Training-free Agent Practice module for low-cost, gradient-free improvement and an end-to-end Agent RL pipeline for scalable parameter optimization. Empirical results show strong performance with open-weight models on WebWalkerQA and GAIA, high tool-synthesis success, and meaningful improvements on AIME benchmarks, alongside 40% RL training speedups and scalable 128-GPU RL training. Collectively, the framework enables automated agent construction, continuous self-improvement, and scalable learning, lowering the barrier to deploying evolvable, robust LLM agents in real-world tasks.

Abstract

Existing Large Language Model (LLM) agent frameworks face two significant challenges: high configuration costs and static capabilities. Building a high-quality agent often requires extensive manual effort in tool integration and prompt engineering, while deployed agents struggle to adapt to dynamic environments without expensive fine-tuning. To address these issues, we propose \textbf{Youtu-Agent}, a modular framework designed for the automated generation and continuous evolution of LLM agents. Youtu-Agent features a structured configuration system that decouples execution environments, toolkits, and context management, enabling flexible reuse and automated synthesis. We introduce two generation paradigms: a \textbf{Workflow} mode for standard tasks and a \textbf{Meta-Agent} mode for complex, non-standard requirements, capable of automatically generating tool code, prompts, and configurations. Furthermore, Youtu-Agent establishes a hybrid policy optimization system: (1) an \textbf{Agent Practice} module that enables agents to accumulate experience and improve performance through in-context optimization without parameter updates; and (2) an \textbf{Agent RL} module that integrates with distributed training frameworks to enable scalable and stable reinforcement learning of any Youtu-Agents in an end-to-end, large-scale manner. Experiments demonstrate that Youtu-Agent achieves state-of-the-art performance on WebWalkerQA (71.47\%) and GAIA (72.8\%) using open-weight models. Our automated generation pipeline achieves over 81\% tool synthesis success rate, while the Practice module improves performance on AIME 2024/2025 by +2.7\% and +5.4\% respectively. Moreover, our Agent RL training achieves 40\% speedup with steady performance improvement on 7B LLMs, enhancing coding/reasoning and searching capabilities respectively up to 35\% and 21\% on Maths and general/multi-hop QA benchmarks.

Figures (8)

• Figure 1: Automated generation mechanism. Left: user input describing the desired agent. Middle: two generation paradigms---Workflow mode (top) follows a deterministic four-stage pipeline, while Meta-Agent mode (bottom) deploys an architect agent with flexible tool access. Right: the generated agent configuration ready for deployment.
• Figure 2: Training-free GRPO mechanism. Given only a few dozen of samples, the agent performs multiple rollouts per task. An LLM evaluator assesses group relative trajectory quality, and distilling contextual memory of experiential knowledge by comparing successful and failed trials. During online testing, such learned experiences are injected as "textual LoRA" to guide reasoning.
• Figure 3: An end-to-end RL training pipeline with Youtu-Agent. Left: Data flow in the RL training framework sheng2025hybridflow. Middle: Connector between RL framework and Agent framework microsoft2025agentlightning. Right: Data flow in the Youtu-Agent inference framework.
• Figure 4: Performance comparison on WebWalkerQA, including both training-free and trained agent approaches.
• Figure 5: During the learning stage of Training-free GRPO, performance improves steadily while tool usage becomes more efficient.