AgentMath: Empowering Mathematical Reasoning for Large Language Models via Tool-Augmented Agent

TL;DR

AgentMath tackles the challenge of mathematical reasoning with LRMs by integrating reasoning with exact computation via a code execution tool. It introduces an automated tool-augmented data synthesis pipeline, a novel agentic reinforcement learning paradigm that interleaves natural language thinking with code execution, and a scalable asynchronous training infrastructure. Empirical results on AIME24, AIME25, and HMMT25 show state-of-the-art performance across multiple model scales, with large configurations achieving high accuracies and enabling competitive performance relative to proprietary systems. The work demonstrates emergent code self-correction, data-efficient learning through tool augmentation, and a practical path toward scalable, robust mathematical reasoning agents.

Abstract

Large Reasoning Models (LRMs) like o3 and DeepSeek-R1 have achieved remarkable progress in natural language reasoning with long chain-of-thought. However, they remain computationally inefficient and struggle with accuracy when solving problems requiring complex mathematical operations. In this work, we present AgentMath, an agent framework that seamlessly integrates language models' reasoning capabilities with code interpreters' computational precision to efficiently tackle complex mathematical problems. Our approach introduces three key innovations: (1) An automated method that converts natural language chain-of-thought into structured tool-augmented trajectories, generating high-quality supervised fine-tuning (SFT) data to alleviate data scarcity; (2) A novel agentic reinforcement learning (RL) paradigm that dynamically interleaves natural language generation with real-time code execution. This enables models to autonomously learn optimal tool-use strategies through multi-round interactive feedback, while fostering emergent capabilities in code refinement and error correction; (3) An efficient training system incorporating innovative techniques, including request-level asynchronous rollout scheduling, agentic partial rollout, and prefix-aware weighted load balancing, achieving 4-5x speedup and making efficient RL training feasible on ultra-long sequences with scenarios with massive tool calls.Extensive evaluations show that AgentMath achieves state-of-the-art performance on challenging mathematical competition benchmarks including AIME24, AIME25, and HMMT25. Specifically, AgentMath-30B-A3B attains 90.6%, 86.4%, and 73.8% accuracy respectively, achieving advanced capabilities.These results validate the effectiveness of our approach and pave the way for building more sophisticated and scalable mathematical reasoning agents.

Figures (9)

• Figure 1: This diagram outlines a three-stage pipeline for creating a high-quality tool-augmented trajectories for training agents, including Agentic Trajectory Generation via Code Injection, Multi-Faceted Quality Refinement and Self-Correction Capability Injection. This automated process transforms pure-text reasoning into verified, executable agentic trajectories.
• Figure 2: The diagram of agentic reinforcement learning. It depicts the structure and workflow of our agentic reinforcement learning system with core functions including Agent Loop, Asynchronous Scheduler, and Partial Rollout, along with key performance improvement. Based on the Asynchronous Scheduler, the Agent Loop continues running by default. It will stop early only when conditions are met: either the content length exceeds the max length (i.e., 32k) or the number of tool calls exceeds the maximum constraint.
• Figure 3: Evolution of key metrics during multi-stage RL training: (a-c) accuracy on AIME24 and AIME25, response length, length clip ratio; (d-f) code ratio, tool call counts, code clip ratio.
• Figure 4: Performance Comparison of AgentMath vs. Text-Based Model in the RL phase on AIME24/25. Both models were initialized from their best SFT checkpoint trained on 20k data.
• Figure 5: The evolution of sequence lengths for AgentMath and Text-Based model during RL training and on the AIME24 and AIME25. Both models started from their best SFT checkpoints trained on 20k data.