AlphaQuanter: An End-to-End Tool-Orchestrated Agentic Reinforcement Learning Framework for Stock Trading

TL;DR

AlphaQuanter tackles the core limitations of LLM-based trading by introducing a single-agent, tool-augmented RL framework that end-to-end optimizes a transparent decision workflow. It frames stock trading as a tool-augmented MDP and trains the agent to actively acquire information and selectively use tools, yielding interpretable reasoning traces and robust, profitable policies. Empirical results show state-of-the-art performance in ARR and SR across multiple stocks, with the 7B variant delivering consistent improvements and rich information-seeking behavior. The work highlights the value of optimizing the entire decision process (not just final predictions) for robust, auditable, and human-trader-friendly automated trading systems.

Abstract

While Large Language Model (LLM) agents show promise in automated trading, they still face critical limitations. Prominent multi-agent frameworks often suffer from inefficiency, produce inconsistent signals, and lack the end-to-end optimization required to learn a coherent strategy from market feedback. To address this, we introduce AlphaQuanter, a single-agent framework that uses reinforcement learning (RL) to learn a dynamic policy over a transparent, tool-augmented decision workflow, which empowers a single agent to autonomously orchestrate tools and proactively acquire information on demand, establishing a transparent and auditable reasoning process. Extensive experiments demonstrate that AlphaQuanter achieves state-of-the-art performance on key financial metrics. Moreover, its interpretable reasoning reveals sophisticated strategies, offering novel and valuable insights for human traders. Our code for data acquisition and agent training is publicly available at: https://github.com/AlphaQuanter/AlphaQuanter

Figures (8)

• Figure 1: The overall architecture and workflow of AlphaQuanter. The central panel shows the agent's iterative rollout process. Starting from an initial state ($S_0$), the agent first forms an initial plan ($P_0$) before generating further reasoning traces ($R_{i+1}$) with <think> tag. In each step, it decides whether to continue acquiring information by executing a tool-based action ($A_{i+1}$) and receiving its environmental feedback ($E_{i+1}$), or to conclude by outputting a final decision with an <answer> tag. Throughout this process, the agent can query multi-dimensional financial data sources (bottom panel, Section \ref{['sec:info_sources']}). Once a decision is made, the entire trajectory will be evaluated to compute a reward (top-right panel, Section \ref{['sec:score']}), which updates the agent's policy. The overall workflow (top-left panel, Section \ref{['sec:cognitive_workflow']}) is designed to mimic a human trader's cognitive process of reasoning and acquiring data on demand.
• Figure 2: Comparison of training dynamics for the AlphaQuanter-3B and -7B models.
• Figure 3: Comparison of key backtesting metrics for the AlphaQuanter-3B and -7B models on the validation set.
• Figure 4: Evolution of the tool selection strategies for the AlphaQuanter-3B and -7B models during training. The heatmap color intensity shows the percentile-based reliance on each information at different training steps. The symbols [M], [S], [X], and [F] represent the four categories of data sources, respectively.
• Figure 5: The effect of different decision threshold ($\theta$) values on the agent's action distribution during training.