Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A New DAPO Algorithm for Stock Trading

Ruijian Zha, Bojun Liu

TL;DR

This paper addresses data-efficient, risk-aware stock trading with reinforcement learning by integrating a memory-efficient Group Relative Policy Optimization (GRPO) framework with Dynamic Sampling Policy Optimization (DAPO) ideas and large-language-model–derived sentiment and risk signals. It introduces an exponent-weighted sentiment–risk reward and a GRPO-inspired DAPO loss with decoupled clipping and dynamic sampling to navigate exploration–stability trade-offs, achieving strong NASDAQ-100 performance: a cumulative return of $230.49\%$ and an information ratio of $0.37$ over 100 training epochs, with substantial reductions in training time (≈$2.5$ hours) and RAM usage compared with CPPO-DeepSeek. The approach leverages pre-extracted LLM signals from FinRL-DeepSeek on the FinRL data corpus and demonstrates robustness across different $\alpha$/$\beta$ settings, including a sentiment-emphasis configuration that yields high profits. These results suggest a scalable path toward data-efficient, interpretable RL agents for financial trading that can operate with lower computational overhead while maintaining strong performance.

Abstract

Recent advances in reinforcement learning, such as Dynamic Sampling Policy Optimization (DAPO), show strong performance when paired with large language models (LLMs). Motivated by this success, we ask whether similar gains can be realized in financial trading. We design a trading agent that combines an improved Group Relative Policy Optimization (GRPO) algorithm, augmented with ideas from DAPO, with LLM-based risk and sentiment signals extracted from financial news. On the NASDAQ-100 index (FNSPID dataset), our agent attains a cumulative return of 230.49 percent and an information ratio of 0.37, outperforming the CPPO-DeepSeek baseline. It also cuts training time from about 8 hours to 2.5 hours over 100 epochs while markedly reducing RAM usage. The proposed RL-LLM framework offers a scalable path toward data-efficient trading agents. Code: https://github.com/Ruijian-Zha/FinRL-DAPO-SR/

A New DAPO Algorithm for Stock Trading

TL;DR

This paper addresses data-efficient, risk-aware stock trading with reinforcement learning by integrating a memory-efficient Group Relative Policy Optimization (GRPO) framework with Dynamic Sampling Policy Optimization (DAPO) ideas and large-language-model–derived sentiment and risk signals. It introduces an exponent-weighted sentiment–risk reward and a GRPO-inspired DAPO loss with decoupled clipping and dynamic sampling to navigate exploration–stability trade-offs, achieving strong NASDAQ-100 performance: a cumulative return of and an information ratio of over 100 training epochs, with substantial reductions in training time (≈ hours) and RAM usage compared with CPPO-DeepSeek. The approach leverages pre-extracted LLM signals from FinRL-DeepSeek on the FinRL data corpus and demonstrates robustness across different / settings, including a sentiment-emphasis configuration that yields high profits. These results suggest a scalable path toward data-efficient, interpretable RL agents for financial trading that can operate with lower computational overhead while maintaining strong performance.

Abstract

Recent advances in reinforcement learning, such as Dynamic Sampling Policy Optimization (DAPO), show strong performance when paired with large language models (LLMs). Motivated by this success, we ask whether similar gains can be realized in financial trading. We design a trading agent that combines an improved Group Relative Policy Optimization (GRPO) algorithm, augmented with ideas from DAPO, with LLM-based risk and sentiment signals extracted from financial news. On the NASDAQ-100 index (FNSPID dataset), our agent attains a cumulative return of 230.49 percent and an information ratio of 0.37, outperforming the CPPO-DeepSeek baseline. It also cuts training time from about 8 hours to 2.5 hours over 100 epochs while markedly reducing RAM usage. The proposed RL-LLM framework offers a scalable path toward data-efficient trading agents. Code: https://github.com/Ruijian-Zha/FinRL-DAPO-SR/
Paper Structure (14 sections, 6 equations, 2 figures, 3 tables)

This paper contains 14 sections, 6 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Reinforcement Learning for Trading
  4. GRPO and DAPO
  5. Methodology
  6. Stock Trading Environment
  7. GRPO with Exponentiated Sentiment-Risk Reward
  8. GRPO-Inspired DAPO
  9. Preliminary Results and Discussion
  10. Dataset
  11. Impacts of Risk and Sentiment Signals
  12. Effects of Varying $\alpha$ and $\beta$
  13. Improved Computation Efficiency
  14. Conclusion and Future Work

Figures (2)

  • Figure 1: Comparison of the Baseline, Risk‐Only, and Sentiment‐Only reward settings vs. the NASDAQ‐100, showing 2020–2023 cumulative returns after 100 training epochs on six years of data.
  • Figure 2: Comparison of four sentiment–risk weightings vs. the NASDAQ‐100, showing 2020–2023 cumulative returns after 100 training epochs on six years of data.