Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Alpha Weighting with PPO: Enhancing Prompt-Based LLM-Generated Alphas in Quant Trading

Qizhao Chen, Hiroaki Kawashima

TL;DR

A reinforcement learning framework that employs Proximal Policy Optimization (PPO) to dynamically optimize the weights of multiple large language model (LLM)-generated formulaic alphas for stock trading strategies and shows the potential of combining LLM-generated signals with adaptive optimization for robust financial forecasting and trading.

Abstract

This paper introduces a reinforcement learning framework that employs Proximal Policy Optimization (PPO) to dynamically optimize the weights of multiple large language model (LLM)-generated formulaic alphas for stock trading strategies. Formulaic alphas are mathematically defined trading signals derived from price, volume, sentiment, and other data. Although recent studies have shown that LLMs can generate diverse and effective alphas, a critical challenge lies in how to adaptively integrate them under varying market conditions. To address this gap, we leverage a DeepSeek model to generate fifty alphas for ten stocks, and then use PPO to adjust their weights in real time. Experimental results indicate that the PPO-optimized strategy does not consistently deliver the highest cumulative returns across all stocks, but it achieves comparatively higher Sharpe ratios and smaller maximum drawdowns in most cases. When compared with baseline strategies, including equal-weighted, buy-and-hold, random entry/exit, and momentum approaches, PPO demonstrates more stable risk-adjusted performance. The findings highlight the importance of reinforcement learning in the allocation of alpha weights and show the potential of combining LLM-generated signals with adaptive optimization for robust financial forecasting and trading.

Adaptive Alpha Weighting with PPO: Enhancing Prompt-Based LLM-Generated Alphas in Quant Trading

TL;DR

A reinforcement learning framework that employs Proximal Policy Optimization (PPO) to dynamically optimize the weights of multiple large language model (LLM)-generated formulaic alphas for stock trading strategies and shows the potential of combining LLM-generated signals with adaptive optimization for robust financial forecasting and trading.

Abstract

This paper introduces a reinforcement learning framework that employs Proximal Policy Optimization (PPO) to dynamically optimize the weights of multiple large language model (LLM)-generated formulaic alphas for stock trading strategies. Formulaic alphas are mathematically defined trading signals derived from price, volume, sentiment, and other data. Although recent studies have shown that LLMs can generate diverse and effective alphas, a critical challenge lies in how to adaptively integrate them under varying market conditions. To address this gap, we leverage a DeepSeek model to generate fifty alphas for ten stocks, and then use PPO to adjust their weights in real time. Experimental results indicate that the PPO-optimized strategy does not consistently deliver the highest cumulative returns across all stocks, but it achieves comparatively higher Sharpe ratios and smaller maximum drawdowns in most cases. When compared with baseline strategies, including equal-weighted, buy-and-hold, random entry/exit, and momentum approaches, PPO demonstrates more stable risk-adjusted performance. The findings highlight the importance of reinforcement learning in the allocation of alpha weights and show the potential of combining LLM-generated signals with adaptive optimization for robust financial forecasting and trading.

Paper Structure

This paper contains 32 sections, 21 equations, 9 figures, 17 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. Data
  5. Model Setup
  6. Prompt-Based LLM for Alpha Generation
  7. PPO for Alpha Weight Optimization
  8. Implementation Details
  9. Evaluation
  10. Baselines
  11. Buy and Hold (B&H) Strategy
  12. Random Entry/Exit Strategy
  13. Momentum (MOM) Strategy
  14. Evaluation Metrics
  15. Statistical Comparison via Diebold-Mariano Test
  16. ...and 17 more sections

Figures (9)

  • Figure 1: Flow Chart of the Proposed Method
  • Figure 2: Whole Picture of the Proposed Method
  • Figure 3: Mutual Information (in Blue) and Information Coefficient (in Red) of LLM-generated Alphas
  • Figure 4: Feature Importance of LLM-generated Alphas
  • Figure 5: Performance Analysis and Execution Dynamics of the PPO Strategy for Pepsi Stock. The panels illustrate the cumulative portfolio value (top), the volatility-adjusted position exposure $p_t$ (middle), and the composite alpha signal strength relative to decision thresholds (bottom).
  • ...and 4 more figures