Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Feature-driven reinforcement learning for photovoltaic in continuous intraday trading

Arega Getaneh Abate, Xiufeng Liu, Ruyu Liu, Xiaobing Zhang

TL;DR

This paper addresses PV intraday trading under generation and price uncertainty by introducing a feature-driven reinforcement learning framework trained with Proximal Policy Optimization. It combines a predominantly linear policy with a risk-aware, feature-augmented RL formulation to learn sequential bidding decisions in continuous intraday markets, validated on Danish market data. The approach demonstrates profit uplift over baselines while controlling tail risk, achieves rapid convergence, and maintains sub-millisecond inference latency for real-time deployment, with interpretability through the linear policy weights. The work offers a practical pathway to active intraday participation for PV producers and highlights the dominant role of market microstructure and forecast features in decision making, while outlining avenues for portfolio-level and multi-agent extensions.

Abstract

Photovoltaic (PV) operators face substantial uncertainty in generation and short-term electricity prices. Continuous intraday markets enable producers to adjust their positions in real time, potentially improving revenues and reducing imbalance costs. We propose a feature-driven reinforcement learning (RL) approach for PV intraday trading that integrates data-driven features into the state and learns bidding policies in a sequential decision framework. The problem is cast as a Markov Decision Process with a reward that balances trading profit and imbalance penalties and is solved with Proximal Policy Optimization (PPO) using a predominantly linear, interpretable policy. Trained on historical market data and evaluated out-of-sample, the strategy consistently outperforms benchmark baselines across diverse scenarios. Extensive validation shows rapid convergence, real-time inference, and transparent decision rules. Learned weights highlight the central role of market microstructure and historical features. Taken together, these results indicate that feature-driven RL offers a practical, data-efficient, and operationally deployable pathway for active intraday participation by PV producers.

Feature-driven reinforcement learning for photovoltaic in continuous intraday trading

TL;DR

This paper addresses PV intraday trading under generation and price uncertainty by introducing a feature-driven reinforcement learning framework trained with Proximal Policy Optimization. It combines a predominantly linear policy with a risk-aware, feature-augmented RL formulation to learn sequential bidding decisions in continuous intraday markets, validated on Danish market data. The approach demonstrates profit uplift over baselines while controlling tail risk, achieves rapid convergence, and maintains sub-millisecond inference latency for real-time deployment, with interpretability through the linear policy weights. The work offers a practical pathway to active intraday participation for PV producers and highlights the dominant role of market microstructure and forecast features in decision making, while outlining avenues for portfolio-level and multi-agent extensions.

Abstract

Photovoltaic (PV) operators face substantial uncertainty in generation and short-term electricity prices. Continuous intraday markets enable producers to adjust their positions in real time, potentially improving revenues and reducing imbalance costs. We propose a feature-driven reinforcement learning (RL) approach for PV intraday trading that integrates data-driven features into the state and learns bidding policies in a sequential decision framework. The problem is cast as a Markov Decision Process with a reward that balances trading profit and imbalance penalties and is solved with Proximal Policy Optimization (PPO) using a predominantly linear, interpretable policy. Trained on historical market data and evaluated out-of-sample, the strategy consistently outperforms benchmark baselines across diverse scenarios. Extensive validation shows rapid convergence, real-time inference, and transparent decision rules. Learned weights highlight the central role of market microstructure and historical features. Taken together, these results indicate that feature-driven RL offers a practical, data-efficient, and operationally deployable pathway for active intraday participation by PV producers.
Paper Structure (15 sections, 14 equations, 6 figures, 4 tables, 2 algorithms)

This paper contains 15 sections, 14 equations, 6 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Model Formulation
  3. Market Setting
  4. Feature-driven decision model
  5. RL approach for policy training
  6. Feature-augmented RL for continuous intraday trading
  7. Numerical experiments
  8. Data and market context
  9. Experimental design and simulation results
  10. Model training and benchmarking
  11. Performance across stress scenarios
  12. Trading behavior and learning dynamics
  13. Computational efficiency
  14. Policy interpretability
  15. Conclusion and future work

Figures (6)

  • Figure 1: Left: order book at $T=1$. Right: clearing a new buy order added at $T=2$ with limit 60 (equal to the best ask).
  • Figure 2: Performance comparison of FDRL against baseline strategies across key metrics. FDRL achieves comparable risk-adjusted returns (Sharpe ratio) to spot-only while generating positive uplift through selective trading (41 trades). The oracle demonstrates a remaining performance gap that can be exploited with perfect information.
  • Figure 3: Profit uplift relative to spot-only, mean trade counts, and 5% CVaR across the evaluated scenarios. Error bars denote 95% confidence intervals.
  • Figure 4: Feature importance analysis. Top learned policy weights by absolute magnitude, color-coded by feature type. Market microstructure (blue, green) and forecast signals (orange) dominate the learned trading rules.
  • Figure 5: Trading pattern analysis from 2024 out-of-sample evaluation. (a) Action space utilization indicates a strong preference for conservative limit orders over aggressive market orders. (b) Trade size distribution reveals concentrated small-to-medium volume trades with a long tail.
  • ...and 1 more figures