A Reinforcement Learning-based Transmission Expansion Framework Considering Strategic Bidding in Electricity Markets

TL;DR

This paper proposes a Reinforcement Learning (RL)-based co-optimization framework that simultaneously learns transmission investment decisions and generator bidding strategies within a unified training process based on a multiagent RL framework for market simulation.

Abstract

Transmission expansion planning in electricity markets is tightly coupled with the strategic bidding behaviors of generation companies. This paper proposes a Reinforcement Learning (RL)-based co-optimization framework that simultaneously learns transmission investment decisions and generator bidding strategies within a unified training process. Based on a multiagent RL framework for market simulation, the proposed method newly introduces a design policy layer that jointly optimizes continuous/discrete transmission expansion decisions together with strategic bidding policies. Through iterative interaction between market clearing and investment design, the framework effectively captures their mutual influence and achieves consistent co-optimization of expansion and bidding decisions. Case studies on the IEEE 30-bus system are provided for proof-of-concept validation of the proposed co-optimization framework.

Figures (3)

• Figure 1: Overview of the proposed co-optimization framework
• Figure 2: Topology of IEEE 30-bus system
• Figure 3: Total cost comparison and component breakdown