Deep Reinforcement Learning-Based Optimization of Second-Life Battery Utilization in Electric Vehicles Charging Stations
Rouzbeh Haghighi, Ali Hassan, Van-Hai Bui, Akhtar Hussain, Wencong Su
TL;DR
The paper tackles uncertainty in the operation of EV charging stations equipped with second-life batteries as BESS. It introduces a Soft Actor-Critic (SAC) DRL framework trained on a full year of data with a tailored reward function to handle price and arrival-time uncertainty. The model integrates SOC dynamics, degradation costs, and planning of SLB configurations to reduce capital costs while maintaining service quality. Results show that SLBs can reduce total costs compared with fresh BESS, and SAC demonstrates superior performance over alternatives, highlighting practical potential for grid-friendly, cost-effective EVCS deployment.
Abstract
The rapid rise in electric vehicle (EV) adoption presents significant challenges in managing the vast number of retired EV batteries. Research indicates that second-life batteries (SLBs) from EVs typically retain considerable residual capacity, offering extended utility. These batteries can be effectively repurposed for use in EV charging stations (EVCS), providing a cost-effective alternative to new batteries and reducing overall planning costs. Integrating battery energy storage systems (BESS) with SLBs into EVCS is a promising strategy to alleviate system overload. However, efficient operation of EVCS with integrated BESS is hindered by uncertainties such as fluctuating EV arrival and departure times and variable power prices from the grid. This paper presents a deep reinforcement learning-based (DRL) planning framework for EV charging stations with BESS, leveraging SLBs. We employ the advanced soft actor-critic (SAC) approach, training the model on a year's worth of data to account for seasonal variations, including weekdays and holidays. A tailored reward function enables effective offline training, allowing real-time optimization of EVCS operations under uncertainty.