Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimizing Electric Vehicle Charging Station Placement Using Reinforcement Learning and Agent-Based Simulations

Minh-Duc Nguyen, Dung D. Le, Phi Long Nguyen

TL;DR

The paper tackles EV charging-station siting under dynamic urban conditions by coupling deep reinforcement learning (DRL) with a GAM-based agent simulation. It introduces a hybrid framework with dual Deep Q-Networks for location and port configuration and a hybrid reward that blends deterministic cues with simulation-derived feedback, enabling adaptive decision-making. In Hanoi, Vietnam, the approach achieves a 53.28% reduction in average waiting time compared with the initial state and outperforms static baselines, demonstrating robustness to real-world variability and scalability to future demand. The work provides a practical, data-driven pathway for scalable EV infrastructure planning, highlighting the value of simulation-informed rewards and Voronoi-based candidate locations to balance coverage, power constraints, and user experience.

Abstract

The rapid growth of electric vehicles (EVs) necessitates the strategic placement of charging stations to optimize resource utilization and minimize user inconvenience. Reinforcement learning (RL) offers an innovative approach to identifying optimal charging station locations; however, existing methods face challenges due to their deterministic reward systems, which limit efficiency. Because real-world conditions are dynamic and uncertain, a deterministic reward structure cannot fully capture the complexities of charging station placement. As a result, evaluation becomes costly and time-consuming, and less reflective of real-world scenarios. To address this challenge, we propose a novel framework that integrates deep RL with agent-based simulations to model EV movement and estimate charging demand in real time. Our approach employs a hybrid RL agent with dual Q-networks to select optimal locations and configure charging ports, guided by a hybrid reward function that combines deterministic factors with simulation-derived feedback. Case studies in Hanoi, Vietnam, show that our method reduces average waiting times by 53.28% compared to the initial state, outperforming static baseline methods. This scalable and adaptive solution enhances EV infrastructure planning, effectively addressing real-world complexities and improving user experience.

Optimizing Electric Vehicle Charging Station Placement Using Reinforcement Learning and Agent-Based Simulations

TL;DR

The paper tackles EV charging-station siting under dynamic urban conditions by coupling deep reinforcement learning (DRL) with a GAM-based agent simulation. It introduces a hybrid framework with dual Deep Q-Networks for location and port configuration and a hybrid reward that blends deterministic cues with simulation-derived feedback, enabling adaptive decision-making. In Hanoi, Vietnam, the approach achieves a 53.28% reduction in average waiting time compared with the initial state and outperforms static baselines, demonstrating robustness to real-world variability and scalability to future demand. The work provides a practical, data-driven pathway for scalable EV infrastructure planning, highlighting the value of simulation-informed rewards and Voronoi-based candidate locations to balance coverage, power constraints, and user experience.

Abstract

The rapid growth of electric vehicles (EVs) necessitates the strategic placement of charging stations to optimize resource utilization and minimize user inconvenience. Reinforcement learning (RL) offers an innovative approach to identifying optimal charging station locations; however, existing methods face challenges due to their deterministic reward systems, which limit efficiency. Because real-world conditions are dynamic and uncertain, a deterministic reward structure cannot fully capture the complexities of charging station placement. As a result, evaluation becomes costly and time-consuming, and less reflective of real-world scenarios. To address this challenge, we propose a novel framework that integrates deep RL with agent-based simulations to model EV movement and estimate charging demand in real time. Our approach employs a hybrid RL agent with dual Q-networks to select optimal locations and configure charging ports, guided by a hybrid reward function that combines deterministic factors with simulation-derived feedback. Case studies in Hanoi, Vietnam, show that our method reduces average waiting times by 53.28% compared to the initial state, outperforming static baseline methods. This scalable and adaptive solution enhances EV infrastructure planning, effectively addressing real-world complexities and improving user experience.

Paper Structure

This paper contains 31 sections, 15 equations, 11 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Traditional Optimization Techniques.
  4. Heuristic and Metaheuristic Methods
  5. Machine Learning and Data-Driven Approaches
  6. Deep Reinforcement Learning with Agent-based Simulation
  7. Charging Station Location Problem
  8. Reinforcement Learning Framework for Charging Station Deployment
  9. Deep Reinforcement Learning Model
  10. State of the Environment
  11. Action
  12. Reward
  13. Learning for Charging Station Placement Agent
  14. Simulation-based Gama model
  15. Experiments
  16. ...and 16 more sections

Figures (11)

  • Figure 1: Workflow of Charging Station Placement Agent with Environment and Simulation Feedback
  • Figure 2: Voronoi region contains the vertices to make the candidate location for the charging station.
  • Figure 3: Simulate the process of moving and searching for a station to charge the electric vehicle in GAMA.
  • Figure 4: Flowchart of Charging Behavior Based on Battery Level and Station Availability
  • Figure 5: Simulation results
  • ...and 6 more figures