Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Plug-in Hybrid Electric Vehicle Energy Management with Clutch Engagement Control via Continuous-Discrete Reinforcement Learning

Changfu Gong, Jinming Xu, Yuan Lin

TL;DR

The proposed CDRL strategy can improve energy efficiency by 8.3% compared to CD-CS, and the energy consumption is just 6.6% higher than the global optimum based on DP, while under a low SOC, the numbers are 4.1% and 3.9%, respectively.

Abstract

Energy management strategy (EMS) is a key technology for plug-in hybrid electric vehicles (PHEVs). The energy management of certain series-parallel PHEVs involves the control of continuous variables, such as engine torque, and discrete variables, such as clutch engagement/disengagement. We establish a control-oriented model for a series-parallel plug-in hybrid system with clutch engagement control from the perspective of mixed-integer programming. Subsequently, we design an EMS based on continuous-discrete reinforcement learning (CDRL), which enables simultaneous output of continuous and discrete variables. During training, we introduce state-of-charge (SOC) randomization to ensure that the hybrid system exhibits optimal energy-saving performance in both high and low SOC. Finally, the effectiveness of the proposed CDRL strategy is verified by comparing EMS based on charge-depleting charge-sustaining (CD-CS) with rule-based clutch engagement control, and Dynamic Programming (DP). The simulation results show that, under a high SOC, the CDRL strategy proposed in this paper can improve energy efficiency by 8.3% compared to CD-CS, and the energy consumption is just 6.6% higher than the global optimum based on DP, while under a low SOC, the numbers are 4.1% and 3.9%, respectively.

Plug-in Hybrid Electric Vehicle Energy Management with Clutch Engagement Control via Continuous-Discrete Reinforcement Learning

TL;DR

The proposed CDRL strategy can improve energy efficiency by 8.3% compared to CD-CS, and the energy consumption is just 6.6% higher than the global optimum based on DP, while under a low SOC, the numbers are 4.1% and 3.9%, respectively.

Abstract

Energy management strategy (EMS) is a key technology for plug-in hybrid electric vehicles (PHEVs). The energy management of certain series-parallel PHEVs involves the control of continuous variables, such as engine torque, and discrete variables, such as clutch engagement/disengagement. We establish a control-oriented model for a series-parallel plug-in hybrid system with clutch engagement control from the perspective of mixed-integer programming. Subsequently, we design an EMS based on continuous-discrete reinforcement learning (CDRL), which enables simultaneous output of continuous and discrete variables. During training, we introduce state-of-charge (SOC) randomization to ensure that the hybrid system exhibits optimal energy-saving performance in both high and low SOC. Finally, the effectiveness of the proposed CDRL strategy is verified by comparing EMS based on charge-depleting charge-sustaining (CD-CS) with rule-based clutch engagement control, and Dynamic Programming (DP). The simulation results show that, under a high SOC, the CDRL strategy proposed in this paper can improve energy efficiency by 8.3% compared to CD-CS, and the energy consumption is just 6.6% higher than the global optimum based on DP, while under a low SOC, the numbers are 4.1% and 3.9%, respectively.
Paper Structure (24 sections, 36 equations, 9 figures, 6 tables, 1 algorithm)

This paper contains 24 sections, 36 equations, 9 figures, 6 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Literature review
  3. Motivation and innovation
  4. Organization
  5. DM-i Hybrid System Modeling
  6. Vehicle dynamics
  7. Engine model
  8. Drive motor and generator models
  9. Battery model
  10. PDQN-TD3 Continuous-Discrete Reinforcement Learning Algorithm
  11. Principles of reinforcement learning
  12. PDQN algorithm
  13. PDQN-TD3 algorithm
  14. Target policy smoothing
  15. Clipped double Q-learning
  16. ...and 9 more sections

Figures (9)

  • Figure 1: Configuration of the BYD DM-i PHEV.
  • Figure 2: (a) Engine fuel consumption map, wherein the unit of BSFC is g/kW·h. (b) Efficiency map of the driving motor. (c) Open-circuit voltage and internal resistance of battery.
  • Figure 3: Illustration of the PDQN architecture.
  • Figure 4: Energy management framework based on PDQN-TD3 algorithm.
  • Figure 5: Velocity profile of the driving cycle.
  • ...and 4 more figures