Constrained Optimal Fuel Consumption of HEV: A Constrained Reinforcement Learning Approach

TL;DR

This work provides the mathematical expression of the constrained optimal fuel consumption (COFC) problem from the perspective of constrained reinforcement learning (CRL) for the first time globally and verifies the effectiveness of proposed CRL approaches to the COFC problem.

Abstract

Hybrid electric vehicles (HEVs) are becoming increasingly popular because they can better combine the working characteristics of internal combustion engines and electric motors. However, the minimum fuel consumption of an HEV for a battery electrical balance case under a specific assembly condition and a specific speed curve still needs to be clarified in academia and industry. Regarding this problem, this work provides the mathematical expression of constrained optimal fuel consumption (COFC) from the perspective of constrained reinforcement learning (CRL) for the first time globally. Also, two mainstream approaches of CRL, constrained variational policy optimization (CVPO) and Lagrangian-based approaches, are utilized for the first time to obtain the vehicle's minimum fuel consumption under the battery electrical balance condition. We conduct case studies on the well-known Prius TOYOTA hybrid system (THS) under the NEDC condition; we give vital steps to implement CRL approaches and compare the performance between the CVPO and Lagrangian-based approaches. Our case study found that CVPO and Lagrangian-based approaches can obtain the lowest fuel consumption while maintaining the SOC balance constraint. The CVPO approach converges stable, but the Lagrangian-based approach can obtain the lowest fuel consumption at 3.95 L/100km, though with more significant oscillations. This result verifies the effectiveness of our proposed CRL approaches to the COFC problem.

Figures (9)

• Figure 1: Battery SOC-allowed range (filled with red dotted lines).
• Figure 2: The perspective of Lagrangian-based approaches.
• Figure 3: The perspective of the CVPO approach.
• Figure 4: The THS structure.
• Figure 5: Add 'YourENV' in environment registration.