Constrained Optimal Fuel Consumption of HEVs under Observational Noise

TL;DR

This work formalizes the Constrained Optimal Fuel Consumption under Observational Noise (COFC-ON) problem, extending prior COFC formulations to explicitly include sensor noise in battery SOC and speed references. It couples a robust constrained reinforcement learning approach with a Lagrangian PPO objective to enforce SOC constraints under all admissible observational perturbations, using a uniform noise model within a bounded $\ell_p$-ball. The authors develop a structured training framework that co-adapts the perturbation range and validate the method on a Prius THS model across NEDC and WLTC cycles, showing SOC balance is preserved and fuel consumption remains robust across noise levels, with cycle-dependent sensitivity. The results provide practical guidance for evaluating and controlling HEV energy management under real-world sensing inaccuracies, informing PEC and dynamometer testing practices and enabling more reliable deployment of CRL-based energy control. The work also discusses limitations, industrial relevance, and pathways to extend robust CRL methods to broader automotive tasks.

Abstract

In our prior work, we investigated the minimum fuel consumption of a hybrid electric vehicle (HEV) under a state-of-charge (SOC) balance constraint, assuming perfect SOC measurements and accurate reference speed profiles. The constrained optimal fuel consumption (COFC) problem was addressed using a constrained reinforcement learning (CRL) framework. However, in real-world scenarios, SOC readings are often corrupted by sensor noise, and reference speeds may deviate from actual driving conditions. To account for these imperfections, this study reformulates the COFC problem by explicitly incorporating observational noise in both SOC and reference speed. We adopt a robust CRL approach, where the noise is modeled as a uniform distribution, and employ a structured training procedure to ensure stability. The proposed method is evaluated through simulations on the Toyota Prius hybrid system (THS), using both the New European Driving Cycle (NEDC) and the Worldwide Harmonized Light Vehicles Test Cycle (WLTC). Results show that fuel consumption and SOC constraint satisfaction remain robust across varying noise levels. Furthermore, the analysis reveals that observational noise in SOC and speed can impact fuel consumption to different extents. To the best of our knowledge, this is the first study to explicitly examine how observational noise -- commonly encountered in dynamometer testing and predictive energy control (PEC) applications -- affects constrained optimal fuel consumption in HEVs.

Figures (7)

• Figure 1: Battery SOC constraint region (red polyhedron) with added speed dimension, extended from Fig. 1 of our seminal work yancofc.
• Figure 2: Training procedure: Algorithm 2 as a specific implementation of Algorithm 1 using Lagrangian-based PPO algorithm.
• Figure 3: The Prius THS structure.
• Figure 4: Vehicle behavior under the NEDC cycle using the policy trained with the n12 noise configuration (seed 0). Results are shown under both noise-free and noisy conditions, with reference speed, engine torque, motor torque, and battery SOC from top to bottom.
• Figure 5: Vehicle behavior under the WLTC cycle using the policy trained with the w12 noise configuration (seed 0). Results are shown under both noise-free and noisy conditions, with reference speed, engine torque, motor torque, and battery SOC from top to bottom.