Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Out-of-Distribution-Aware Electric Vehicle Charging

Tongxin Li, Chenxi Sun

TL;DR

The paper tackles EV charging under distribution shifts by proposing OOD-Charging, a learning-augmented policy that fuses a value-based neural policy with MPC. A real-time, TD-error–driven awareness radius governs how much the neural advice is trusted, yielding a principled consistency-robustness trade-off. The authors provide theoretical guarantees on performance, and validate the approach with case studies based on ACN-Data, including COVID-19 and solar-variation scenarios, showing improved adaptability and reliability in OOD conditions. This framework offers a practical path to robust, high-performance EV charging in real-world, dynamically changing environments.

Abstract

We tackle the challenge of learning to charge Electric Vehicles (EVs) with Out-of-Distribution (OOD) data. Traditional scheduling algorithms typically fail to balance near-optimal average performance with worst-case guarantees, particularly with OOD data. Model Predictive Control (MPC) is often too conservative and data-independent, whereas Reinforcement Learning (RL) tends to be overly aggressive and fully trusts the data, hindering their ability to consistently achieve the best-of-both-worlds. To bridge this gap, we introduce a novel OOD-aware scheduling algorithm, denoted OOD-Charging. This algorithm employs a dynamic "awareness radius", which updates in real-time based on the Temporal Difference (TD)-error that reflects the severity of OOD. The OOD-Charging algorithm allows for a more effective balance between consistency and robustness in EV charging schedules, thereby significantly enhancing adaptability and efficiency in real-world charging environments. Our results demonstrate that this approach improves the scheduling reward reliably under real OOD scenarios with remarkable shifts of EV charging behaviors caused by COVID-19 in the Caltech ACN-Data.

Out-of-Distribution-Aware Electric Vehicle Charging

TL;DR

The paper tackles EV charging under distribution shifts by proposing OOD-Charging, a learning-augmented policy that fuses a value-based neural policy with MPC. A real-time, TD-error–driven awareness radius governs how much the neural advice is trusted, yielding a principled consistency-robustness trade-off. The authors provide theoretical guarantees on performance, and validate the approach with case studies based on ACN-Data, including COVID-19 and solar-variation scenarios, showing improved adaptability and reliability in OOD conditions. This framework offers a practical path to robust, high-performance EV charging in real-world, dynamically changing environments.

Abstract

We tackle the challenge of learning to charge Electric Vehicles (EVs) with Out-of-Distribution (OOD) data. Traditional scheduling algorithms typically fail to balance near-optimal average performance with worst-case guarantees, particularly with OOD data. Model Predictive Control (MPC) is often too conservative and data-independent, whereas Reinforcement Learning (RL) tends to be overly aggressive and fully trusts the data, hindering their ability to consistently achieve the best-of-both-worlds. To bridge this gap, we introduce a novel OOD-aware scheduling algorithm, denoted OOD-Charging. This algorithm employs a dynamic "awareness radius", which updates in real-time based on the Temporal Difference (TD)-error that reflects the severity of OOD. The OOD-Charging algorithm allows for a more effective balance between consistency and robustness in EV charging schedules, thereby significantly enhancing adaptability and efficiency in real-world charging environments. Our results demonstrate that this approach improves the scheduling reward reliably under real OOD scenarios with remarkable shifts of EV charging behaviors caused by COVID-19 in the Caltech ACN-Data.
Paper Structure (28 sections, 1 theorem, 31 equations, 7 figures, 2 tables, 3 algorithms)

This paper contains 28 sections, 1 theorem, 31 equations, 7 figures, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Reinforcement learning for EV charging
  3. Learning-augmented scheduling algorithms
  4. Integration of model-free and model-based methods
  5. Problem Formulation
  6. Preliminary: Markov Decision Process
  7. OOD-Aware EV Charging Problem
  8. Charging states and actions
  9. Charging dynamics
  10. Charging costs
  11. Model assumptions
  12. Scheduling Baseline: Model Predictive Control
  13. An OOD-Aware EV Charging Algorithm
  14. Value-Based Neural Policy
  15. Scheduling Policy Description
  16. ...and 13 more sections

Key Result

Theorem 4.1

Consider the EV charging problem in Section sec:model with Assumption assump:bounded-costs-and-dynamics and assump:uniform-stability. Suppose the neural policy satisfies Assumption ass:Q-Lip. With the choice of the awareness radius in eq:budget there exists $\beta>0$ such that OOD-Charging is $\left bounds the ratio of expectations of the MPC baseline (Procedure alg:mpc_update) with $\overline{\la

Figures (7)

  • Figure 1: Impact of pricing and social distancing policies on charging behaviors in the ACN-Data lee2018large where "ACN" represents Adaptive Charging Network. Blue Curve: number of EV charging sessions; Red Curve: average energy delivered to users.
  • Figure 2: OOD phenomenon observed in ACN-Data lee2019acn. (Left) Distribution Shift: Charging behaviors affected by the change of social distancing policies; (Right) User Input Data: Error in user inputs in terms of the energy demand and departure time. The user input predictions are used in the practically implemented MPC at Caltech EV charging station. See Section \ref{['sec:mpc_baseline']} for more details.
  • Figure 3: A system diagram of OOD-Charging (see Algorithm \ref{['alg:ppp']}), implemented at a workplace charging station with rooftop solar PV integration, as a motivating example of the OOD EV charging model (Section \ref{['sec:model']}). The distribution of $(\ell'_t-\Delta h'_t:t\in [T])$ shifts when charging behaviors and solar injections change over time.
  • Figure 4: Comparison of EV charging sessions on May 1, 2019 (pre-COVID), and July 1, 2021 (post-COVID), illustrating the distribution shift central to this case study. Post-COVID arrival times exhibit a flatter slope, aligning with the reduced density observed in the top-left section of Figure \ref{['fig:compare_together']}.
  • Figure 5: We evaluate the resilience of awards against distribution shifts by varying the tuning parameter $\beta$. The plot showcases average awards for $\beta$ values of $0, 0.1, 1, 10$, and $\infty$—the latter representing the direct application of the MPC baseline—within OOD-Charging's robustness framework. The shaded regions indicate the standard deviation across $10$ independent experiments. The implemented MPC (Pure MPC) is identical to the scheduling algorithms currently used at real EV charging stations at Caltech and JPL garages. Importantly, this implementation does not rely on historical charging data, which implies that its reward remains constant with respect to the training episode.
  • ...and 2 more figures

Theorems & Definitions (4)

  • Definition 4.1
  • Definition 4.2: Consistency and Robustness
  • Theorem 4.1
  • Definition 1: $r$-locally $p$-Wasserstein robustness li2023beyond