Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Physics-Informed Reinforcement Learning Approach for Degradation-Aware Long-Term Charging Optimization in Batteries

Shanthan Kumar Padisala, Bharatkumar Hegde, Ibrahim Haskara, Satadru Dey

TL;DR

This work tackles the challenge of aging in battery charging by making CCCV charging adaptive to long-term health through a physics-informed reinforcement learning framework. By jointly estimating Loss of Active Material (LAM) and adjusting the CCCV current, the approach embeds DFN-based battery physics and a reduced-order voltage predictor from a Single Particle Model into a PPO-trained agent within an OpenAI Gym environment. The authors demonstrate that incorporating physics into the RL reward and state representation yields lower capacity fade over 100 cycles compared to a constant-CCCV baseline and a physics-agnostic RL, with the proposed method showing 8.54% fade versus 9.61% and 10.34% for the baselines respectively. This indicates the practical potential of degradation-aware charging policies for extending battery life in EV applications, through a physics-informed data-driven control framework suitable for integration into battery management systems.

Abstract

Batteries degrade with usage and continuous cycling. This aging is typically reflected through the resistance growth and the capacity fade of battery cells. Over the years, various charging methods have been presented in the literature that proposed current profiles in order to enable optimal, fast, and/or health-conscious charging. However, very few works have attempted to make the ubiquitous Constant Current Constant Voltage (CCCV) charging protocol adaptive to the changing battery health as it cycles. This work aims to address this gap and proposes a framework that optimizes the constant current part of the CCCV protocol adapting to long-term battery degradation. Specifically, a physics-informed Reinforcement Learning (RL) approach has been used that not only estimates a key battery degradation mechanism, namely, Loss of Active Material (LAM), but also adjusts the current magnitude of CCCV as a result of this particular degradation. The proposed framework has been implemented by combining PyBamm, an open-source battery modeling tool, and Stable-baselines where the RL agent was trained using a Proximal Policy Optimization (PPO) network. Simulation results show the potential of the proposed framework for enhancing the widely used CCCV protocol by embedding physics information in RL algorithm. A comparative study of this proposed agent has also been discussed with 2 other charging protocols generated by a non-physics-based RL agent and a constant CCCV for all the cycles.

A Physics-Informed Reinforcement Learning Approach for Degradation-Aware Long-Term Charging Optimization in Batteries

TL;DR

This work tackles the challenge of aging in battery charging by making CCCV charging adaptive to long-term health through a physics-informed reinforcement learning framework. By jointly estimating Loss of Active Material (LAM) and adjusting the CCCV current, the approach embeds DFN-based battery physics and a reduced-order voltage predictor from a Single Particle Model into a PPO-trained agent within an OpenAI Gym environment. The authors demonstrate that incorporating physics into the RL reward and state representation yields lower capacity fade over 100 cycles compared to a constant-CCCV baseline and a physics-agnostic RL, with the proposed method showing 8.54% fade versus 9.61% and 10.34% for the baselines respectively. This indicates the practical potential of degradation-aware charging policies for extending battery life in EV applications, through a physics-informed data-driven control framework suitable for integration into battery management systems.

Abstract

Batteries degrade with usage and continuous cycling. This aging is typically reflected through the resistance growth and the capacity fade of battery cells. Over the years, various charging methods have been presented in the literature that proposed current profiles in order to enable optimal, fast, and/or health-conscious charging. However, very few works have attempted to make the ubiquitous Constant Current Constant Voltage (CCCV) charging protocol adaptive to the changing battery health as it cycles. This work aims to address this gap and proposes a framework that optimizes the constant current part of the CCCV protocol adapting to long-term battery degradation. Specifically, a physics-informed Reinforcement Learning (RL) approach has been used that not only estimates a key battery degradation mechanism, namely, Loss of Active Material (LAM), but also adjusts the current magnitude of CCCV as a result of this particular degradation. The proposed framework has been implemented by combining PyBamm, an open-source battery modeling tool, and Stable-baselines where the RL agent was trained using a Proximal Policy Optimization (PPO) network. Simulation results show the potential of the proposed framework for enhancing the widely used CCCV protocol by embedding physics information in RL algorithm. A comparative study of this proposed agent has also been discussed with 2 other charging protocols generated by a non-physics-based RL agent and a constant CCCV for all the cycles.

Paper Structure

This paper contains 5 sections, 26 equations, 6 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. Dynamics of Battery Systems: Nominal Behavior and Degradation Mechanism
  3. Reinforcement Learning Based Long-Term Charging Optimizer
  4. Results and Discussion
  5. Conclusion

Figures (6)

  • Figure 1: Schematic of the charging optimizer framework.
  • Figure 2: Applied charging current and corresponding terminal voltage responses under various points of battery age.
  • Figure 3: Changes in applied charging current and corresponding changes in terminal voltage responses under various points of battery age.
  • Figure 4: Top Plot: Lithium concentrations in anode and cathode during charging under various points of battery age. Bottom Plot: Zoomed-in Lithium concentrations in anode and cathode during charging under various points of battery age.
  • Figure 5: The estimation performance by the proposed framework (RL with LAM estimate): Comparison of the true cathode active material volume fraction and the estimates by the RL agent.
  • ...and 1 more figures