Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reinforcement Learning-Based Energy Management for Industrial Park with Heterogeneous Batteries under Demand Response

Meng Yuan, Tinghui Yan, Zhezhuang Xu

Abstract

The integration of photovoltaic (PV) systems, stationary energy storage systems (ESSs), and electric vehicles (EVs) alongside demand response (DR) programmes in industrial parks presents opportunities to reduce costs and improve renewable energy utilisation. Coordinating these resources is challenging because office and production zones have distinct operational objectives, and battery ageing costs are often ignored. This paper proposes a DR-based energy management framework that jointly optimises grid interaction costs, thermal comfort, EV departure state-of-charge requirements, carbon emissions, and battery ageing. We model heterogeneous load characteristics using a dynamic energy distribution ratio and incorporate dispatch-level ageing models for both ESS and EV batteries. The problem is formulated as a Markov decision process (MDP) and solved with a deep deterministic policy gradient (DDPG) algorithm. High-fidelity simulations using data from a practical industrial park in China show the framework maintains indoor comfort while significantly reducing total operating costs, yielding savings of 44.58\% and 40.68\% compared with a rule-based DR strategy and a conventional time-of-use arbitrage approach, respectively.

Reinforcement Learning-Based Energy Management for Industrial Park with Heterogeneous Batteries under Demand Response

Abstract

The integration of photovoltaic (PV) systems, stationary energy storage systems (ESSs), and electric vehicles (EVs) alongside demand response (DR) programmes in industrial parks presents opportunities to reduce costs and improve renewable energy utilisation. Coordinating these resources is challenging because office and production zones have distinct operational objectives, and battery ageing costs are often ignored. This paper proposes a DR-based energy management framework that jointly optimises grid interaction costs, thermal comfort, EV departure state-of-charge requirements, carbon emissions, and battery ageing. We model heterogeneous load characteristics using a dynamic energy distribution ratio and incorporate dispatch-level ageing models for both ESS and EV batteries. The problem is formulated as a Markov decision process (MDP) and solved with a deep deterministic policy gradient (DDPG) algorithm. High-fidelity simulations using data from a practical industrial park in China show the framework maintains indoor comfort while significantly reducing total operating costs, yielding savings of 44.58\% and 40.68\% compared with a rule-based DR strategy and a conventional time-of-use arbitrage approach, respectively.

Paper Structure

This paper contains 33 sections, 42 equations, 8 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. System Models and Problem Formulation
  3. System Modelling
  4. PV Model
  5. HVAC Model
  6. Battery Model
  7. Power Balancing
  8. Multi-Objective Optimisation Problem Formulation
  9. Demand Response Revenue
  10. Grid Interaction Cost
  11. Carbon Emission Cost
  12. Battery Ageing Cost
  13. Temperature Deviation Penalty
  14. SoC Departure Constraint Penalty
  15. Proposed Method
  16. ...and 18 more sections

Figures (8)

  • Figure 1: Schematic diagram of the industrial park energy management system.
  • Figure 2: Cycling ageing results of LFP/NMC batteries evaluated at 40$^{\circ}\mathrm{C}$ and a C-rate of 1.0.
  • Figure 3: Agent-environment interaction in the MDP.
  • Figure 4: Architecture of the DDPG algorithm.
  • Figure 5: Episode reward during the DDPG training process.
  • ...and 3 more figures