Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Reinforcement Learning for the Heat Transfer Control of Pulsating Impinging Jets

Sajad Salavatidezfouli, Giovanni Stabile, Gianluigi Rozza

TL;DR

The paper tackles active thermal control of forced convection using a pulsating impinging jet on a heated plate within a CFD environment. It systematically compares vanilla DQN and variants—Double DQN, Soft Double DQN, and Dueling DQN—for jet-velocity control, while examining sensor placement and episode-length effects. The findings show that Soft Double DQN and Dueling DQN achieve stable, near-setpoint surface temperatures with high reliability (e.g., >98% of the control cycle), whereas classical DQN and Hard Double DQN struggle with instability or larger temperature gradients. This work demonstrates the viability of DRL-CFD for active thermal management in impinging-jet cooling and highlights practical design considerations for state representation and target-update strategies.

Abstract

This research study explores the applicability of Deep Reinforcement Learning (DRL) for thermal control based on Computational Fluid Dynamics. To accomplish that, the forced convection on a hot plate prone to a pulsating cooling jet with variable velocity has been investigated. We begin with evaluating the efficiency and viability of a vanilla Deep Q-Network (DQN) method for thermal control. Subsequently, a comprehensive comparison between different variants of DRL is conducted. Soft Double and Duel DQN achieved better thermal control performance among all the variants due to their efficient learning and action prioritization capabilities. Results demonstrate that the soft Double DQN outperforms the hard Double DQN. Moreover, soft Double and Duel can maintain the temperature in the desired threshold for more than 98% of the control cycle. These findings demonstrate the promising potential of DRL in effectively addressing thermal control systems.

Deep Reinforcement Learning for the Heat Transfer Control of Pulsating Impinging Jets

TL;DR

The paper tackles active thermal control of forced convection using a pulsating impinging jet on a heated plate within a CFD environment. It systematically compares vanilla DQN and variants—Double DQN, Soft Double DQN, and Dueling DQN—for jet-velocity control, while examining sensor placement and episode-length effects. The findings show that Soft Double DQN and Dueling DQN achieve stable, near-setpoint surface temperatures with high reliability (e.g., >98% of the control cycle), whereas classical DQN and Hard Double DQN struggle with instability or larger temperature gradients. This work demonstrates the viability of DRL-CFD for active thermal management in impinging-jet cooling and highlights practical design considerations for state representation and target-update strategies.

Abstract

This research study explores the applicability of Deep Reinforcement Learning (DRL) for thermal control based on Computational Fluid Dynamics. To accomplish that, the forced convection on a hot plate prone to a pulsating cooling jet with variable velocity has been investigated. We begin with evaluating the efficiency and viability of a vanilla Deep Q-Network (DQN) method for thermal control. Subsequently, a comprehensive comparison between different variants of DRL is conducted. Soft Double and Duel DQN achieved better thermal control performance among all the variants due to their efficient learning and action prioritization capabilities. Results demonstrate that the soft Double DQN outperforms the hard Double DQN. Moreover, soft Double and Duel can maintain the temperature in the desired threshold for more than 98% of the control cycle. These findings demonstrate the promising potential of DRL in effectively addressing thermal control systems.
Paper Structure (15 sections, 13 equations, 14 figures, 1 table)

This paper contains 15 sections, 13 equations, 14 figures, 1 table.

Table of Contents

  1. Introduction
  2. Reinforcement Learning
  3. Deep Q Network
  4. DQN Improvements
  5. DQN Algorithm
  6. Impingement Jet
  7. Physics of the Flow
  8. Setup Description
  9. Results and Disccusion
  10. DRL Specifications
  11. Sensitivity Analysis
  12. Classical DQN
  13. Episode Number
  14. DQN Variants
  15. Conclusion

Figures (14)

  • Figure 1: General overview of the DRL-CFD framework
  • Figure 2: Formation of different regions in the context of the impinging jet on a flat plate sodjavi2015impinging
  • Figure 3: Schematic representation of the computational domain along with the dimension data of the jet and hot plate
  • Figure 4: Representation of the structured mesh for the domain
  • Figure 5: Probes location for measurement of states during DRL are shown with blue dots
  • ...and 9 more figures