Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stabilizing Rayleigh-Benard convection with reinforcement learning trained on a reduced-order model

Qiwei Chen, C. Ricardo Constante-Amores

TL;DR

DManD-RL is established as a physically interpretable, scalable approach for turbulence control in high-dimensional flows by establishing DManD-RL as a physically interpretable, scalable approach for turbulence control in high-dimensional flows.

Abstract

Rayleigh-Benard convection (RBC) is a canonical system for buoyancy-driven turbulence and heat transport, central to geophysical and industrial flows. Developing efficient control strategies remains challenging at high Rayleigh numbers, where fully resolved simulations are computationally expensive. We use a control framework that couples data-driven manifold dynamics (DManD) with reinforcement learning (RL) to suppress convective heat transfer. We find a coordinate transformation to a low-dimensional system using POD and autoencoders, and then learn an evolution equation for this low-dimensional state using neural ODEs. The reduced model reproduces key system features while enabling rapid policy training. Policies trained in the DManD environment and deployed in DNS achieve a 16-23 % reduction in the Nusselt number for both single- and dual-boundary actuation. Physically, the learned strategy modulates near-wall heat flux to stabilize and thicken the thermal boundary layer, weaken plume ejection, and damp the wall-driven instabilities that seed convective bursts. Crucially, the controller drives the flow toward a quasi-steady state characterized by suppressed temporal fluctuations and spatially steady heat-flux patterns. This work establishes DManD-RL as a physically interpretable, scalable approach for turbulence control in high-dimensional flows.

Stabilizing Rayleigh-Benard convection with reinforcement learning trained on a reduced-order model

TL;DR

DManD-RL is established as a physically interpretable, scalable approach for turbulence control in high-dimensional flows by establishing DManD-RL as a physically interpretable, scalable approach for turbulence control in high-dimensional flows.

Abstract

Rayleigh-Benard convection (RBC) is a canonical system for buoyancy-driven turbulence and heat transport, central to geophysical and industrial flows. Developing efficient control strategies remains challenging at high Rayleigh numbers, where fully resolved simulations are computationally expensive. We use a control framework that couples data-driven manifold dynamics (DManD) with reinforcement learning (RL) to suppress convective heat transfer. We find a coordinate transformation to a low-dimensional system using POD and autoencoders, and then learn an evolution equation for this low-dimensional state using neural ODEs. The reduced model reproduces key system features while enabling rapid policy training. Policies trained in the DManD environment and deployed in DNS achieve a 16-23 % reduction in the Nusselt number for both single- and dual-boundary actuation. Physically, the learned strategy modulates near-wall heat flux to stabilize and thicken the thermal boundary layer, weaken plume ejection, and damp the wall-driven instabilities that seed convective bursts. Crucially, the controller drives the flow toward a quasi-steady state characterized by suppressed temporal fluctuations and spatially steady heat-flux patterns. This work establishes DManD-RL as a physically interpretable, scalable approach for turbulence control in high-dimensional flows.

Paper Structure

This paper contains 15 sections, 22 equations, 14 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Formulation
  3. Direct numerical simulation of RBC
  4. Learning manifold coordinates
  5. Linear reduction via POD
  6. Nonlinear reduction via autoencoders
  7. Learning NODE in actuated dynamics
  8. Learning control strategy in the low dimensional model
  9. Apply control strategy in the full DNS
  10. Results
  11. Dimension reduction
  12. Neural ODE for latent dynamics
  13. DNS evaluation of DManD-RL control strategies
  14. Physical interpretation of the $Nu$ Reduction
  15. Conclusion

Figures (14)

  • Figure 1: Framework of boundary-forcing configurations used to generate the dataset.
  • Figure 2: Linear dimension reduction: (a) Eigenvalue spectrum. (b) Temporal evolution of the reconstruction error $\| \mathbf{q}_i-\tilde{\mathbf{q}}_i\|_2/\|\mathbf{q}_i\|_2$ on the test set for single and dual actuation cases. (c–d) Snapshots of the temperature field for the DNS and POD reconstructions (right) for single- and dual-boundary cases.
  • Figure 3: Non-linear dimension reduction via autoencoders. (a) Average relative error in POD coefficients as a function of the latent dimension $d_h$. (b–c) Temporal evolution of reconstruction errors for the dual- and single-boundary cases at $r=d_h=88$.
  • Figure 4: Comparison of DNS and DManD reconstructed fields for (a) single-boundary and (b) dual-boundary control cases. Each panel shows $\|\mathbf{u}\|$ and $T$ at three representative times.
  • Figure 5: Pointwise time series of temperature $T$ at a probe located at $(x,y)=(\tfrac{\pi}{2},\,0.1)$ for (a) single-boundary and (b) dual-boundary control cases.
  • ...and 9 more figures