Optimal Lattice Boltzmann Closures through Multi-Agent Reinforcement Learning

TL;DR

The paper tackles the instability of under-resolved LBM by deploying a data-driven, multi-agent reinforcement learning framework that autonomously tunes the local over-relaxation parameter to stabilize simulations and recover DNS-like energy spectra. By integrating a fully convolutional MARL policy with LBM and a centralized critic during training, the method achieves robust stability and accurate spectral fidelity across Kolmogorov-flow turbulence, decaying flows, and high-Reynolds-number tests, while remaining computationally efficient. The approach demonstrates superior spectral accuracy and generalization compared to traditional closures such as KBC and BGK, and it maintains transferability to unseen flow scenarios without requiring DNS data during training. This work highlights MARL as a promising path for data-driven, scalable, and physically consistent closures in lattice Boltzmann simulations, with potential extensions to complex boundaries and porous-media flows.

Abstract

The Lattice Boltzmann method (LBM) offers a powerful and versatile approach to simulating diverse hydrodynamic phenomena, spanning microfluidics to aerodynamics. The vast range of spatiotemporal scales inherent in these systems currently renders full resolution impractical, necessitating the development of effective closure models for under-resolved simulations. Under-resolved LBMs are unstable, and while there is a number of important efforts to stabilize them, they often face limitations in generalizing across scales and physical systems. We present a novel, data-driven, multiagent reinforcement learning (MARL) approach that drastically improves stability and accuracy of coarse-grained LBM simulations. The proposed method uses a convolutional neural network to dynamically control the local relaxation parameter for the LB across the simulation grid. The LB-MARL framework is showcased in turbulent Kolmogorov flows. We find that the MARL closures stabilize the simulations and recover the energy spectra of significantly more expensive fully resolved simulations while maintaining computational efficiency. The learned closure model can be transferred to flow scenarios unseen during training and has improved robustness and spectral accuracy compared to traditional LBM models. We believe that MARL closures open new frontiers for efficient and accurate simulations of a multitude of complex problems not accessible to present-day LB methods alone.

Figures (7)

• Figure 1: Reinforcement learning framework for adaptive control of the over-relaxation parameter in LBGK. Each agent samples an action from its policy, with actions interpolated across the grid and applied to the environment. Agents receive local state observations and a global reward based on the alignment of the coarse-grained simulation energy spectrum with the target DNS spectrum.
• Figure 2: Example for a policy network parametrization in an environment with local and cooperating agents. Figure \ref{['img:local-actor']} shows the network architecture for a fully local agent. The agent only receives the state at its location which in this case is a six dimensional vector. The network then consists of two fully convolutional layers and has two output neurons, namely the mean $\mu$ and standard deviation $\sigma$. These parametrize a normal distribution from which the agent can sample actions $\pi_i(a|s)\sim \mathcal{N}(\mu, \sigma).$ Figure \ref{['img:vectorized-actors']} show the vectorized version of Fig. \ref{['img:local-actor']}. The experience of all agents is combined and handled as a single evaluation of a fully convolutional network.
• Figure 3: Neural network architecture for a centralized critic network used in the learning phase by the MARL PPO algorithm. The Network receives a state $S$ as input, which is compressed by six convolutional and three fully connected layers, and outputs a value function estimate $V(S)$.
• Figure 4: Evaluation of trained models on Kolmogorov flow at $Re=10^4$. (\ref{['img:corrs_1e4']}) show the vorticity correlation of models with the DNS. All three models are able to stabilize the simulation. (\ref{['img:spectra_1e4']}) shows the energy spectra scaled by $k^5$, averaged over the second half of the simulation $T\in[113, 227]$. All three models reproduce the target spectrum of the DNS, with small deviations at higher wave numbers.
• Figure 5: Evaluation of trained models on an unforced decaying flow at $Re=10^4$.(\ref{['img:corrs_1e4_decay']}) show the vorticity correlation of models with the DNS. (\ref{['img:spectra_1e4_decay']}) shows the energy spectra scaled by $k^5$, averaged over the second half of the simulation $T\in[113, 227]$.