Invariant Control Strategies for Active Flow Control using Graph Neural Networks

TL;DR

This work addresses the generalization and computational-cost limitations of reinforcement learning for active flow control by introducing graph neural networks that inherently handle unstructured flow data and enforce permutation, rotational, and reflection invariances. Using PPO within the Relexi framework and high-order FLEXI simulations, the authors compare a GCNN policy to a baseline MLP on a canonical cylinder wake problem at $\mathrm{Re}_D=100$ and $\mathrm{Ma}=0.2$, with observations from 11 pressure probes. The GCNN achieves drag-reduction performance comparable to the MLP while exhibiting superior permutation invariance and generalization, and trains efficiently due to weight sharing and invariant architectures. The results demonstrate the viability of GNNs for RL-based active flow control and suggest their applicability to more complex, large-scale 3D flows, supported by the HPC-enabled Relexi/FLEXI workflow.

Abstract

Reinforcement learning has gained traction for active flow control tasks, with initial applications exploring drag mitigation via flow field augmentation around a two-dimensional cylinder. RL has since been extended to more complex turbulent flows and has shown significant potential in learning complex control strategies. However, such applications remain computationally challenging due to its sample inefficiency and associated simulation costs. This fact is worsened by the lack of generalization capabilities of these trained policy networks, often being implicitly tied to the input configurations of their training conditions. In this work, we propose the use of graph neural networks to address this particular limitation, effectively increasing the range of applicability and getting more value out of the upfront RL training cost. GNNs can naturally process unstructured, threedimensional flow data, preserving spatial relationships without the constraints of a Cartesian grid. Additionally, they incorporate rotational, reflectional, and permutation invariance into the learned control policies, thus improving generalization and thereby removing the shortcomings of commonly used CNN or MLP architectures. To demonstrate the effectiveness of this approach, we revisit the well-established two-dimensional cylinder benchmark problem for active flow control. The RL training is implemented using Relexi, a high-performance RL framework, with flow simulations conducted in parallel using the high-order discontinuous Galerkin framework FLEXI. Our results show that GNN-based control policies achieve comparable performance to existing methods while benefiting from improved generalization properties. This work establishes GNNs as a promising architecture for RL-based flow control and highlights the capabilities of Relexi and FLEXI for large-scale RL applications in fluid dynamics.

Figures (11)

• Figure 1: Simulation setup for the flow around a two-dimensional cylinder in a channel. The positions of the 11 pressure probes are highlighted by white circles and the jets by white areas at the top and bottom of the cylinder. The field solution shows the velocity magnitude. The domain is clipped here to a length of $y=16$ for better visual representation.
• Figure 2: MLP policy network architecture using 2 hidden layers with 256 neurons each.
• Figure 3: Pressure probes' connectivity used to build the adjacency matrix for the graph net.
• Figure 4: GNN policy network architecture. The data exchange between a pair of neighbors is demonstrated, where the blue boxes represent the message passing layers and the red boxes are standard dense layers. By virtue of their construction, the weights are shared across the nodes, and hence this architecture can be considered to induce the MARL paradigm.
• Figure 5: Actor's learning rate decay throughout training. The learning rate becomes constant after 100 iterations with the last value (i.e. $10^{-4}$ in this case).