From synthetic turbulence to true solutions: A deep diffusion model for discovering periodic orbits in the Navier-Stokes equations

TL;DR

This work demonstrates how a generative diffusion model can be used to uncover previously unknown solutions of a nonlinear partial differential equation: the two-dimensional Navier-Stokes equations in a turbulent regime and suggests a broader role for generative AI: not as replacements for simulation and existing solvers, but as a complementary tool for navigating the complex solution spaces of nonlinear dynamical systems.

Abstract

Generative artificial intelligence has shown remarkable success in synthesizing data that mimic complex real-world systems, but its potential role in the discovery of mathematically meaningful structures in physical models remains underexplored. In this work, we demonstrate how a generative diffusion model can be used to uncover previously unknown solutions of a nonlinear partial differential equation: the two-dimensional Navier-Stokes equations in a turbulent regime. Trained on data from a direct numerical simulation of turbulence, the model learns to generate time series that resemble physically plausible trajectories. By carefully modifying the temporal structure of the model and enforcing the symmetries of the governing equations, we produce synthetic trajectories that are periodic in time, despite the fact that the training data did not contain periodic trajectories. These synthetic trajectories are then refined into true solutions using an iterative solver, yielding 111 new periodic orbits (POs) with very short periods. Our results reveal a previously unobserved richness in the PO structure of this system and suggest a broader role for generative AI: not as replacements for simulation and existing solvers, but as a complementary tool for navigating the complex solution spaces of nonlinear dynamical systems.

Figures (10)

• Figure 1: The rate of energy dissipation $D$ against production $P$ for a typical chaotic trajectory of length $T=500$. The trajectory spends the overwhelming majority of the time at low dissipations, with $D\le0.15$, but occasional 'bursts' to higher $D$ are visible.
• Figure 2: The symmetric structure of the kernels for convolutions of type (a) and (b), assuming a size $5\times5$, with respectively 9 and 4 unique elements. In practice the kernels are $5\times 5\times 5 \times d_i \times d_o$ arrays, but no special structure is imposed in the third (time) dimension, nor in the dimensions covering the number of input and output channels $d_i$ and $d_o$.
• Figure 3: The equivariant convolution block used throughout the neural network. The sizes shown are for the final conv of \ref{['fig:unet']}. The symmetric and antisymmetric convolutions have with kernels of type (a) and (b) respectively, as depicted in \ref{['fig:kernels']}. Each is preceded by periodic padding in the spatial dimensions and carefully chosen padding in the third dimension, and followed by tanhshrink activation. The additional conv3d on the left is a $1\times1\times1$ convolution to reshape the residual connection before the final addition. The forcing layer multiplies the MLP outputs by the vorticity forcing field and concatenates this with the other channels.
• Figure 4: The structure of the equivariant U-Net used by the diffusion model. Each conv block is as per \ref{['fig:conv']}. We use average pooling and trilinear upsampling. The concatenation is along the 4th dimension. The sizes given are those used during generation; during training, all layers were $8$ times larger in the third dimension.
• Figure 5: Energy dissipation $D$ versus production $P$ for synthetic orbits generated with $(N,M)=(64,64)$. Ten examples from each of the symmetries are shown. The examples were chosen at random: many of these converged to true solutions, many did not.