tempoGAN: A Temporally Coherent, Volumetric GAN for Super-resolution Fluid Flow

TL;DR

TempoGAN tackles extracting high-fidelity, temporally coherent 4D flow fields from a single low-resolution snapshot by integrating a conditional GAN with a novel temporal discriminator $D_t$. By incorporating physics-aware inputs such as velocity $\mathbf{v}$ and vorticity $\boldsymbol{\omega}$ and employing physics-aware data augmentation, it learns advection-driven details without requiring temporal sequences in the input, producing consistent $3D$ and $4D$ outputs efficiently via tiling. The method introduces a feature-space loss $\mathcal{L}_f$ and jointly optimizes a spatial discriminator $D_s$ and the temporal discriminator $D_t$ to promote realism and temporal coherence, achieving high-quality results in both 2D and 3D flows and offering artistic control through input conditioning. Overall, tempoGAN provides a general, scalable framework for physics-informed super-resolution of turbulent flows with potential extensions to other physical domains and to non-physical sequence data.

Abstract

We propose a temporally coherent generative model addressing the super-resolution problem for fluid flows. Our work represents a first approach to synthesize four-dimensional physics fields with neural networks. Based on a conditional generative adversarial network that is designed for the inference of three-dimensional volumetric data, our model generates consistent and detailed results by using a novel temporal discriminator, in addition to the commonly used spatial one. Our experiments show that the generator is able to infer more realistic high-resolution details by using additional physical quantities, such as low-resolution velocities or vorticities. Besides improvements in the training process and in the generated outputs, these inputs offer means for artistic control as well. We additionally employ a physics-aware data augmentation step, which is crucial to avoid overfitting and to reduce memory requirements. In this way, our network learns to generate advected quantities with highly detailed, realistic, and temporally coherent features. Our method works instantaneously, using only a single time-step of low-resolution fluid data. We demonstrate the abilities of our method using a variety of complex inputs and applications in two and three dimensions.

Figures (18)

• Figure 1: This figure gives a high level overview of our approach: a generator on the left, is guided during training by two discriminator networks (right), one of which focuses on space ($D_s$), while the other one focuses on temporal aspects ($D_t$). At runtime, both are discarded, and only the generator network is evaluated.
• Figure 2: From left to right: a) a sample, low-resolution input, b) a CNN output with naive $L_2$ loss (no GAN training), c) our tempoGAN output, and d) the high-resolution reference. The $L_2$ version learns a smooth result without small scale details, while our output in (c) surpasses the detail of the reference in certain regions.
• Figure 3: A comparison of different approaches for temporal coherence. The top two rows show the inferred densities, while the bottom two rows contain the time derivative of the frame content computed with a finite difference between frame $t$ and $t+1$. Positive and negative values are color-coded with red and blue, respectively. From left to right: no temporal loss applied, $\mathcal{L}_{2,t}$ loss applied, $\mathcal{L}_{D_t'}$, i.e., applied without advection, $\mathcal{L}_{D_t}$ applied with advection (our full tempoGAN approach), and the ground-truth $y$. From left to right across the different versions, the derivatives become less jagged and less noisy, as well as more structured and narrow. This means the temporal coherence is improved, esp. for the result from our algorithm ($\mathcal{L}_{D_t}$).
• Figure 4: These images highlight data alignment due to advection. Three consecutive frames are encoded as R, G, B channels of a single image, thus, ideally a fully aligned image would only contain shades of grey. The two rows contain front and top views in the top and bottom row, respectively. We show two examples, a) and b). Each of them contains $\widetilde{Y}$ left, and $\widetilde{Y}_{\mathcal{A}}$ right. The RGB channels are the three input frames, t-1, t, and t+1. Compared with $\widetilde{Y}$, $\widetilde{Y}_{\mathcal{A}}$ is significantly less saturated, i.e., better aligned.
• Figure 5: Here an overview of our tempoGAN architecture is shown. The three neural networks (blue boxes) are trained in conjunction. The data flow between them is highlighted by the red and black arrows. Note that $x$ and $y$ denote fluid data that contains velocity and/or vorticity fields, as well as density depending on the chosen architecture (see Sec. \ref{['sec:inputs']}).