End-to-End Mesh Optimization of a Hybrid Deep Learning Black-Box PDE Solver

TL;DR

This work tackles end-to-end training of a CFD-GCN hybrid when the external PDE solver is a black box and cannot be differentiated. It introduces zeroth-order gradient estimators—Coordinate-ZO, Gaussian-ZO, and Gaussian-Coordinate-ZO—to approximate solver gradients through forward propagation, enabling joint optimization of neural parameters $\theta$ and coarse-mesh coordinates $M_{\text{coarse}}$ given a fine mesh $M_{\text{fine}}$ and flow parameters. Experimental results show zeroth-order methods can outperform a frozen-mesh baseline and achieve competitive performance relative to gradient-based methods, with a simple warm-start strategy significantly improving convergence and generalization. The findings demonstrate the feasibility of integrating non-differentiable physics solvers into differentiable learning workflows, potentially extending to a wide range of CFD and other physics-based simulations.

Abstract

Deep learning has been widely applied to solve partial differential equations (PDEs) in computational fluid dynamics. Recent research proposed a PDE correction framework that leverages deep learning to correct the solution obtained by a PDE solver on a coarse mesh. However, end-to-end training of such a PDE correction model over both solver-dependent parameters such as mesh parameters and neural network parameters requires the PDE solver to support automatic differentiation through the iterative numerical process. Such a feature is not readily available in many existing solvers. In this study, we explore the feasibility of end-to-end training of a hybrid model with a black-box PDE solver and a deep learning model for fluid flow prediction. Specifically, we investigate a hybrid model that integrates a black-box PDE solver into a differentiable deep graph neural network. To train this model, we use a zeroth-order gradient estimator to differentiate the PDE solver via forward propagation. Although experiments show that the proposed approach based on zeroth-order gradient estimation underperforms the baseline that computes exact derivatives using automatic differentiation, our proposed method outperforms the baseline trained with a frozen input mesh to the solver. Moreover, with a simple warm-start on the neural network parameters, we show that models trained by these zeroth-order algorithms achieve an accelerated convergence and improved generalization performance.

Figures (33)

• Figure 1: Illustration of CFD-GCN belbute2020combining. Both the GCN model parameters and the coarse mesh's node positions are trainable.
• Figure 2: Test loss comparison among Coordinate-ZO, Grad and Grad-FrozenMesh.
• Figure 3: Visualization of the pressure fields predicted by Grad, Grad-FrozenMesh, and Coordinate-ZO with $b=4$ for $\text{AoA}=9.0$ and $\text{mach number}=0.8$. We further evaluate the difference between the prediction over meshes updated by first-order and Coordinate-ZO method in Fig.(e) and Fig.(f). Notably, exhibits a significantly higher light intensity than Fig. (f), indicating a larger divergence between the figures. This observation implies that even when updating the mesh is not applicable (e.g. the auto-differentiation is not supported), we can still apply the zeroth-order method (Coordinate-ZO) to update these parameters and obtain more consistent result as it does (Grad). We have adjusted the brightness and contrast to better distinguish the difference.
• Figure 4: Comparison between the mesh before (blue) and after (red) being optimized.
• Figure 5: Test loss comparison among Gaussian-ZO, Grad, and Grad-FrozenMesh.

Theorems & Definitions (3)

• Remark
• Remark
• Remark