Meta-Auto-Decoder for Solving Parametric Partial Differential Equations

TL;DR

<3-5 sentence high-level summary> MAD tackles solving parametric PDEs by learning a mesh-free, unsupervised solver using a neural decoder $u_\theta(\tilde{x},z)$ that encodes heterogeneous PDE parameters into latent vectors $z$. Leveraging meta-learning, it pre-trains a shared model with task-specific latent codes $\{z_i^*\}$ to form a global initializer $\theta^*$, then solves new tasks via two fine-tuning schemes: MAD-L (freeze $\theta^*$, optimize $z$) and MAD-LM (jointly optimize $\theta$ and $z$). This framework is interpreted as manifold learning, treating $G(\mathcal{A})$ as a low-dimensional solution manifold $\bar{G}(Z)$ and performing search on/near it to accelerate adaptation. Empirical results on Burgers' equation, time-domain Maxwell's equations, and Laplace's equation show that MAD-LM achieves fast convergence and competitive or superior accuracy compared with baseline methods, including extrapolation to out-of-distribution PDE parameters. The approach offers a practical, scalable route to rapid, mesh-free PDE solving across heterogeneous and evolving problem settings.

Abstract

Many important problems in science and engineering require solving the so-called parametric partial differential equations (PDEs), i.e., PDEs with different physical parameters, boundary conditions, shapes of computation domains, etc. Recently, building learning-based numerical solvers for parametric PDEs has become an emerging new field. One category of methods such as the Deep Galerkin Method (DGM) and Physics-Informed Neural Networks (PINNs) aim to approximate the solution of the PDEs. They are typically unsupervised and mesh-free, but require going through the time-consuming network training process from scratch for each set of parameters of the PDE. Another category of methods such as Fourier Neural Operator (FNO) and Deep Operator Network (DeepONet) try to approximate the solution mapping directly. Being fast with only one forward inference for each PDE parameter without retraining, they often require a large corpus of paired input-output observations drawn from numerical simulations, and most of them need a predefined mesh as well. In this paper, we propose Meta-Auto-Decoder (MAD), a mesh-free and unsupervised deep learning method that enables the pre-trained model to be quickly adapted to equation instances by implicitly encoding (possibly heterogenous) PDE parameters as latent vectors. The proposed method MAD can be interpreted by manifold learning in infinite-dimensional spaces, granting it a geometric insight. Extensive numerical experiments show that the MAD method exhibits faster convergence speed without losing accuracy than other deep learning-based methods. The project page with code is available: https://gitee.com/mindspore/mindscience/tree/master/MindElec/.

Figures (15)

• Figure 1: Architecture of Meta-Auto-Decoder.
• Figure 2: Illustration of how MAD works from the manifold learning perspective. (a) MAD-L: The function space $\mathcal{U}$ is mapped to a 2-dimensional plane. The blue solid curve represents the solution set $G(\mathcal{A})$ formed by exact solutions corresponding to all possible PDE parameters, and each point on the curve represents an exact solution corresponding to one PDE parameter. The orange dotted curve represents the solution set $G_{\theta^*}(Z)$ obtained by the pre-trained model, and each point on the curve corresponds to a latent vector $z$. Given $\eta_{\text{new}} \in \mathcal{A}$, rather than searching in the entire function space $\mathcal{U}$, MAD-L only searches on the orange dotted curve to find an optimal $z$ such that its corresponding solution $u_{\theta^*}(\cdot,z)$ is nearest to the blue point $u^{\eta_{\text{new}}}$. (b) MAD-LM: The solution set $G(\mathcal{A})$ lies within a neighborhood of $G_{\theta^*}(Z)$ that is represented by a gray shadow band. To find solution $u^{\eta_{\text{new}}}$, we have to fine-tune $\theta$ (i.e., the orange dotted lines) and the latent vector $z$ (i.e., the points on the orange dotted lines) simultaneously to approach the exact solution. As the search scope is limited to a strip with a small width, the fine-tuning process can be expected to converge quickly.
• Figure 3: Visualization of the MAD pre-training and fine-tuning process for the ODE problem.
• Figure 4: The mean $L_2\ error$ convergence with respect to the number of training iterations.
• Figure 5: The mean $L_2\ error$ convergence with respect to the number of training iterations for extrapolation experiments.

Theorems & Definitions (2)

• Remark 1
• Remark 2