Physics informed cell representations for variational formulation of multiscale problems

TL;DR

The paper tackles the challenge of solving multiscale PDEs with PINNs by introducing physics-informed cell representations that couple multilevel, multiresolution grids with an MLP. A variational (weak-form) loss, together with decoupled Dirichlet training and periodic boundary parameter sharing, yields faster convergence and higher accuracy than conventional PINNs, especially for high-frequency and nonlinear boundary conditions. Spectral normalization and GPU-accelerated bilinear interpolation further stabilize and accelerate training, while extensive hyperparameter studies provide practical guidelines. The approach is demonstrated on several Poisson problems with multiscale coefficients and boundary conditions, showing substantial accuracy gains over strong/weak PINNs and competitive performance with FEM baselines. Overall, the cell-based MLP framework offers a scalable, architecture-driven method to resolve multiscale PDE features with improved efficiency and accuracy, along with actionable strategies for boundary enforcement and hyperparameter tuning.

Abstract

With the rapid advancement of graphical processing units, Physics-Informed Neural Networks (PINNs) are emerging as a promising tool for solving partial differential equations (PDEs). However, PINNs are not well suited for solving PDEs with multiscale features, particularly suffering from slow convergence and poor accuracy. To address this limitation of PINNs, this article proposes physics-informed cell representations for resolving multiscale Poisson problems using a model architecture consisting of multilevel multiresolution grids coupled with a multilayer perceptron (MLP). The grid parameters (i.e., the level-dependent feature vectors) and the MLP parameters (i.e., the weights and biases) are determined using gradient-descent based optimization. The variational (weak) form based loss function accelerates computation by allowing the linear interpolation of feature vectors within grid cells. This cell-based MLP model also facilitates the use of a decoupled training scheme for Dirichlet boundary conditions and a parameter-sharing scheme for periodic boundary conditions, delivering superior accuracy compared to conventional PINNs. Furthermore, the numerical examples highlight improved speed and accuracy in solving PDEs with nonlinear or high-frequency boundary conditions and provide insights into hyperparameter selection. In essence, by cell-based MLP model along with the parallel tiny-cuda-nn library, our implementation improves convergence speed and numerical accuracy.

Figures (11)

• Figure 1: The model architecture of the multiresolution cell representations combined with MLPs, where the circles in the figure indicate trainable parameters. (a) The solution domain is divided into several levels of multiresolution grids with each grid further divided into cells and feature vectors (parameters) defined at the grid nodes. Given a sample point location, the model can determine the corresponding cell at all levels that contain the point; (b) In the cells containing the point at each level grid, the model then calculates the corresponding feature vector by linear interpolation from the nodal feature vectors; (c) The concatenated feature vector is passed to the MLP; (d) The MLP with its own trainable parameters predicts the solution variable by taking in the feature vector as input.
• Figure 2: The decoupled training scheme: (a) In the first training phase, we disregard the PDE loss and solely focus on optimizing the model for boundary conditions by sampling the domain boundary. (b) In the second training phase, we freeze the parameters on the Dirichlet boundaries and MLPs (i.e., gray circles in the figure) and optimize the interior parameters with the PDE loss by sampling the interior domain.
• Figure 3: Schematic decription of the parameter sharing scheme. By sharing the nodal parameters on the grid that corresponds to the periodic boundary, the model's output always satisfies the appropriate conditions.
• Figure 4: Model performance evaluation for the Poisson equation with non-constant Dirichlet BC. (a) Solution of cell-based MLP with decoupled training scheme, analytical solution, and the difference between them; (b) The relative error (NRMSE) of the final solutions for different weight factor $\lambda$ in loss function for different models; (c) The relative error (NRMSE) during the training process for different models.
• Figure 5: Model performance evaluation and equation properties for Poisson equation with high-frequency variable coefficient: (a) Solution of cell-based MLP with decoupled training scheme, FEM solution, and the difference between them. The permeability field $a(x,y)$ with (b) $\varepsilon$ = 0.125 and (c) $\varepsilon$ = 1. (d) The relative error (NRMSE) during the training process for different models.