Towards Gaussian Process for operator learning: an uncertainty aware resolution independent operator learning algorithm for computational mechanics

TL;DR

A novel Gaussian Process (GP) based neural operator based neural operator for solving parametric differential equations that leverages the expressive capability of deterministic neural operators and the uncertainty awareness of conventional GP to be positioned as a scalable and reliable operator-learning algorithm for computational mechanics.

Abstract

The growing demand for accurate, efficient, and scalable solutions in computational mechanics highlights the need for advanced operator learning algorithms that can efficiently handle large datasets while providing reliable uncertainty quantification. This paper introduces a novel Gaussian Process (GP) based neural operator for solving parametric differential equations. The approach proposed leverages the expressive capability of deterministic neural operators and the uncertainty awareness of conventional GP. In particular, we propose a ``neural operator-embedded kernel'' wherein the GP kernel is formulated in the latent space learned using a neural operator. Further, we exploit a stochastic dual descent (SDD) algorithm for simultaneously training the neural operator parameters and the GP hyperparameters. Our approach addresses the (a) resolution dependence and (b) cubic complexity of traditional GP models, allowing for input-resolution independence and scalability in high-dimensional and non-linear parametric systems, such as those encountered in computational mechanics. We apply our method to a range of non-linear parametric partial differential equations (PDEs) and demonstrate its superiority in both computational efficiency and accuracy compared to standard GP models and wavelet neural operators. Our experimental results highlight the efficacy of this framework in solving complex PDEs while maintaining robustness in uncertainty estimation, positioning it as a scalable and reliable operator-learning algorithm for computational mechanics.

Figures (11)

• Figure 1: Schematics of Gaussian Process operator. The architecture of the proposed framework is depicted in the above schematic diagram, which provides an overview of the framework. The schematic illustrates the transformation of inputs in the latent space through the utilization of a WNO-embedded kernel. Following this, stochastic dual descent (SDD) optimization is utilized for the training of the framework. The proposed approach transforms the training data via the WNO-embedded kernel thus improving the learning capabilities of the vanilla kernels, and utilizes SDD for efficient optimization. The posterior mean and pathwise sampling steps provide the final predictions with the associated predictive uncertainty.
• Figure 2: 1D Burger: Figure shows the prediction obtained from the proposed framework for three of the representative test inputs. The first row shows one of the three representative test inputs (initial conditions) from the test dataset, while the second row shows the corresponding ground truth and the mean prediction obtained from our proposed framework at time $t=1$ along with a 95 % confidence interval.
• Figure 3: 1D Burger:Figure shows the prediction obtained from the proposed framework for three of the representative test inputs. The first row shows one of the three representative test inputs on a spatial resolution of 1024 which is higher than the training input resolution of 512. The second row shows the corresponding ground truth and the mean prediction obtained from our proposed framework along with a 95 % confidence interval.
• Figure 4: 1D Burger. Figure (a) shows the effect of initialization on the performance of the proposed model. We vary the number of samples $S_{\text{init}}$ used for initializing the model while keeping overall training samples $S_{\text{SDD}} = 1100$. Figure (b) illustrates the effect of the number of training samples $S_{\text{SDD}}$ on the performance of the proposed approach. For all the cases, the model is initialized with $S_{\text{init}}=500$.
• Figure 5: 1D Wave advection. Figure illustrates the results obtained from our proposed framework for three of the representative test samples. The first row shows three representative test samples (initial condition) from the test dataset, while the second row shows the corresponding ground truth and the mean prediction with a 95 % Confidence interval.