Equation Discovery with Bayesian Spike-and-Slab Priors and Efficient Kernels

TL;DR

This work addresses data-sparse, noisy equation discovery for ODEs/PDEs by introducing KBASS, a kernel-based method that couples RKHS regression with a Bayesian spike-and-slab prior for sparse operator selection. An EP-EM algorithm, combined with a mesh-based Kronecker structure, enables scalable posterior inference over operator weights and function estimates. The approach yields principled uncertainty quantification for discovered equations and demonstrates robustness to noise and limited data across diverse benchmark PDEs/ODEs, outperforming SINDy, PINN-SR, and BSL in many settings. The practical impact lies in more reliable, data-efficient discovery of governing laws with quantified confidence, suitable for scientific and engineering applications.

Abstract

Discovering governing equations from data is important to many scientific and engineering applications. Despite promising successes, existing methods are still challenged by data sparsity and noise issues, both of which are ubiquitous in practice. Moreover, state-of-the-art methods lack uncertainty quantification and/or are costly in training. To overcome these limitations, we propose a novel equation discovery method based on Kernel learning and BAyesian Spike-and-Slab priors (KBASS). We use kernel regression to estimate the target function, which is flexible, expressive, and more robust to data sparsity and noises. We combine it with a Bayesian spike-and-slab prior -- an ideal Bayesian sparse distribution -- for effective operator selection and uncertainty quantification. We develop an expectation-propagation expectation-maximization (EP-EM) algorithm for efficient posterior inference and function estimation. To overcome the computational challenge of kernel regression, we place the function values on a mesh and induce a Kronecker product construction, and we use tensor algebra to enable efficient computation and optimization. We show the advantages of KBASS on a list of benchmark ODE and PDE discovery tasks.

Figures (8)

• Figure 1: Solution and weight posterior estimation for the VDP equation with $10$ training examples (marked as green); (c) and (d) show the weight posterior for terms $y$ and $x$, respectively. Their posterior selection probabilities were both estimated as $1.0$.
• Figure 2: Solution and weight posterior estimation for Lorenz 96 using 12 training examples; (c) and (d) show the weight posterior for the force term $F$ and $x_5$. The posterior selection probabilities were estimated as 1.0.
• Figure 3: Solution estimate for Burger's equation with $\nu = 0.005$ with 20% noise on the training data.
• Figure 4: Solution and weight posterior estimation on Allen-cahn equation; $w(\cdot)$ and $s(\cdot)$ denote the weight and selection indicator of the operator. Note the ground-truth weight for $u_{xx}$ is $10^{-4}$.
• Figure 5: Solution estimate for the KS equation with 20% noise on training data.