Leveraging Interpolation Models and Error Bounds for Verifiable Scientific Machine Learning

TL;DR

The paper tackles verifiability in scientific machine learning by leveraging mathematically grounded interpolation error bounds across multiple interpolants (Delaunay, RBF/TPS, Gaussian processes) and by embedding high-dimensional inputs into latent spaces learned by deep models. It proposes a best-of-both-worlds workflow where interpolation bounds provide rigorous error estimates and, when combined with latent-space representations, yield interpretable diagnostics and validation for black-box models. Through empirical studies on synthetic functions and a case study predicting lift-drag ratios from airfoil images, the authors derive practical guidance on method choice, bound validity, and the impact of extrapolation and embedding quality. The work demonstrates that interpolants can match or closely approach neural network performance while offering explicit error bounds, interpretability, and a reproducible framework for verifiable scientific ML pipelines, with open-source code to foster adoption.

Abstract

Effective verification and validation techniques for modern scientific machine learning workflows are challenging to devise. Statistical methods are abundant and easily deployed, but often rely on speculative assumptions about the data and methods involved. Error bounds for classical interpolation techniques can provide mathematically rigorous estimates of accuracy, but often are difficult or impractical to determine computationally. In this work, we present a best-of-both-worlds approach to verifiable scientific machine learning by demonstrating that (1) multiple standard interpolation techniques have informative error bounds that can be computed or estimated efficiently; (2) comparative performance among distinct interpolants can aid in validation goals; (3) deploying interpolation methods on latent spaces generated by deep learning techniques enables some interpretability for black-box models. We present a detailed case study of our approach for predicting lift-drag ratios from airfoil images. Code developed for this work is available in a public Github repository.

Figures (17)

• Figure 1: For $d=5$ fixed and increasing values of $n$, error bounds and actual error are shown in four regimes: interpolation (top) vs. extrapolation (bottom) and low variation (left) vs. high variation (right) in the response function. Of note, the GP bound is orders of magnitude below the actual error in the high variation case.
• Figure 2: For $n=8192$ fixed and increasing values of $d$, error bounds and actual error are shown in four regimes: interpolation (top) vs. extrapolation (bottom) and low variation (left) vs. high variation (right) in the response function. Crossover between the bounds and error as dimension increases is indicative of the exponentially decreasing density for a fixed sample size.
• Figure 3: Delaunay interpolation is not affected by skew in the input data or rescaling, unlike the ReLU MLP approximant. The Delaunay bound is affected by skew, but rescaling helps. The left column here is the same as the left column of Figure \ref{['fig:variation']}a.
• Figure 4: GP and RBF interpolants are affected by skew in the input data and rescaling. If the GP bound does not decrease as the number of samples increases, it may be a sign of skew in the data. The left column here is the same as the left column of Figure \ref{['fig:variation']}b.
• Figure 5: Given $n$ (rows) and $d$ (columns), we create a Sobol sequence data set with $n$ points in $[-1,1]^d$. For 1024 points drawn uniformly from $[-1,1]^d$, we show a histogram of the the $L^2$ norm of each point, colored by whether the point is interpolation (blue) or extrapolation (red) relative to the data set. Notice that extrapolation always dominates as dimension increases, due to the curse of dimensionality.