An approach to discrete operator learning based on sparse high-dimensional approximation

TL;DR

The paper addresses learning solution operators for PDEs in high-dimensional parameter spaces by using a dimension-incremental sparse approximation in bounded orthonormal product bases. It represents the solution as $u({\boldsymbol{x}},{\boldsymbol{a}}) \approx \sum_{\boldsymbol{k} \in I} \hat{u}_{\boldsymbol{k}} \Phi_{\boldsymbol{k}}({\boldsymbol{x}},{\boldsymbol{a}})$ and adaptively detects a sparse index set $I$ from PDE samples, enabling efficient and interpretable high-dimensional approximations. The contributions include (i) a rigorous error-bounding framework combining truncation and coefficient errors, (ii) a dimension-incremental algorithm to identify $I$ using black-box samples and cubature rules, and (iii) extensive numerical demonstrations on 1D and multi-D problems (Poisson, diffusion with random coefficients, heat, Burgers) showing meaningful index structures and competitive accuracy. The approach offers a non-intrusive, interpretable alternative to neural operator methods, with potential to scale to even higher dimensions and more complex parametric dependencies, and comes with open-source code for reproduction.

Abstract

We present a dimension-incremental method for function approximation in bounded orthonormal product bases to learn the solutions of various differential equations. Therefore, we decompose the source function of the differential equation into parameters like Fourier or Spline coefficients and treat the solution of the differential equation as a high-dimensional function w.r.t. the spatial variables, these parameters and also further possible parameters from the differential equation itself. Finally, we learn this function in the sense of sparse approximation in a suitable function space by detecting coefficients of the basis expansion with the largest absolute values. Investigating the corresponding indices of the basis coefficients yields further insights on the structure of the solution as well as its dependency on the parameters and their interactions and allows for a reasonable generalization to even higher dimensions and therefore better resolutions of the decomposed source function.

Figures (14)

• Figure 3.1: The relative approximation error $\text{err}({\boldsymbol{a}})$ for $10000$ randomly drawn ${\boldsymbol{a}}$ when using the Fourier series parametrization. The box-and-whisker plots show the median, the first and the second quartile as well as the maximal and minimal error observed. The five plots indicate different choices for the range of the Fourier coefficient ${\boldsymbol{a}}$, including two cases with complex-valued Fourier coefficients. The range $[-1,1]$ coincides with the training data used.
• Figure 3.2: An abstract visualization of the first 40 indices ${\boldsymbol{k}}$ detected when using the Fourier series parametrization. The leftmost column contains the index ${\boldsymbol{k}}$ corresponding to the largest (in absolute value) basis coefficient $\hat{u}_{{\boldsymbol{k}}}$, the second column the index for the second largest and so on. The rows identify the $10$ dimensions corresponding to the variables $x$ and $a_{-4},\ldots ,a_{4}$ from top to bottom in this order. Zeros are neglected to preserve clarity.
• Figure 3.3: The relative approximation error $\text{err}({\boldsymbol{a}})$ for $10000$ randomly drawn ${\boldsymbol{a}}$ for different choices of the spline order $m$ and the relative tolerance of the solver function solve_ bvp. The box-and-whisker plots show the median, the first and the second quartile as well as the maximal and minimal error observed.
• Figure 3.4: Abstract visualizations of $40$ detected indices ${\boldsymbol{k}}$ (from left to right) for Example \ref{['subsubsec:poisson_1d_bs']}. The indices ${\boldsymbol{k}}$ are sorted in descending order according to the size of the corresponding approximated coefficient $\hat{u}_{{\boldsymbol{k}}}$. The rows identify the $10$ dimensions corresponding to the variables $x$ and $a_{-4},\ldots ,a_{4}$ from top to bottom in this order. Zeros are neglected to preserve clarity.
• Figure 3.5: The (absolute) pointwise approximation error of our approximation when using the right-hand side \ref{['eq:pwc_f']} compared to the exact solution \ref{['eq:pwc_ex']}.

Theorems & Definitions (5)

• Remark 2.1
• Remark 2.2
• Remark 3.1
• Example 3.2: High-dimensional extension of the detected index set $I$
• Remark 3.3