Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Benefits of Active Data Collection in Operator Learning

Unique Subedi, Ambuj Tewari

TL;DR

The paper addresses data efficiency in learning PDE solution operators by introducing active data collection when inputs are drawn from mean-zero processes with covariance kernel $K$. It proposes sampling along the covariance operator’s eigenfunctions and a linear estimator $\widehat{\mathcal{F}}_n=\sum_{i=1}^n w_i\otimes \varphi_i$, achieving a risk bound $\mathbb{E}_{v\sim\mu}\|\widehat{\mathcal{F}}_n(v)-\mathcal{F}(v)\|_{L^2}^2 \le \varepsilon^2\sum_{i=1}^n \lambda_i + \|\mathcal{F}\|_{op}^2 \sum_{i>n} \lambda_i$, with $\lambda_i$ as the eigenvalues of the covariance operator. By tailoring the eigenvalue decay of $K$, arbitrarily fast convergence rates can be obtained, in stark contrast to passive i.i.d. data whose risk cannot vanish faster than linear in $n$; a minimax lower bound confirms the advantage of active data collection. The approach leverages the Karhunen–Loève decomposition and is demonstrated on covariance kernels including fractional Laplacian inverses, RBF, and Brownian motion, with experiments on Poisson and heat equations confirming data-efficiency gains. The work suggests natural extensions to nonlinear operators and operator RKHS frameworks, along with connections to PCANet-like approaches for even faster rates in practice.

Abstract

We study active data collection strategies for operator learning when the target operator is linear and the input functions are drawn from a mean-zero stochastic process with continuous covariance kernels. With an active data collection strategy, we establish an error convergence rate in terms of the decay rate of the eigenvalues of the covariance kernel. We can achieve arbitrarily fast error convergence rates with sufficiently rapid eigenvalue decay of the covariance kernels. This contrasts with the passive (i.i.d.) data collection strategies, where the convergence rate is never faster than linear decay ($\sim n^{-1}$). In fact, for our setting, we show a \emph{non-vanishing} lower bound for any passive data collection strategy, regardless of the eigenvalues decay rate of the covariance kernel. Overall, our results show the benefit of active data collection strategies in operator learning over their passive counterparts.

On the Benefits of Active Data Collection in Operator Learning

TL;DR

The paper addresses data efficiency in learning PDE solution operators by introducing active data collection when inputs are drawn from mean-zero processes with covariance kernel . It proposes sampling along the covariance operator’s eigenfunctions and a linear estimator , achieving a risk bound , with as the eigenvalues of the covariance operator. By tailoring the eigenvalue decay of , arbitrarily fast convergence rates can be obtained, in stark contrast to passive i.i.d. data whose risk cannot vanish faster than linear in ; a minimax lower bound confirms the advantage of active data collection. The approach leverages the Karhunen–Loève decomposition and is demonstrated on covariance kernels including fractional Laplacian inverses, RBF, and Brownian motion, with experiments on Poisson and heat equations confirming data-efficiency gains. The work suggests natural extensions to nonlinear operators and operator RKHS frameworks, along with connections to PCANet-like approaches for even faster rates in practice.

Abstract

We study active data collection strategies for operator learning when the target operator is linear and the input functions are drawn from a mean-zero stochastic process with continuous covariance kernels. With an active data collection strategy, we establish an error convergence rate in terms of the decay rate of the eigenvalues of the covariance kernel. We can achieve arbitrarily fast error convergence rates with sufficiently rapid eigenvalue decay of the covariance kernels. This contrasts with the passive (i.i.d.) data collection strategies, where the convergence rate is never faster than linear decay (). In fact, for our setting, we show a \emph{non-vanishing} lower bound for any passive data collection strategy, regardless of the eigenvalues decay rate of the covariance kernel. Overall, our results show the benefit of active data collection strategies in operator learning over their passive counterparts.

Paper Structure

This paper contains 38 sections, 3 theorems, 128 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Our Contribution
  3. Related Works
  4. Preliminaries
  5. Notation
  6. Distribution Over Function Space
  7. Problem Setting and Goal
  8. Upper Bounds Under Active Data Collection
  9. Data Collection Strategy and The Estimator
  10. Sketch of a Proof of Theorem \ref{['thm:upper']}
  11. Examples of Covariance Kernels
  12. Fractional Inverse of Shifted Laplacian
  13. RBF Kernel
  14. Brownian Motion
  15. Comparison to Traditional Active Learning
  16. ...and 23 more sections

Key Result

Theorem 1

Suppose $\mathcal{F}$ is a bounded linear operator. There exists a deterministic data collection strategy and a deterministic estimation rule such that the estimate $\widehat{\mathcal{F}}_n$ produced after $n$ calls to oracle $\mathcal{O}$ satisfies Here, $\lambda_1\geq \lambda_2 \geq \ldots$ are the eigenvalues of the integral operator of $K$ defined as $(\mathcal{I}_K v)(\cdot)= \int_{\mathcal{

Figures (7)

  • Figure 1: Error Plots for various estimators for Poisson Equation. The blue curve shows the performance of our linear estimator on actively collected data. The orange and red curves include the linear estimator's and FNO's performance on passively collected data. Figure \ref{['fig:Poisson_error_log']} shows the same plot in $\log$-scale.
  • Figure 2: Error Plots for Poisson Equation in $\log$-$\log$ scale.
  • Figure 3: Convergence rate of the active linear estimator for Poisson equation with actively collected data for different values of $\gamma$.
  • Figure 4: Error Plots for the Heat equation in $\log$-scale.
  • Figure 5: Error Plots for various estimators for Poisson Equation. The plot on the right shows the same plot in log scale.
  • ...and 2 more figures

Theorems & Definitions (6)

  • Theorem 1: Upper Bound
  • Definition 1: Perfect Oracle
  • Theorem 2: Lowerbound
  • Proposition 1
  • proof : Proof of Proposition \ref{['prop:eigenpair']}
  • proof