Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Accelerating Data Generation for Neural Operators via Krylov Subspace Recycling

Hong Wang, Zhongkai Hao, Jie Wang, Zijie Geng, Zhen Wang, Bin Li, Feng Wu

TL;DR

Sorting Krylov Recycling (SKR) tackles the data-generation bottleneck in neural operator training by sequencing related linear systems and reusing Krylov subspaces through GCRO-DR. The approach combines a parameter-based sorting step with deflated restarted recycling to reduce the Krylov subspace dimension across a sequence of PDE-derived solves. Theoretical analysis shows robustness to perturbations and practical sorting sufficiency, while experiments across Darcy, thermal, Poisson, and Helmholtz problems demonstrate speedups up to 13.9x and substantial iteration reductions. This acceleration directly lowers the data-generation cost for neural operators, enabling scalable training for forward and inverse PDE tasks.

Abstract

Learning neural operators for solving partial differential equations (PDEs) has attracted great attention due to its high inference efficiency. However, training such operators requires generating a substantial amount of labeled data, i.e., PDE problems together with their solutions. The data generation process is exceptionally time-consuming, as it involves solving numerous systems of linear equations to obtain numerical solutions to the PDEs. Many existing methods solve these systems independently without considering their inherent similarities, resulting in extremely redundant computations. To tackle this problem, we propose a novel method, namely Sorting Krylov Recycling (SKR), to boost the efficiency of solving these systems, thus significantly accelerating data generation for neural operators training. To the best of our knowledge, SKR is the first attempt to address the time-consuming nature of data generation for learning neural operators. The working horse of SKR is Krylov subspace recycling, a powerful technique for solving a series of interrelated systems by leveraging their inherent similarities. Specifically, SKR employs a sorting algorithm to arrange these systems in a sequence, where adjacent systems exhibit high similarities. Then it equips a solver with Krylov subspace recycling to solve the systems sequentially instead of independently, thus effectively enhancing the solving efficiency. Both theoretical analysis and extensive experiments demonstrate that SKR can significantly accelerate neural operator data generation, achieving a remarkable speedup of up to 13.9 times.

Accelerating Data Generation for Neural Operators via Krylov Subspace Recycling

TL;DR

Sorting Krylov Recycling (SKR) tackles the data-generation bottleneck in neural operator training by sequencing related linear systems and reusing Krylov subspaces through GCRO-DR. The approach combines a parameter-based sorting step with deflated restarted recycling to reduce the Krylov subspace dimension across a sequence of PDE-derived solves. Theoretical analysis shows robustness to perturbations and practical sorting sufficiency, while experiments across Darcy, thermal, Poisson, and Helmholtz problems demonstrate speedups up to 13.9x and substantial iteration reductions. This acceleration directly lowers the data-generation cost for neural operators, enabling scalable training for forward and inverse PDE tasks.

Abstract

Learning neural operators for solving partial differential equations (PDEs) has attracted great attention due to its high inference efficiency. However, training such operators requires generating a substantial amount of labeled data, i.e., PDE problems together with their solutions. The data generation process is exceptionally time-consuming, as it involves solving numerous systems of linear equations to obtain numerical solutions to the PDEs. Many existing methods solve these systems independently without considering their inherent similarities, resulting in extremely redundant computations. To tackle this problem, we propose a novel method, namely Sorting Krylov Recycling (SKR), to boost the efficiency of solving these systems, thus significantly accelerating data generation for neural operators training. To the best of our knowledge, SKR is the first attempt to address the time-consuming nature of data generation for learning neural operators. The working horse of SKR is Krylov subspace recycling, a powerful technique for solving a series of interrelated systems by leveraging their inherent similarities. Specifically, SKR employs a sorting algorithm to arrange these systems in a sequence, where adjacent systems exhibit high similarities. Then it equips a solver with Krylov subspace recycling to solve the systems sequentially instead of independently, thus effectively enhancing the solving efficiency. Both theoretical analysis and extensive experiments demonstrate that SKR can significantly accelerate neural operator data generation, achieving a remarkable speedup of up to 13.9 times.
Paper Structure (46 sections, 1 theorem, 18 equations, 14 figures, 32 tables)

This paper contains 46 sections, 1 theorem, 18 equations, 14 figures, 32 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Data-Efficient Neural Operators and Learned PDE Solvers
  4. Krylov Subspace Recycling
  5. Preliminaries
  6. Discretization of PDEs in Neural Operator Training
  7. Krylov Subspace Method
  8. Method
  9. The Sorting Algorithm
  10. Krylov Subspace Recycling
  11. Theoretical Analysis
  12. Convergence Analysis
  13. The Rationality of Sorting Algorithms
  14. Experimental
  15. Set up
  16. ...and 31 more sections

Key Result

Theorem 1

Given a space ${\mathcal{C}= {\rm range}({\bm{C}}_k)}$, let ${\mathcal{V}} ={\rm range} ({\bm{V}}_{m-k+1}\underline{{\bm{H}}}_{m-k})$ be the $(m-k)$ dimensional Krylov subspace generated by GCRO-DR as in (eq:GCRO-DR Arnoldi). Let ${\bm{r}}_0 \in \mathds{C}^n$, and let ${\bm{r}}_1 = ({\bm{I}}-{\bm{\v where $\gamma = \Vert ({\bm{I}} -{\bm{\varPi}}_{{\mathcal{C}}}) {\bm{P}}_{{\mathcal{Q}}}\Vert_2$.

Figures (14)

  • Figure 1: Left. Generation process of the NO dataset. 1. Generate a set of random parameters from NO. 2. Export the corresponding PDE based on these parameters. 3. Transform the PDE into a system of linear equations using discretization methods. 4. Invoke linear equation solvers to solve these systems independently. 5. Obtain solutions to the linear systems and convert them into PDE solutions. 6. Assemble into a dataset. Right. The accuracy change over time for our SKR algorithm and the baseline GMRES algorithm. As seen, the SKR algorithm can significantly accelerate the solution of the system of linear equations, achieving a speed-up of up to 13.9 times.
  • Figure 2: Algorithm Flow Diagram: a. Derive the PDEs to be solved from the NO. b. Transform these PDEs into a system of linear equations. c. Apply the SKR algorithm to sort the linear systems, obtaining a sequence with strong correlations. d. Traditional algorithms independently solve the linear systems from step b using Krylov methods. d1, d2, d3. The SKR algorithm utilizes 'recycling' to sequentially solve the linear systems, reducing the dimension of the Krylov subspace. e. Obtain the solutions and assemble them into a dataset.
  • Figure 3: The Sorting Algorithm
  • Figure 4: FEM mesh for the Poisson equation
  • Figure 5: Solutions of Darcy flow equations with close parameters
  • ...and 9 more figures

Theorems & Definitions (1)

  • Theorem 1