Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

GEPS: Boosting Generalization in Parametric PDE Neural Solvers through Adaptive Conditioning

Armand Kassaï Koupaï, Jorge Mifsut Benet, Yuan Yin, Jean-Noël Vittaut, Patrick Gallinari

TL;DR

GEPS is proposed, a simple adaptation mechanism to boost GEneralization in Pde Solvers via a first-order optimization and low-rank rapid adaptation of a small set of context parameters.

Abstract

Solving parametric partial differential equations (PDEs) presents significant challenges for data-driven methods due to the sensitivity of spatio-temporal dynamics to variations in PDE parameters. Machine learning approaches often struggle to capture this variability. To address this, data-driven approaches learn parametric PDEs by sampling a very large variety of trajectories with varying PDE parameters. We first show that incorporating conditioning mechanisms for learning parametric PDEs is essential and that among them, $\textit{adaptive conditioning}$, allows stronger generalization. As existing adaptive conditioning methods do not scale well with respect to the number of parameters to adapt in the neural solver, we propose GEPS, a simple adaptation mechanism to boost GEneralization in Pde Solvers via a first-order optimization and low-rank rapid adaptation of a small set of context parameters. We demonstrate the versatility of our approach for both fully data-driven and for physics-aware neural solvers. Validation performed on a whole range of spatio-temporal forecasting problems demonstrates excellent performance for generalizing to unseen conditions including initial conditions, PDE coefficients, forcing terms and solution domain. $\textit{Project page}$: https://geps-project.github.io

GEPS: Boosting Generalization in Parametric PDE Neural Solvers through Adaptive Conditioning

TL;DR

GEPS is proposed, a simple adaptation mechanism to boost GEneralization in Pde Solvers via a first-order optimization and low-rank rapid adaptation of a small set of context parameters.

Abstract

Solving parametric partial differential equations (PDEs) presents significant challenges for data-driven methods due to the sensitivity of spatio-temporal dynamics to variations in PDE parameters. Machine learning approaches often struggle to capture this variability. To address this, data-driven approaches learn parametric PDEs by sampling a very large variety of trajectories with varying PDE parameters. We first show that incorporating conditioning mechanisms for learning parametric PDEs is essential and that among them, , allows stronger generalization. As existing adaptive conditioning methods do not scale well with respect to the number of parameters to adapt in the neural solver, we propose GEPS, a simple adaptation mechanism to boost GEneralization in Pde Solvers via a first-order optimization and low-rank rapid adaptation of a small set of context parameters. We demonstrate the versatility of our approach for both fully data-driven and for physics-aware neural solvers. Validation performed on a whole range of spatio-temporal forecasting problems demonstrates excellent performance for generalizing to unseen conditions including initial conditions, PDE coefficients, forcing terms and solution domain. : https://geps-project.github.io

Paper Structure

This paper contains 66 sections, 19 equations, 18 figures, 12 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Description
  3. Motivations: ERM versus adaptive approaches for parametric PDEs
  4. In-distribution generalization for IVP: classical vs. adaptive conditioning approaches
  5. Scaling w.r.t. the number of environments.
  6. Out-of-distribution generalization to new environments for IVP: classical vs. adaptive conditioning approaches
  7. In and out-of-distribution generalization performance with temporal conditioning
  8. GEPS method
  9. Adaptation rule
  10. Formulation
  11. Two-step training procedure
  12. Hybrid formulation for learning dynamics
  13. Experiments
  14. Dynamical systems
  15. Evaluation Setting
  16. ...and 51 more sections

Figures (18)

  • Figure 1: Multi-environment setup for the Kolmogorov PDE. The model is trained on multiple environments with several trajectories per environment (left). At inference, for a new unseen environment it is adapted on one trajectory (right).
  • Figure 2: Comparison of ERM approaches (shades of blue) and Poseidon foundation model (green) with our framework GEPS (red) when increasing the number of training environments.
  • Figure 3: Comparison of ERM approaches (shades of blue) and Poseidon (green) with our framework GEPS (red) when increasing the number of trajectories per environment.
  • Figure 4: Out-distribution generalization on 4 new environments using one trajectory per environment for fine-tuning or adaptation. Models have either been pretrained on 4 environments (left column) or 1024 environments (right columns). Metric is Relative L2 loss.
  • Figure 5: Our adaptation framework for our data-driven model. Block in blue refers to the data-driven module $\mathcal{G}_a$. Blocks $L_i$ in pink refer to the trainable modules. The green block describes the adaptation mechanism for the data-driven component, with $\bm{W}_{L_i}$ the weights of layer $L_i$. Context vector ${\bm{c}}^e$ conditions all the layers $\bm{W}_{L_i}$.
  • ...and 13 more figures