Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural semi-Lagrangian method for high-dimensional advection-diffusion problems

Emmanuel Franck, Victor Michel-Dansac, Laurent Navoret, Vincent Vigon

TL;DR

This work introduces a Neural Semi-Lagrangian (NSL) method for high-dimensional advection-diffusion equations, combining the stability benefits of semi-Lagrangian schemes with the representational power of neural networks. The solution is modeled as a neural network u_θ(t,x) and evolves by solving nonlinear least-squares problems that transport the previous step along backwards characteristics, enabling a mesh-free, GPU-friendly algorithm with no time-step stability constraint. The authors provide a rough error framework decomposing integration, optimization, approximation, and characteristic errors, and demonstrate the method on diverse high-dimensional benchmarks, including level-set deformation and Vlasov-Poisson-like systems, where NSL often outperforms traditional PINNs, dPINNs, and Neural Galerkin methods in accuracy and/or efficiency. The results suggest NSL as a practical tool for large-scale transport problems and motivate further enhancements such as natural gradient preconditioning, adaptive sampling, and stronger boundary-condition handling to broaden applicability to complex kinetic and level-set applications.

Abstract

This work is devoted to the numerical approximation of high-dimensional advection-diffusion equations. It is well-known that classical methods, such as the finite volume method, suffer from the curse of dimensionality, and that their time step is constrained by a stability condition. The semi-Lagrangian method is known to overcome the stability issue, while recent time-discrete neural network-based approaches overcome the curse of dimensionality. In this work, we propose a novel neural semi-Lagrangian method that combines these last two approaches. It relies on projecting the initial condition onto a finite-dimensional neural space, and then solving an optimization problem, involving the backwards characteristic equation, at each time step. It is particularly well-suited for implementation on GPUs, as it is fully parallelizable and does not require a mesh. We provide rough error estimates present several high-dimensional numerical experiments to assess the performance of our approach, and compare it to other neural methods.

Neural semi-Lagrangian method for high-dimensional advection-diffusion problems

TL;DR

This work introduces a Neural Semi-Lagrangian (NSL) method for high-dimensional advection-diffusion equations, combining the stability benefits of semi-Lagrangian schemes with the representational power of neural networks. The solution is modeled as a neural network u_θ(t,x) and evolves by solving nonlinear least-squares problems that transport the previous step along backwards characteristics, enabling a mesh-free, GPU-friendly algorithm with no time-step stability constraint. The authors provide a rough error framework decomposing integration, optimization, approximation, and characteristic errors, and demonstrate the method on diverse high-dimensional benchmarks, including level-set deformation and Vlasov-Poisson-like systems, where NSL often outperforms traditional PINNs, dPINNs, and Neural Galerkin methods in accuracy and/or efficiency. The results suggest NSL as a practical tool for large-scale transport problems and motivate further enhancements such as natural gradient preconditioning, adaptive sampling, and stronger boundary-condition handling to broaden applicability to complex kinetic and level-set applications.

Abstract

This work is devoted to the numerical approximation of high-dimensional advection-diffusion equations. It is well-known that classical methods, such as the finite volume method, suffer from the curse of dimensionality, and that their time step is constrained by a stability condition. The semi-Lagrangian method is known to overcome the stability issue, while recent time-discrete neural network-based approaches overcome the curse of dimensionality. In this work, we propose a novel neural semi-Lagrangian method that combines these last two approaches. It relies on projecting the initial condition onto a finite-dimensional neural space, and then solving an optimization problem, involving the backwards characteristic equation, at each time step. It is particularly well-suited for implementation on GPUs, as it is fully parallelizable and does not require a mesh. We provide rough error estimates present several high-dimensional numerical experiments to assess the performance of our approach, and compare it to other neural methods.
Paper Structure (50 sections, 3 theorems, 117 equations, 20 figures, 14 tables, 3 algorithms)

This paper contains 50 sections, 3 theorems, 117 equations, 20 figures, 14 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Classical and neural numerical methods
  3. Classical methods
  4. Spacetime Galerkin method
  5. Galerkin method
  6. Semi-Lagrangian Galerkin Method
  7. Neural methods
  8. Physics-Informed Neural Networks (PINNs)
  9. Discrete PINNs and the Neural Galerkin method
  10. Discrete PINNs StiCha2023biesek2023burgers
  11. Neural Galerkin method LI2022110958finzi2023stableBRUNA2024112588KAST2024112986
  12. Brief comparison of space-time and time-sequential neural methods
  13. Space-time vs time-sequential methods
  14. Discrete PINNs vs Neural Galerkin
  15. Neural Semi-Lagrangian method
  16. ...and 35 more sections

Key Result

Lemma 2

Let $n \geqslant 0$, and define the function $\mathcal{X}^{n+1}: x \mapsto \mathcal{X}(t^n; t^{n+1}, x)$. Then, for all $f \in V$,

Figures (20)

  • Figure 1: Visual comparison of the classical (left) and neural (right) semi-Lagrangian schemes in one space dimension. In the mesh-based classical scheme, the update is performed on a mesh, and thus an interpolation onto the mesh is needed after computing the foot of the characteristic curve. Conversely, in the meshless neural scheme, the approximation at the previous time step is directly evaluated at the foot of the characteristic curve.
  • Figure 2: 1D constant advection from \ref{['sec:1D_transport_constant_non_parametric']}: comparison of the different methods for several values of the variance $\nu$, and for a fixed time step (${\Delta t} = 0.05$, i.e., $20$ time steps in the dPINN method; ${\Delta t} = 0.25$, i.e., $4$ time steps in the NG and SL methods). From left to right: prediction of the solution at $t=1$, pointwise errors between the predicted solutions and the exact solution at $t=1$, and time evolution of the $L^2$ error.
  • Figure 3: 1D constant advection from \ref{['sec:1D_transport_constant_non_parametric']}: comparison of the different methods for several values of the time step ${\Delta t}$ in the NG and SL methods and a fixed variance, equal to $0.1$. From top to bottom and left to right, we take ${\Delta t} \in \{0.01, 0.02, 0.05, 0.1, 0.2, 0.5\}$, and we display the time evolution of the $L^2$ error $e_t$.
  • Figure 4: 1D parametric advection from \ref{['sec:1D_transport_constant_parametric']}: comparison of the different methods for several values of the parameters and with a fixed time step. Top panels: approximate solutions; bottom panels: absolute values of the errors. From left to right: $\mu = (0.075, 0.55)$, $\mu = (0.1, 0.75)$, and $\mu = (0.125, 0.95)$.
  • Figure 5: 2D parametric rotating advection from \ref{['sec:2D_transport_rotating_parametric']}: from left to right, results of the PINN, NG and NSL schemes for $\mu = (0.06, 0.25)$, with a fixed time step for NG (${\Delta t} = 0.02$) and NSL (${\Delta t} = 0.5$). Top panels: approximate solutions; bottom panels: absolute value of the pointwise error.
  • ...and 15 more figures

Theorems & Definitions (9)

  • Remark 1
  • Lemma 2
  • proof
  • Proposition 3
  • proof
  • Corollary 4
  • proof
  • Remark 5
  • Remark 6