Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hierarchical deep learning-based adaptive time-stepping scheme for multiscale simulations

Asif Hamid, Danish Rafiq, Shahkar Ahmad Nahvi, Mohammad Abid Bazaz

TL;DR

Multiscale dynamical systems pose challenges for closures and Taylor-based time stepping. The authors introduce Adaptive Hierarchical Time-Stepping (AHiTS), a data-driven framework that learns a hierarchy of ResNet-based neural network time steppers (NNTS) at different timescales and adaptively selects them during integration using a tolerance $ε$. Across canonical ODEs and PDEs, including the FitzHugh–Nagumo and Kuramoto–Sivashinsky equations, AHiTS matches or surpasses the accuracy of the state-of-the-art multiscale HiTS while significantly reducing the number of steps and computational cost, with added robustness to noise. The approach enables scalable, data-driven multiscale simulation and can be extended with hybrids and encoder–decoder schemes for high-dimensional problems, accompanied by open-source code for training and testing.

Abstract

Multiscale is a hallmark feature of complex nonlinear systems. While the simulation using the classical numerical methods is restricted by the local \textit{Taylor} series constraints, the multiscale techniques are often limited by finding heuristic closures. This study proposes a new method for simulating multiscale problems using deep neural networks. By leveraging the hierarchical learning of neural network time steppers, the method adapts time steps to approximate dynamical system flow maps across timescales. This approach achieves state-of-the-art performance in less computational time compared to fixed-step neural network solvers. The proposed method is demonstrated on several nonlinear dynamical systems, and source codes are provided for implementation. This method has the potential to benefit multiscale analysis of complex systems and encourage further investigation in this area.

Hierarchical deep learning-based adaptive time-stepping scheme for multiscale simulations

TL;DR

Multiscale dynamical systems pose challenges for closures and Taylor-based time stepping. The authors introduce Adaptive Hierarchical Time-Stepping (AHiTS), a data-driven framework that learns a hierarchy of ResNet-based neural network time steppers (NNTS) at different timescales and adaptively selects them during integration using a tolerance . Across canonical ODEs and PDEs, including the FitzHugh–Nagumo and Kuramoto–Sivashinsky equations, AHiTS matches or surpasses the accuracy of the state-of-the-art multiscale HiTS while significantly reducing the number of steps and computational cost, with added robustness to noise. The approach enables scalable, data-driven multiscale simulation and can be extended with hybrids and encoder–decoder schemes for high-dimensional problems, accompanied by open-source code for training and testing.

Abstract

Multiscale is a hallmark feature of complex nonlinear systems. While the simulation using the classical numerical methods is restricted by the local \textit{Taylor} series constraints, the multiscale techniques are often limited by finding heuristic closures. This study proposes a new method for simulating multiscale problems using deep neural networks. By leveraging the hierarchical learning of neural network time steppers, the method adapts time steps to approximate dynamical system flow maps across timescales. This approach achieves state-of-the-art performance in less computational time compared to fixed-step neural network solvers. The proposed method is demonstrated on several nonlinear dynamical systems, and source codes are provided for implementation. This method has the potential to benefit multiscale analysis of complex systems and encourage further investigation in this area.
Paper Structure (22 sections, 8 equations, 6 figures, 10 tables, 2 algorithms)

This paper contains 22 sections, 8 equations, 6 figures, 10 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Hierarchical deep learning using residual neural networks
  3. Flow map learning via ResNets
  4. Coupling multiple ResNets for multiscale systems
  5. Deep learning based adaptive time stepping scheme
  6. Data collection
  7. Training NNTS models at different timescales
  8. Adaptive time-stepping estimation
  9. Iterative vectorized computations
  10. Prediction
  11. Numerical Simulations
  12. Benchmark ODEs
  13. Benchmark PDEs
  14. FitzHugh-Nagumo model
  15. Kuramoto-Sivashinsky equation
  16. ...and 7 more sections

Figures (6)

  • Figure 1: Schematic for one step approximation using ResNet
  • Figure 2: Coupling various ResNets for multiscale simulations (adapted from Ref. liu2022hierarchical)
  • Figure 3: Block diagram of AHiTS algorithm: (Step I) Collection of data at different step sizes, (Step II) Training NNTS, (Step III) Calculation of step size according to system dynamics, (Step IV) Piece-wise vectorisation, (Step V) Predicting the output response on random test data.
  • Figure 4: Flowcharts for steps 3 and 4 of the proposed AHiTS scheme
  • Figure 5: Performance of AHiTS vs individual NNTS models on canonical ODEs
  • ...and 1 more figures