Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improved deep learning of chaotic dynamical systems with multistep penalty losses

Dibyajyoti Chakraborty, Seung Whan Chung, Ashesh Chattopadhyay, Romit Maulik

TL;DR

This paper introduces a novel framework that addresses the long-term behavior of chaotic systems by leveraging the recently proposed multi-step penalty (MP) optimization technique, and extends the applicability of MP optimization to a wide range of deep learning architectures, including Fourier Neural Operators and UNETs.

Abstract

Predicting the long-term behavior of chaotic systems remains a formidable challenge due to their extreme sensitivity to initial conditions and the inherent limitations of traditional data-driven modeling approaches. This paper introduces a novel framework that addresses these challenges by leveraging the recently proposed multi-step penalty (MP) optimization technique. Our approach extends the applicability of MP optimization to a wide range of deep learning architectures, including Fourier Neural Operators and UNETs. By introducing penalized local discontinuities in the forecast trajectory, we effectively handle the non-convexity of loss landscapes commonly encountered in training neural networks for chaotic systems. We demonstrate the effectiveness of our method through its application to two challenging use-cases: the prediction of flow velocity evolution in two-dimensional turbulence and ocean dynamics using reanalysis data. Our results highlight the potential of this approach for accurate and stable long-term prediction of chaotic dynamics, paving the way for new advancements in data-driven modeling of complex natural phenomena.

Improved deep learning of chaotic dynamical systems with multistep penalty losses

TL;DR

This paper introduces a novel framework that addresses the long-term behavior of chaotic systems by leveraging the recently proposed multi-step penalty (MP) optimization technique, and extends the applicability of MP optimization to a wide range of deep learning architectures, including Fourier Neural Operators and UNETs.

Abstract

Predicting the long-term behavior of chaotic systems remains a formidable challenge due to their extreme sensitivity to initial conditions and the inherent limitations of traditional data-driven modeling approaches. This paper introduces a novel framework that addresses these challenges by leveraging the recently proposed multi-step penalty (MP) optimization technique. Our approach extends the applicability of MP optimization to a wide range of deep learning architectures, including Fourier Neural Operators and UNETs. By introducing penalized local discontinuities in the forecast trajectory, we effectively handle the non-convexity of loss landscapes commonly encountered in training neural networks for chaotic systems. We demonstrate the effectiveness of our method through its application to two challenging use-cases: the prediction of flow velocity evolution in two-dimensional turbulence and ocean dynamics using reanalysis data. Our results highlight the potential of this approach for accurate and stable long-term prediction of chaotic dynamics, paving the way for new advancements in data-driven modeling of complex natural phenomena.
Paper Structure (6 sections, 4 equations, 5 figures)

This paper contains 6 sections, 4 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Methodology
  3. Results
  4. Kolmogorov Flow
  5. Ocean Reanalysis Data
  6. Conclusion

Figures (5)

  • Figure 1: A schematic for MP optimization. The model $F$ can be any autoregressive machine learning model. The intermediate discontinuity $\delta$ is introduced after every $r$ rollouts.
  • Figure 2: Vorticity of 2D Kolmogorov flow from predicted velocity fields. 't' here is the rollout step of the model.
  • Figure 3: A comparison between FNO and MP-FNO for (a) angle-averaged total kinetic energy spectrum and (b) correlation with DNS. In (a) we check the performance for an invariant statistic, and for (b) we assess how the MP FNO technique improves accuracy with forecast duration compared to vanilla FNO.
  • Figure 4: Prediction performance of UNET and MP-UNET for the GoM LCE shedding event: Eddy Sverdrup
  • Figure 5: RMSE comparison between UNET, MP-UNET and Persistence. Persistence is an elementary model used to compare the performance of other models. It assumes that the weather is static and the initial condition itself is the forecast.