Modeling chaotic Lorenz ODE System using Scientific Machine Learning

TL;DR

SciML models can prove to be a reliable tool for modeling climate by combining the interpretability of physical climate models with the computational power of neural networks, indicating a shift from the traditional black box-based machine learning modeling of climate systems to physics-informed decision-making, leading to effective climate policy implementation.

Abstract

In climate science, models for global warming and weather prediction face significant challenges due to the limited availability of high-quality data and the difficulty in obtaining it, making data efficiency crucial. In the past few years, Scientific Machine Learning (SciML) models have gained tremendous traction as they can be trained in a data-efficient manner, making them highly suitable for real-world climate applications. Despite this, very little attention has been paid to chaotic climate system modeling utilizing SciML methods. In this paper, we have integrated SciML methods into foundational weather models, where we have enhanced large-scale climate predictions with a physics-informed approach that achieves high accuracy with reduced data. We successfully demonstrate that by combining the interpretability of physical climate models with the computational power of neural networks, SciML models can prove to be a reliable tool for modeling climate. This indicates a shift from the traditional black box-based machine learning modeling of climate systems to physics-informed decision-making, leading to effective climate policy implementation.

Figures (8)

• Figure 1: Plots showcasing the solution to the chaotic Lorenz ODE system of equations which generate a pair of spiral swirls when projected in three dimensions.
• Figure 2: Both tanh and ReLU struggle to consistently reduce the loss, showing fluctuations but largely stagnating. In contrast, the sigmoid activation function resulted in a steadily decreasing, exponential loss curve. The second Figure shows the trend of exponential loss when sigmoid is employed.
• Figure 3: Illustration of training and extended forecasting of the chaotic Lorenz system using Neural ODEs. The markers represent the true $u_1$, $u_2$, and $u_3$ and lines represent predicted values of the three variables over time.
• Figure 4: Comparison between architectures consisting of a deeper neural network vs a shallow neural network: There is some improvement in the loss when a network is trained on the deeper network, but this increases time significantly and does not provide substantial results when compared to a shallow network.
• Figure 5: a) demonstrates the ability of the model to recover missing terms from the equations and white b) performs accurate predictions over increased time.