Attention-Enhanced Convolutional Autoencoder and Structured Delay Embeddings for Weather Prediction
Amirpasha Hedayat, Karthik Duraisamy
TL;DR
The paper tackles short-range weather prediction for a chaotic, high-dimensional system by proposing an efficient reduced-order modeling (ROM) framework that couples an attention-enhanced convolutional autoencoder (CAE) with a time-delayed linear operator to evolve latent states. By employing a ResNet-based CAE with Convolutional Block Attention Modules (CBAM) and a structured time-delay embedding, the approach aims to capture dominant weather patterns with reduced computational cost. Results show that CAE+CBAM can achieve superior reconstruction quality compared to POD at similar compression and that larger time-delays improve interpolation within the training window, though generalization to unseen futures remains challenging and reconstruction error is the primary bottleneck. The findings highlight both the potential and limitations of efficient ROMs as baselines and motivate hybrid strategies that pair ROM efficiency with targeted AI enhancements for climate-scale forecasting.
Abstract
Weather prediction is a quintessential problem involving the forecasting of a complex, nonlinear, and chaotic high-dimensional dynamical system. This work introduces an efficient reduced-order modeling (ROM) framework for short-range weather prediction and investigates fundamental questions in dimensionality reduction and reduced order modeling of such systems. Unlike recent AI-driven models, which require extensive computational resources, our framework prioritizes efficiency while achieving reasonable accuracy. Specifically, a ResNet-based convolutional autoencoder augmented by block attention modules is developed to reduce the dimensionality of high-dimensional weather data. Subsequently, a linear operator is learned in the time-delayed embedding of the latent space to efficiently capture the dynamics. Using the ERA5 reanalysis dataset, we demonstrate that this framework performs well in-distribution as evidenced by effectively predicting weather patterns within training data periods. We also identify important limitations in generalizing to future states, particularly in maintaining prediction accuracy beyond the training window. Our analysis reveals that weather systems exhibit strong temporal correlations that can be effectively captured through linear operations in an appropriately constructed embedding space, and that projection error rather than inference error is the main bottleneck. These findings shed light on some key challenges in reduced-order modeling of chaotic systems and point toward opportunities for hybrid approaches that combine efficient reduced-order models as baselines with more sophisticated AI architectures, particularly for applications in long-term climate modeling where computational efficiency is paramount.