Masked Autoregressive Model for Weather Forecasting

TL;DR

The Masked Autoregressive Model for Weather Forecasting (MAM4WF) combines the advantages of both autoregressive and lead time embedding methods, offering flexibility in lead time modeling while iteratively integrating predictions.

Abstract

The growing impact of global climate change amplifies the need for accurate and reliable weather forecasting. Traditional autoregressive approaches, while effective for temporal modeling, suffer from error accumulation in long-term prediction tasks. The lead time embedding method has been suggested to address this issue, but it struggles to maintain crucial correlations in atmospheric events. To overcome these challenges, we propose the Masked Autoregressive Model for Weather Forecasting (MAM4WF). This model leverages masked modeling, where portions of the input data are masked during training, allowing the model to learn robust spatiotemporal relationships by reconstructing the missing information. MAM4WF combines the advantages of both autoregressive and lead time embedding methods, offering flexibility in lead time modeling while iteratively integrating predictions. We evaluate MAM4WF across weather, climate forecasting, and video frame prediction datasets, demonstrating superior performance on five test datasets.

Figures (5)

• Figure 1: Comparison of (a) autoregressive and (b) masked autoregressive inference methods. (a) repeatedly uses frames with the trained input length for the next prediction. (b) is a structure that flexibly adjusts the input length using masked frames for the next prediction step.
• Figure 2: An overview of MAM4WF training process. MAM4WF is an implicit model for time step $t$. MAM4WF consists of a spatial encoder $e(\cdot)$, a spatio-temporal predictor $p(\cdot)$, and a spatial decoder $d(\cdot)$. Note that, MAM4WF is trained masked autoregressive manner with error-prone queue.
• Figure 3: Performance comparison experiment according to output length and time interval changes of SimVP and MAM4VP on the Moving MNIST dataset.
• Figure 4: Prediction results of MAM4WF on the ICRA-ENSO dataset. The color bar means SST anomalies on the global map. Best viewed with zoom.
• Figure 5: Prediction results of MAM4WF and Earthformer on the SEVIR dataset, represented by vertically integrated liquid water contents (0-255 scale) shown on the color bar.