Forecasting Fails: Unveiling Evasion Attacks in Weather Prediction Models

TL;DR

The paper reveals a vulnerability in AI-based weather forecasting by introducing WAAPO, a targeted adversarial perturbation framework that imposes channel sparsity, spatial localization, and smoothness to craft stealthy initial-condition perturbations. Using ERA5 data and FourCastNet, WAAPO demonstrates how small, localized changes can steer forecasts toward predefined targets, highlighting risks to operational forecasting. It balances attack efficacy with realism through a composite loss and shows robust optimization improvements, underscoring the need for defenses in weather prediction systems. The work motivates broader evaluation across models and variables and calls for verification mechanisms to mitigate adversarial risks in climate-related forecasting tasks.

Abstract

With the increasing reliance on AI models for weather forecasting, it is imperative to evaluate their vulnerability to adversarial perturbations. This work introduces Weather Adaptive Adversarial Perturbation Optimization (WAAPO), a novel framework for generating targeted adversarial perturbations that are both effective in manipulating forecasts and stealthy to avoid detection. WAAPO achieves this by incorporating constraints for channel sparsity, spatial localization, and smoothness, ensuring that perturbations remain physically realistic and imperceptible. Using the ERA5 dataset and FourCastNet (Pathak et al. 2022), we demonstrate WAAPO's ability to generate adversarial trajectories that align closely with predefined targets, even under constrained conditions. Our experiments highlight critical vulnerabilities in AI-driven forecasting models, where small perturbations to initial conditions can result in significant deviations in predicted weather patterns. These findings underscore the need for robust safeguards to protect against adversarial exploitation in operational forecasting systems.

Figures (7)

• Figure 1: The weather forecasting process involves several key steps: data collection, data assimilation, forecasting, analysis, and dissemination. Data from sources like weather stations and satellites is processed through assimilation and forecasting models, refined through analysis, and shared with users via devices such as phones or TVs. Our study highlights vulnerabilities in this process, showing that adversaries can exploit the data collection phase to introduce perturbations and generate targeted false forecasts.
• Figure 2: Pointwise temperature differences (in Kelvin) comparing the 24-hour perturbed forecast with both the 120-hour adversarial target ($t_{\text{adv}}$) and the 24-hour ground truth (GT/True Value). Despite the adversarial target representing conditions 120 hours into the future, the unconstrained attack effectively manipulates the model’s prediction to align more closely with the adversarial target than the actual 24-hour ground truth. This highlights the attack's capability to fabricate a false global temperature event, significantly overriding the model’s original forecast.
• Figure 3: These results illustrate the Weather Adaptive Adversarial Perturbation Optimization (WAAPO) framework from Algorithm \ref{['alg:adversarial_perturbation']}. In (a), WAAPO targets only the temperature ($t2m$) channel (Kelvin), showing that even single-channel perturbations can significantly alter predictions, closely mirroring the behavior seen in Figure \ref{['fig:pointwise-mse']}. In (b), a spatial mask $M$ is applied over South America, demonstrating that localized, channel-specific perturbations can substantially reshape forecasts within the targeted region.
• Figure 4: This figure demonstrates why a smoothness constraint is essential for creating imperceptible perturbations using WAPPO. The top row, without smoothness constraints, displays a clearly visible patch in the targeted area, whereas applying a smoothness constraint in the bottom row yields more diffused, subtle perturbations that blend naturally and are harder to detect.
• Figure 5: Top: Normal Prediction. Bottom: Perturbed Prediction. Illustration of the effect of the unconstrained adversarial attack on weather forecasts.