Setting the Standard: Recommended Practices for Data Preprocessing in Data-Driven Climate Prediction

TL;DR

The paper argues that data preprocessing quality is a critical driver of predictive skill in climate-focused AI/ML, especially given nonstationarity and spatiotemporal correlations. It proposes a structured set of preprocessing practices, covering initial problem framing, anomaly calculation, handling of non-Gaussian distributions, and leakage prevention through careful data splitting and cross-validation. Through two case studies, it demonstrates how preprocessing choices can dramatically alter inferred regimes and predictive skill, underscoring the need for transparency and reproducibility. The work aims to standardize preprocessing to improve robustness, comparability, and trust in AI-driven climate predictions across timescales, while calling for open-source tooling and broader benchmarking.

Abstract

Artificial intelligence (AI) - and specifically machine learning (ML) - applications for climate prediction across timescales are proliferating quickly. The emergence of these methods prompts a revisit to the impact of data preprocessing, a topic familiar to the climate community, as more traditional statistical models work with relatively small sample sizes. Indeed, the skill and confidence in the forecasts produced by data-driven models are directly influenced by the quality of the datasets and how they are treated during model development, thus yielding the colloquialism, "garbage in, garbage out." As such, this article establishes protocols for the proper preprocessing of input data for AI/ML models designed for climate prediction (i.e., subseasonal to decadal and longer). The three aims are to: (1) educate researchers, developers, and end users on the effects that preprocessing has on climate predictions; (2) provide recommended practices for data preprocessing for such applications; and (3) empower end users to decipher whether the models they are using are properly designed for their objectives. Specific topics covered in this article include the creation of (standardized) anomalies, dealing with non-stationarity and the spatiotemporally correlated nature of climate data, and handling of extreme values and variables with potentially complex distributions. Case studies will illustrate how using different preprocessing techniques can produce different predictions from the same model, which can create confusion and decrease confidence in the overall process. Ultimately, implementing the recommended practices set forth in this article will enhance the robustness and transparency of AI/ML in climate prediction studies.

Figures (4)

• Figure 1: Illustration of various time series preprocessing transformations using ERA5 hersbach2020. Preprocessing data leakage for global (latitude weighted) monthly mean of daily maximum temperature anomalies derived from a monthly climatology (1981-2010) is shown in panels (a) and (b), where the black lines represent no data leakage due to detrending using only the training set period, and the pink lines represent leakage due to detrending using the training and (hypothetical) testing periods. Various kernel density estimations of data transformations are shown in panel (c) for a non-normal precipitation variable (black dashed line).
• Figure 2: Summary figure illustrating a typical AI/ML workflow, detailing the preprocessing steps for numerical and categorical data, presented in a recommended order. However, preprocessing steps (and their order) may be application-specific.
• Figure 3: a) Intercluster correlation, with star markers indicating the "preferred" number of clusters, which keep the correlation between centroids at or just below zero. b) The annual climatological frequency of weather regimes. The control and experiments in panels a) and b) are specified in the legend of panel a). c-h) Standardized Z500 anomalies for the "control." When the respective regimes were identified in the experiments, contour lines were overlaid. White contour lines represent Experiment 1, while black contour lines denote Experiment 2. Solid lines indicate positive anomalies (+0.5 and +1.0 contour), and dashed lines represent negative anomalies (-0.5 and -1.0).
• Figure 4: Temperature anomaly (black; the "truth") and neural network predictions using test datasets preprocessed in different ways: (1) trend computed during the test period (trend; orange), (2) climatology computed during the validation and test periods (climo; green), (3) data splits with potential leakage (split; red), and (4) a combination of the trend, climo, and split preprocessing steps (split_trend_climo; purple) (5) no data leakage (clean; blue). Corresponding skill scores are shown in parentheses. The MAE (mean absolute error; ° C) is first calculated using the incorrectly computed test labels for each experiment, producing inflated skill estimates. The "adjusted" MAE is calculated using the properly cleaned test labels to obtain an accurate measure of prediction error.