Explainable AI-Based Interface System for Weather Forecasting Model

TL;DR

This work presents a user-centered workflow for Explainable AI in weather forecasting, defining three concrete explanation requirements—model performance by rainfall type, output reasoning, and confidence—to support forecasters. It maps these requirements to a rainfall-type classifier with a performance diagram, feature-attribution-based output reasoning, and probability calibration (including Local Temperature Scaling) to produce a practical XAI interface. A UNet2-based very short-term rainfall predictor is explained, and a pilot interface is prototyped and evaluated with forecasters, showing increased trust and decision utility, though some explanations remain challenging to interpret and require integration with existing systems. The study highlights limitations such as sample size and domain scope and points to future work on multi-modal inputs and interactive dialogue to enhance user acceptance and operational utility.

Abstract

Machine learning (ML) is becoming increasingly popular in meteorological decision-making. Although the literature on explainable artificial intelligence (XAI) is growing steadily, user-centered XAI studies have not extend to this domain yet. This study defines three requirements for explanations of black-box models in meteorology through user studies: statistical model performance for different rainfall scenarios to identify model bias, model reasoning, and the confidence of model outputs. Appropriate XAI methods are mapped to each requirement, and the generated explanations are tested quantitatively and qualitatively. An XAI interface system is designed based on user feedback. The results indicate that the explanations increase decision utility and user trust. Users prefer intuitive explanations over those based on XAI algorithms even for potentially easy-to-recognize examples. These findings can provide evidence for future research on user-centered XAI algorithms, as well as a basis to improve the usability of AI systems in practice.

Figures (16)

• Figure 1: Workflow for developing a user-centered explainable artificial intelligence (XAI) interface system. The system is developed based on the procedures established in the previous literature liao2021questionliao2020questioning. The scope of explanations is defined based on the requirements set by the practitioners; appropriate XAI algorithms are selected based on the defined scope; and the interface is designed with user feedback
• Figure 2: The target precipitation forecasting model and data. The data consists of radar hybrid scan reflectivity.
• Figure 3: The performance of the target model. UNet1 and UNet2 built by NIMS are comparable to MetNet sonderby2020metnet and HRRR numerical model for very short-term predictions. Reproduced from sonderby2020metnet and yun2021development.
• Figure 4: The structure of the precipitation classifier (A) and the resulting confusion matrix (B). The rainfall types are based on a SOM-based weather classification study (an unpublished result of shin2022classification with the same research procedure on a specific region).
• Figure 5: Confusion matrices to calculate performance metrics on the imbalanced data.