Operational Solar Flare Forecasting System Using an Explainable Large Language Model

TL;DR

This work tackles operational forecasting of major solar flares by introducing LLMFlareNet, an explainable transformer-based system that treats time-series AR magnetogram features as inputs to a pre-trained BERT model within the Frozen Pretrained Transformer framework. It combines two data pipelines—ten CV SHARP-based datasets and daily comparison datasets—to evaluate both cross-validated performance and real-time operational capability, using SHAP to interpret feature importance and model decisions. The key findings show that LLMFlareNet achieves superior $TSS$ scores (e.g., $0.720 \pm 0.040$ on mixed AR CV data and $0.799$ on single AR CV data) and that $R\_VALUE$ is the most influential feature in predictions, aligning with magnetic reconnection theory. The paper also demonstrates a practical operational system that outperforms NASA/CCMC and SolarFlareNet in daily-mode comparisons and discusses future work towards multi-class flare forecasting and multi-source data integration for enhanced space weather monitoring and warning.

Abstract

This study focuses on forecasting major (>=M-class) solar flares that can severely impact the near-Earth environment. We construct two types of datasets using the Space Weather HMI Active Region Patches (SHARP), and develop a flare prediction network based on large language model (LLMFlareNet). We apply SHapley Additive exPlanations (SHAP) to explain the model predictions. We develop an operational forecasting system based on the LLMFlareNet model. We adopt a daily mode for performance comparison across various operational forecasting systems under identical active region (AR) number and prediction date, using daily operational observational data. The main results are as follows. (1) Through ablation experiments and comparison with baseline models, LLMFlareNet achieves the best TSS scores of 0.720 +/- 0.040 on the ten cross-validation (CV) dataset with mixed ARs. (2) By both global and local SHAP analyses, we identify that R_VALUE is the most influential physical feature for the prediction of LLMFlareNet, aligning with flare magnetic reconnection theory. (3) In daily mode, LLMFlareNet achieves TSS scores of 0.680/0.571 (0.689/0.661, respectively) on the dataset with single/mixed ARs, markedly outperforming NASA/CCMC (SolarFlareNet, respectively). This work introduces the first application of a large language model as a universal computation engine with explainability method in this domain, and presents the first comparison between operational flare forecasting systems in daily mode. The proposed LLMFlareNet-based system demonstrates substantial improvements over existing systems.

Figures (16)

• Figure 1: The model structure of LLMFlareNet.
• Figure 2: The architecture diagram of AR flare operational forecasting system based on the B/S architecture.
• Figure 3: A bar chart illustrating the global importance of the 10 physical features for LLMFlareNet on one testing dataset from ten CV datasets. The x-axis represents the mean SHAP value and the y-axis lists the 10 physical features sorted in descending order of importance.
• Figure 4: A beeswarm plot illustrating the impact of the ten features on LLMFlareNet for each AR. Each point corresponds to one AR. The x-axis represents the summed SHAP value of a physical feature across all time steps for each AR, with the color of the scatter points indicating the relative magnitude of the feature value.
• Figure 5: The force plot for the correct prediction of a positive class for AR11380. Figure \ref{['fig:newfigure6']}(a) shows the force plot for AR11380 across all time steps, while Figure \ref{['fig:newfigure6']}(b) depicts the force plot for AR11380 at the 23rd time step. Each colored band corresponds to a physical feature. At a certain time step, the width of each band represents the SHAP value of the corresponding feature among the ten features. Red color indicates features that increase the output probability of the model, while blue color indicates features that decrease it. The AR11380 produced an M-class flare at 20:12 UT on December 26, 2011.