Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Embedding Ordinality to Binary Loss Function for Improving Solar Flare Forecasting

Chetraj Pandey, Anli Ji, Jinsu Hong, Rafal A. Angryk, Berkay Aydin

TL;DR

This work addresses binary solar flare forecasting by leveraging the ordinal structure of flare classes. It introduces an ordinality-aware binary cross-entropy loss (BCE-SF) that weights sub-classes within each binary label, enabling the model to respect the severity ordering of flare classes. Using a ResNet34 CNN with 1-channel AR patch images derived from SHARP magnetograms spanning the full solar disk, the authors demonstrate that BCE-SF improves discrimination and calibration across central and limb regions, quantified by the Composite Skill Score (CSS) which combines TSS and HSS as $CSS = \begin{cases}0,& TSS \times HSS < 0 \\\ sqrt{TSS \ times HSS},& \text{otherwise} \end{cases}$. The results show notable gains in CSS (up to about 7% in some zones) and successful near-limb predictions (CSS = 0.34; TSS = 0.50, HSS = 0.23), underscoring the approach's potential for more reliable and comprehensive solar flare forecasts. The work also provides a public codebase, enhancing reproducibility and enabling further extensions such as multi-modal data and spatiotemporal models.

Abstract

In this paper, we propose a novel loss function aimed at optimizing the binary flare prediction problem by embedding the intrinsic ordinal flare characteristics into the binary cross-entropy (BCE) loss function. This modification is intended to provide the model with better guidance based on the ordinal characteristics of the data and improve the overall performance of the models. For our experiments, we employ a ResNet34-based model with transfer learning to predict $\geq$M-class flares by utilizing the shape-based features of magnetograms of active region (AR) patches spanning from $-$90$^{\circ}$ to $+$90$^{\circ}$ of solar longitude as our input data. We use a composite skill score (CSS) as our evaluation metric, which is calculated as the geometric mean of the True Skill Score (TSS) and the Heidke Skill Score (HSS) to rank and compare our models' performance. The primary contributions of this work are as follows: (i) We introduce a novel approach to encode ordinality into a binary loss function showing an application to solar flare prediction, (ii) We enhance solar flare forecasting by enabling flare predictions for each AR across the entire solar disk, without any longitudinal restrictions, and evaluate and compare performance. (iii) Our candidate model, optimized with the proposed loss function, shows an improvement of $\sim$7%, $\sim$4%, and $\sim$3% for AR patches within $\pm$30$^\circ$, $\pm$60$^\circ$, and $\pm$90$^\circ$ of solar longitude, respectively in terms of CSS, when compared with standard BCE. Additionally, we demonstrate the ability to issue flare forecasts for ARs in near-limb regions (regions between $\pm$60$^{\circ}$ to $\pm$90$^{\circ}$) with a CSS=0.34 (TSS=0.50 and HSS=0.23), expanding the scope of AR-based models for solar flare prediction. This advances the reliability of solar flare forecasts, leading to more effective prediction capabilities.

Embedding Ordinality to Binary Loss Function for Improving Solar Flare Forecasting

TL;DR

This work addresses binary solar flare forecasting by leveraging the ordinal structure of flare classes. It introduces an ordinality-aware binary cross-entropy loss (BCE-SF) that weights sub-classes within each binary label, enabling the model to respect the severity ordering of flare classes. Using a ResNet34 CNN with 1-channel AR patch images derived from SHARP magnetograms spanning the full solar disk, the authors demonstrate that BCE-SF improves discrimination and calibration across central and limb regions, quantified by the Composite Skill Score (CSS) which combines TSS and HSS as . The results show notable gains in CSS (up to about 7% in some zones) and successful near-limb predictions (CSS = 0.34; TSS = 0.50, HSS = 0.23), underscoring the approach's potential for more reliable and comprehensive solar flare forecasts. The work also provides a public codebase, enhancing reproducibility and enabling further extensions such as multi-modal data and spatiotemporal models.

Abstract

In this paper, we propose a novel loss function aimed at optimizing the binary flare prediction problem by embedding the intrinsic ordinal flare characteristics into the binary cross-entropy (BCE) loss function. This modification is intended to provide the model with better guidance based on the ordinal characteristics of the data and improve the overall performance of the models. For our experiments, we employ a ResNet34-based model with transfer learning to predict M-class flares by utilizing the shape-based features of magnetograms of active region (AR) patches spanning from 90 to 90 of solar longitude as our input data. We use a composite skill score (CSS) as our evaluation metric, which is calculated as the geometric mean of the True Skill Score (TSS) and the Heidke Skill Score (HSS) to rank and compare our models' performance. The primary contributions of this work are as follows: (i) We introduce a novel approach to encode ordinality into a binary loss function showing an application to solar flare prediction, (ii) We enhance solar flare forecasting by enabling flare predictions for each AR across the entire solar disk, without any longitudinal restrictions, and evaluate and compare performance. (iii) Our candidate model, optimized with the proposed loss function, shows an improvement of 7%, 4%, and 3% for AR patches within 30, 60, and 90 of solar longitude, respectively in terms of CSS, when compared with standard BCE. Additionally, we demonstrate the ability to issue flare forecasts for ARs in near-limb regions (regions between 60 to 90) with a CSS=0.34 (TSS=0.50 and HSS=0.23), expanding the scope of AR-based models for solar flare prediction. This advances the reliability of solar flare forecasts, leading to more effective prediction capabilities.
Paper Structure (10 sections, 7 equations, 8 figures, 1 table)

This paper contains 10 sections, 7 equations, 8 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Data and Model
  4. Ordinality-aware Loss Function Design
  5. Experimental Settings
  6. Dataset
  7. Model Parameters
  8. Evaluation Metrics
  9. Experimental Evaluation
  10. Conclusion and Future Work

Figures (8)

  • Figure 1: An illustrative example of (a) Original raw input magnetogram of HMI AR patch corresponding to HARP number: 7115 (NOAA AR number: 12673) observed on 2017-09-06 at 06:00:00 UTC, (b) Bitmap corresponding to HMI AR patch in (a) showing the high activity region (region of interest) indicated by white pixels, (c) Final processed image of AR patch in (a) now sized to 512$\times$512, that is used to train our models.
  • Figure 2: The overall schema of the data processing pipeline used in this work. It shows a sequential pipeline for creating JPEG images from magnetogram rasters and corresponding bitmaps used for cropping the regions with relevant information. Boxes colored in green collectively defines our dataset.
  • Figure 3: (a) A bar plot representing the overall distribution of flare classes in our dataset (b) A bar plot showing binarized ($\geq$M) flare distributions across the four temporally non-overlapping tri-monthly data partitions. Note: The height of the bars are in logarithmic scale.
  • Figure 4: An illustrative plot showing: (a) Standard binary cross-entropy (BCE) loss. (b-c) BCE for solar flare prediction (BCE-SF) which encodes ordinal flare characteristics as loss weighting mechanism with $\alpha$=1 and $\alpha$=4 respectively. Note: FL class indicates target 1 and NF class indicates target 0.
  • Figure 5: An illustrative example of (a) input magnetogram of HMI AR patch corresponding to HARP number: 7115 (NOAA AR number: 12673). (b-f) five different augmentations applied to AR patch in (a). These augmentations are applied to the processed magnetograms before scaling to 0-255.
  • ...and 3 more figures