Global Cross-Time Attention Fusion for Enhanced Solar Flare Prediction from Multivariate Time Series

TL;DR

The paper tackles forecasting intense solar flares from multivariate time series by introducing Global Cross-Time Attention Fusion (GCTAF), a transformer-based model that uses learnable global tokens to summarize globally salient temporal patterns via cross-attention and fuses them with local temporal features. Evaluated on the SWAN-SF MVTS benchmark with chronologically split partitions, GCTAF outperforms multiple baselines across accuracy and skill metrics, demonstrating the value of global temporal context for imbalanced flare prediction. The method combines a transformer encoder with cross-attention to global tokens, followed by pooling and an MLP classifier, achieving robust performance while addressing the rarity of M/X-class events. The results suggest refined transformer architectures with global temporal summarization as a promising direction for solar flare prediction and MVTS learning in imbalanced time-series tasks.

Abstract

Multivariate time series classification is increasingly investigated in space weather research as a means to predict intense solar flare events, which can cause widespread disruptions across modern technological systems. Magnetic field measurements of solar active regions are converted into structured multivariate time series, enabling predictive modeling across segmented observation windows. However, the inherently imbalanced nature of solar flare occurrences, where intense flares are rare compared to minor flare events, presents a significant barrier to effective learning. To address this challenge, we propose a novel Global Cross-Time Attention Fusion (GCTAF) architecture, a transformer-based model to enhance long-range temporal modeling. Unlike traditional self-attention mechanisms that rely solely on local interactions within time series, GCTAF injects a set of learnable cross-attentive global tokens that summarize salient temporal patterns across the entire sequence. These tokens are refined through cross-attention with the input sequence and fused back into the temporal representation, enabling the model to identify globally significant, non-contiguous time points that are critical for flare prediction. This mechanism functions as a dynamic attention-driven temporal summarizer that augments the model's capacity to capture discriminative flare-related dynamics. We evaluate our approach on the benchmark solar flare dataset and show that GCTAF effectively detects intense flares and improves predictive performance, demonstrating that refining transformer-based architectures presents a high-potential alternative for solar flare prediction tasks.

Figures (5)

• Figure 1: The GCTAF model for solar flare prediction takes input of shape $[B, \tau, N]$ and learnable global tokens $[1, G, N]$ shared across batches. The global tokens attend to the input via cross-attention, producing $[B, G, N]$. The input and global tokens are concatenated to $[B, \tau+G, N]$ and processed by transformer encoder blocks. The output is split into local $[B, \tau, N]$ and global $[B, G, N]$ tokens. Local tokens are pooled to $[B, N]$, global tokens averaged to $[B, N]$, then concatenated into $[B, 2N]$ and passed through an MLP for final logits.
• Figure 2: Effect of number of global tokens on TSS score.
• Figure 3: Training and validation loss curves for GCTAF over 20 epochs.
• Figure 4: Mean TSS results for variants of GCTAF.
• Figure 5: Bar chart showing flare prediction scores across four metrics.