Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Inspectorch: Efficient rare event exploration in solar observations

C. J. Díaz Baso, I. J. Soler Poquet, C. Kuckein, M. van Noort, N. Poirier

TL;DR

Inspectorch, an open-source framework that utilizes flow-based models: flexible density estimators capable of learning the multidimensional distribution of solar observations, demonstrates that density estimation using flow-based models offers a powerful approach to identifying rare events in large solar datasets.

Abstract

The Sun is observed in unprecedented detail, enabling studies of its activity on very small spatiotemporal scales. However, the large volume of data collected by our telescopes cannot be fully analyzed with conventional methods. Popular machine learning methods identify general trends from observations, but tend to overlook unusual events due to their low frequency of occurrence. We study the applicability of unsupervised probabilistic methods to efficiently identify rare events in multidimensional solar observations and optimize our computational resources to the study of these extreme phenomena. We introduce Inspectorch, an open-source framework that utilizes flow-based models: flexible density estimators capable of learning the multidimensional distribution of solar observations. Once optimized, it assigns a probability to each sample, allowing us to identify unusual events. We apply this approach by applying it to observations from the Hinode Spectro-Polarimeter, the Interface Region Imaging Spectrograph, the Microlensed Hyperspectral Imager at Swedish 1-m Solar Telescope, the Atmospheric Imaging Assembly on board the Solar Dynamics Observatory and the Extreme Ultraviolet Imager on board Solar Orbiter. We find that the algorithm assigns consistently lower probabilities to spectra that exhibit unusual features. For example, it identifies profiles with very strong Doppler shifts, uncommon broadening, and temporal dynamics associated with small-scale reconnection events, among others. As a result, Inspectorch demonstrates that density estimation using flow-based models offers a powerful approach to identifying rare events in large solar datasets. The resulting probabilistic anomaly scores allow computational resources to be focused on the most informative and physically relevant events. We make our Python package publicly available at https://github.com/cdiazbas/inspectorch.

Inspectorch: Efficient rare event exploration in solar observations

TL;DR

Inspectorch, an open-source framework that utilizes flow-based models: flexible density estimators capable of learning the multidimensional distribution of solar observations, demonstrates that density estimation using flow-based models offers a powerful approach to identifying rare events in large solar datasets.

Abstract

The Sun is observed in unprecedented detail, enabling studies of its activity on very small spatiotemporal scales. However, the large volume of data collected by our telescopes cannot be fully analyzed with conventional methods. Popular machine learning methods identify general trends from observations, but tend to overlook unusual events due to their low frequency of occurrence. We study the applicability of unsupervised probabilistic methods to efficiently identify rare events in multidimensional solar observations and optimize our computational resources to the study of these extreme phenomena. We introduce Inspectorch, an open-source framework that utilizes flow-based models: flexible density estimators capable of learning the multidimensional distribution of solar observations. Once optimized, it assigns a probability to each sample, allowing us to identify unusual events. We apply this approach by applying it to observations from the Hinode Spectro-Polarimeter, the Interface Region Imaging Spectrograph, the Microlensed Hyperspectral Imager at Swedish 1-m Solar Telescope, the Atmospheric Imaging Assembly on board the Solar Dynamics Observatory and the Extreme Ultraviolet Imager on board Solar Orbiter. We find that the algorithm assigns consistently lower probabilities to spectra that exhibit unusual features. For example, it identifies profiles with very strong Doppler shifts, uncommon broadening, and temporal dynamics associated with small-scale reconnection events, among others. As a result, Inspectorch demonstrates that density estimation using flow-based models offers a powerful approach to identifying rare events in large solar datasets. The resulting probabilistic anomaly scores allow computational resources to be focused on the most informative and physically relevant events. We make our Python package publicly available at https://github.com/cdiazbas/inspectorch.
Paper Structure (21 sections, 8 equations, 13 figures)

This paper contains 21 sections, 8 equations, 13 figures.

Table of Contents

  1. Introduction
  2. Anomaly detection as density estimation
  3. Normalizing Flows
  4. Results: Applications to solar data
  5. Detecting supersonic downflows with Hinode/SP
  6. Comparison with Isolation Forest
  7. Probing the transition region with IRIS
  8. Extreme events in high-cadence MiHI data
  9. Disentangling multi-channel information with SDO/AIA
  10. Generalizing to spatial and temporal features
  11. Spatial anomaly detection in Hinode/SP
  12. Temporal anomaly detection
  13. Ellerman bombs vs. network bright points in SDO/AIA
  14. Transient coronal events with Solar Orbiter/EUI
  15. Scaling density estimation via Flow Matching
  16. ...and 6 more sections

Figures (13)

  • Figure 1: Application of Inspectorch to a Hinode/SP dataset containing a sunspot. The left panels show the continuum intensity map at 630.10 nm and log-probability $\log p_\phi(\mathbf{x})$ map. The color scale is clipped to highlight the most extreme values, although some pixels have log-probabilities below $-110$. The right panel shows an example of two of the most unusual spectra extracted from the low-probability regions. The spectra are normalized to their maximum value. A vertical dashed line indicates Doppler shifts of $10$ km s$^{-1}$ and an average quiet-Sun profile in gray is shown for reference.
  • Figure 2: Application of Inspectorch to an IRIS dataset of a coronal hole. The left panels show the intensity map at the core of the Si iv line and log-probability $\log p_\phi(\mathbf{x})$ map. The color scale in the log-probability map is clipped to a minimum of $-150$ to enhance contrast, though some pixels exhibit lower values. The right panel shows an example of two of the most unusual spectra extracted from the low-probability regions. The spectra are normalized to their maximum value. Vertical dashed lines indicate Doppler shifts of $\pm70$ km s$^{-1}$ and an average quiet-Sun profile in gray is also shown for reference.
  • Figure 3: Temporal evolution of the minimum (red line) and maximum (gray line) log-probability values in a time series of MiHI observations. Sharp dips in the minimum log-probability indicate the occurrence of very unusual events.
  • Figure 4: Distribution of the log-probability values in the MiHI dataset. A vertical line indicates the threshold used to select the rarest pixels.
  • Figure 5: Application of Inspectorch to the spatial dimension of the Hinode/SP dataset, with a patch size of $5\times5$ pixels. The log-probability map highlights regions with unusual spatial patterns.
  • ...and 8 more figures