Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimizing Supernova Classification with Interpretable Machine Learning Models

Anurag Garg

TL;DR

This study tackles photometric Type Ia supernova classification under severe class imbalance by advocating metric-aware evaluation (PR-AUC and F1) and leveraging a Bayesian-tuned XGBoost ensemble over deeper architectures. Using the SPCC dataset with pre-engineered light-curve features, the authors find that XGBoost without SMOTE or thresholding delivers strong, interpretable performance (PR-AUC ≈ 0.996, F1 ≈ 0.923, ROC-AUC ≈ 0.973) while maintaining computational efficiency. The work demonstrates that optimized, transparent ensembles can rival deep learning in precision-recall trade-offs, reduce follow-up costs, and scale to large surveys like LSST. It also emphasizes the adoption of PR-AUC as a standard metric for imbalanced astrophysical classification and highlights avenues for incorporating astrophysical priors to further boost robustness.

Abstract

Photometric classification of Type Ia supernovae (SNe Ia) is critical for cosmological studies but remains difficult due to class imbalance and observational noise. While deep learning models have been explored, they are often resource-intensive and lack interpretability. We present a computationally efficient and interpretable classification framework that maintains high performance on imbalanced datasets. We emphasize the use of PR-AUC and F1-score as more informative metrics than ROC-AUC in severely imbalanced settings. Using an XGBoost ensemble optimized via Bayesian hyperparameter tuning, we classified light curves from the Supernova Photometric Classification Challenge (SPCC), comprising 21,318 events with a 3.19 imbalance ratio (non-Ia to Ia). Our model achieved a PR-AUC of $0.993^{+0.03}_{-0.02}$, an F1-score of $0.923 \pm 0.008$, and a ROC-AUC of $0.976 \pm 0.004$, matching or exceeding deep learning performance on precision-recall trade-offs while using fewer resources. Despite slightly lower overall accuracy, our method balances false positives and false negatives, improving the efficiency of spectroscopic follow-up. We show that optimized ensemble models offer a reproducible and lightweight alternative to complex architectures, particularly for large-scale surveys such as the Legacy Survey of Space and Time (LSST) where transparency and efficiency are essential.

Optimizing Supernova Classification with Interpretable Machine Learning Models

TL;DR

This study tackles photometric Type Ia supernova classification under severe class imbalance by advocating metric-aware evaluation (PR-AUC and F1) and leveraging a Bayesian-tuned XGBoost ensemble over deeper architectures. Using the SPCC dataset with pre-engineered light-curve features, the authors find that XGBoost without SMOTE or thresholding delivers strong, interpretable performance (PR-AUC ≈ 0.996, F1 ≈ 0.923, ROC-AUC ≈ 0.973) while maintaining computational efficiency. The work demonstrates that optimized, transparent ensembles can rival deep learning in precision-recall trade-offs, reduce follow-up costs, and scale to large surveys like LSST. It also emphasizes the adoption of PR-AUC as a standard metric for imbalanced astrophysical classification and highlights avenues for incorporating astrophysical priors to further boost robustness.

Abstract

Photometric classification of Type Ia supernovae (SNe Ia) is critical for cosmological studies but remains difficult due to class imbalance and observational noise. While deep learning models have been explored, they are often resource-intensive and lack interpretability. We present a computationally efficient and interpretable classification framework that maintains high performance on imbalanced datasets. We emphasize the use of PR-AUC and F1-score as more informative metrics than ROC-AUC in severely imbalanced settings. Using an XGBoost ensemble optimized via Bayesian hyperparameter tuning, we classified light curves from the Supernova Photometric Classification Challenge (SPCC), comprising 21,318 events with a 3.19 imbalance ratio (non-Ia to Ia). Our model achieved a PR-AUC of , an F1-score of , and a ROC-AUC of , matching or exceeding deep learning performance on precision-recall trade-offs while using fewer resources. Despite slightly lower overall accuracy, our method balances false positives and false negatives, improving the efficiency of spectroscopic follow-up. We show that optimized ensemble models offer a reproducible and lightweight alternative to complex architectures, particularly for large-scale surveys such as the Legacy Survey of Space and Time (LSST) where transparency and efficiency are essential.
Paper Structure (18 sections, 1 equation, 5 figures, 4 tables)

This paper contains 18 sections, 1 equation, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Rationale for Metric and Model Selection
  3. Methodology
  4. Dataset and Preprocessing
  5. Machine Learning Models
  6. Quantitative Model Comparison and Selection Justification
  7. Feature Engineering:
  8. Missing Value Handling:
  9. Oversampling (SMOTE):
  10. Threshold Tuning:
  11. Evaluation Metrics
  12. Results and Comparative Analysis
  13. Evaluation Metrics of the Current Model
  14. Comparative Analysis with Historical Studies
  15. Misclassification Analysis
  16. ...and 3 more sections

Figures (5)

  • Figure 1: Class Distribution in the Training data set
  • Figure 2: Precision-Recall Curve (PR-AUC = 0.99)
  • Figure 3: Prediction Probability Distribution for Type Ia (blue) and Non-Type Ia (orange) Supernovae. The overlapping region highlights classification ambiguity.
  • Figure 4: False Positive vs. False Negative Counts in Misclassification Analysis.
  • Figure 5: Cumulative Gain Curve