NeurIPS 2024 Ariel Data Challenge: Characterisation of Exoplanetary Atmospheres Using a Data-Centric Approach

TL;DR

This paper investigates exoplanetary atmospheric retrieval from simulated transit spectra in the NeurIPS 2024 Ariel Data Challenge using a data-centric workflow. It emphasizes data quality, calibration, feature engineering, and uncertainty-aware modeling, implementing a bagging-based regression to predict per-wavelength means and uncertainties across 283 wavelengths. The results show that uncertainty estimation critically shapes the Gaussian Log-Likelihood score and reveal limits of tabular feature engineering for spectroscopic retrieval under out-of-distribution conditions; the final approach attains a cross-validation GLL of about 66% and a private GLL around 50%. These findings highlight the importance of data-centric processing for robust atmospheric characterization in mission-like contexts and inform future retrieval strategies that prioritize data quality over model complexity.

Abstract

The characterization of exoplanetary atmospheres through spectral analysis is a complex challenge. The NeurIPS 2024 Ariel Data Challenge, in collaboration with the European Space Agency's (ESA) Ariel mission, provided an opportunity to explore machine learning techniques for extracting atmospheric compositions from simulated spectral data. In this work, we focus on a data-centric business approach, prioritizing generalization over competition-specific optimization. We briefly outline multiple experimental axes, including feature extraction, signal transformation, and heteroskedastic uncertainty modeling. Our experiments demonstrate that uncertainty estimation plays a crucial role in the Gaussian Log-Likelihood (GLL) score, impacting performance by several percentage points. Despite improving the GLL score by 11%, our results highlight the inherent limitations of tabular modeling and feature engineering for this task, as well as the constraints of a business-driven approach within a Kaggle-style competition framework. Our findings emphasize the trade-offs between model simplicity, interpretability, and generalization in astrophysical data analysis.

Figures (6)

• Figure 1: Illustration of the transformation applied to the raw images signal, from image domain to time domain, in order to predict the spectral domain.
• Figure 2: Overview of the proposed methodology. The workflow consists of a local data-centric pipeline including data pre-processing, feature engineering, and uncertainty-aware modeling. The trained model is then deployed on the Kaggle platform to perform inference and evaluation on the public and private challenge datasets. Insights from the public leader board and model evaluation are fed back into the pipeline (yellow arrow) to improve data processing and enhance model generalization in a data-centric approach.
• Figure 3: Mean Representation and Binning of AIRS signal
• Figure 4: Illustration of the signal correction process. The green line represents the raw signal obtained from the AIRS instrument, showing the initial data with potential noise and drifts. The blue line indicates the local detrended signal, where initial corrections have been applied to mitigate trends and noise. The orange line shows the global corrected signal after applying slope and trend adjustments across all segments.
• Figure 5: Variation of the Gaussian Log-Likelihood (GLL) score defined in the equation (\ref{['eq:final_score']}) with respect to the $\sigma_{user}$ uncertainty and prediction ($\mu_{user}$) values. The heatmap illustrates how different combinations of $\mu_{user}$ and $\sigma_{user}$ impact the final score. When the uncertainty ($\sigma_{user}$) is smaller than the prediction error, the score drastically decreases, as seen in the region above the diagonal. Conversely, when uncertainty is larger than the prediction error, the model is more tolerant, leading to a higher score. The region below the diagonal shows this behavior.