PERTURB-c: Correlation Aware Perturbation Explainability for Regression Techniques to Understand Retrieval Black-boxes

TL;DR

PERTURB-c tackles the challenge of interpreting black-box regression models for exoplanet spectral retrievals in the presence of strong input correlations. It introduces a correlation-aware perturbation framework that perturbs adjacent wavelength groups with a Gaussian mask, bounds perturbations by instrument noise, and visualizes results via per-wavelength heatmaps. In a WASP-107b-like case study, PERTURB-c provides physics-aligned explanations of model predictions, identifies where SiO and SO2 bands drive the inference, and achieves substantial speedups over SHAP due to improved sample efficiency. The framework offers a practical, model-agnostic tool to validate ML-based retrievals and to guide observational strategies in the JWST and Ariel era, enhancing transparency and adoption of machine learning in atmospheric characterization.

Abstract

In this paper we introduce PERTURB-c, a correlation-aware framework for interpreting black box regression models with one-dimensional structured inputs. We demonstrate this framework on a simulated case study with machine learning based transit spectroscopy retrievals of exoplanet WASP-107b. Characterising many exoplanet atmospheres can answer important questions about planetary populations, but traditional retrievals are very resource intensive; machine learning based methods offer a fast alternative however (i) they require high volumes data (only obtainable through simulations) to train and (ii) their complexity renders them black-boxes. Better understanding how they reach predictions can allow us to inspect for biases, which is especially important with simulated data, and verify that predictions are made on the basis of physically plausible features. This ultimately improves the ease of adoption of machine learning techniques. The most used methods to explain machine learning model predictions (such as SHAP and other methods that rely on stochastic sample generation) suffer from high computational complexity and struggle to account for interactions between inputs. PERTURB-c addresses these issues by leveraging physical knowledge of the known spectral correlation. For visualisation of this analysis, we propose a heat-map-based representation which is better suited to large numbers of input features along a single dimension, and that is more intuitive to those who are already familiar with retrieval methods. Note that while we chose this exoplanet retrieval context to demonstrate our methodologies, the PERTURB-c framework is model agnostic and in a broader context has potential value across a plethora of adjacent regression problems.

Figures (18)

• Figure 1: Subset of key molecular abundances retrieved by neural network Clarke2025inprep on simulated WASP-107b-like planet for an idealised 100-bin spectrograph. Full retrieval (excluding plotting of corner plot) was performed in 0.2456 seconds using our machine learning accelerated hybrid retrieval strategy, and full corner plot can be seen in \ref{['fig:full_retrieval']}.
• Figure 2: The predictive performance of the neural network retrieval for $SO_2$. The red dashed line indicates perfect accuracy. Values on both axes are shown in log abundance of $SO_2$
• Figure 3: Ceteris paribus profile visualisation for the simplest 2D case. For a model $\hat{\mathbf{y}}=f(\mathbf{x})$ with input features (or wavelengths) $x_1$ and $x_2$, which predicts the abundance of one molecular species $\mathbf{y}$, the ceteris paribus profiles for both input features of a specific given observation $\mathbf{x}_n$ are visualised.
• Figure 4: Visualisation of the set of possible coalitions relevant for determining the relative contribution of wavelength $x_{\star 1}$ for a simplified observation with only $M=3$ wavelength bins. The coalitions are split into those including wavelength $x_{\star 1}$ and those excluding it ($\mathbf{X}^\prime$ and $\mathbf{X}^{\prime\prime}$, respectively).
• Figure 5: (Top) SHAP interpretability for the prediction of $SO_2$ in WASP-107b case study sample, shown as positive (red) and negative (blue) feature importance bars. (Middle) Shap is accurately assigning importance to the upper end of the observation, but beyond that, there is very little correlation between the SHAP interpretability and the observation datapoints (black) or the $SO_2$ contribution bands (red). The wavelength regions where $SO_2$ is the dominant molecule ought to correlate with positive feature importance given that the model has accurately captured the feature representation, but due to the effects of correlation, this has not been captured by the SHAP analysis. (Bottom) The PERTURB-c analysis however clearly highlights that the model is focusing on defining the leading edge of the $SO_2$ band at around 13µm. The high abundance prediction is given due to the combination of transmission in this band being high, and the presence of the band gap at 12µm (more on how to read PERTURB-c plots in \ref{['fig:example_reading']}).