Knobs and dials of retrieving JWST transmission spectra. II. Impacts of pipeline-level differences on retrieval posteriors

TL;DR

This study interrogates how JWST transmission spectra retrieved for WASP-39 b depend on data-reduction and input spectra. By generating random perturbations of SP-TW and comparing with two independently reduced spectra RU-23 and CA-24 using TauREx3, the authors identify three posterior archetypes: stable Gaussian posteriors for H$_2$O and CO$_2$, stable upper limits for CO and CH$_4$, and unstable heavy-tailed posteriors for SO$_2$, C$_2$H$_2$, and CH$_4$-related features, with the $p$-$T$ profile and Rp robust to perturbations. They demonstrate that independent reductions yield differing posteriors, challenging robust interpretation and underscoring the need for carefully chosen credible intervals (e.g., $CCI_{95}$) when reporting exoplanet atmospheric constraints. The results highlight the importance of accounting for pipeline- and data-reduction systematics in JWST-era atmospheric retrievals and provide guidance for transparent uncertainty reporting in future analyses.

Abstract

Since the launch of JWST, observations of exoplanetary atmospheres have seen a revolution in data quality. Given that atmospheric parameter inferences depend heavily on the underlying data, a re-evaluation of current methodologies is warranted to assess the reliability of these results. We investigate the impact of variations in input spectra on atmospheric retrievals for the hot Jupiter WASP-39 b using JWST transit data. Specifically, we analyse the reliability of parameter estimations from random perturbations of the underlying spectrum and their sensitivity to three transmission spectra derived from the same observational data. Using the NIRSpec PRISM observation from a single transit of WASP-39 b, we perform retrievals with the TauREx framework. As a baseline, we use a spectrum derived with the Eureka! data reduction pipeline. To evaluate retrieval reliability, we analyse posterior distributions under deviations from this spectrum. We simulate random noise by performing retrievals on scattered instances of this spectrum and compare them with retrievals based on existing spectra reduced from the same raw observation. Our analysis identifies three types of posterior distributions: (1) Stable, Gaussian distributions for species constrained across the entire spectrum (e.g., H2O, CO2); (2) Uniform posteriors with upper bounds for weakly constrained species (e.g., CO, CH4); and (3) Unstable, heavy-tailed posteriors for species constrained by minor spectrum features (e.g., SO2, C2H2). We find that other parameters, such as the planetary radius and p-T profile, are stable under spectral perturbations. Posterior distributions differ for retrievals on independently reduced transmission spectra from the same raw data, complicating interpretation, particularly for skewed distributions. Based on this, we advocate for careful assessment and selection of credible interval sizes to reflect this.

Figures (12)

• Figure 1: Comparison of transmission spectra used in this work. Data associated with the spectrum produced in this work (SP-TW) are shown in black, while data associated with rustamkulov_2023 (RU-23) and carter_2024 (CA-24) are shown in red and blue, respectively. (a) Transmission spectra, showing wavelength (in $\mu m$) on the x-axis and transit depth (in %) on the y-axis. (b) Residual distribution of RU-23 and CA-24, normalised to the transit depth uncertainty of SP-TW. For display purposes, the residuals are binned in steps of $0.5\,\sigma_\mathrm{SP-TW}$. (c) Empirical cumulative distribution functions (eCDF) for the residuals of RU-23 and CA-24. The values of a one-sample K-S test for a standard normal distribution are given in the legend of the figure. In (b) and (c), the black dashed line represents the PDF and CDF of $\mathcal{N}(0,1)$, respectively.
• Figure 2: Retrieval results of the fiducial model applied to SP-TW (the spectrum produced in our work), showing the posterior distributions of the molecular mixing ratios. Marginalised posterior distributions (main diagonal) show the parameter estimate median (points) and CCI$_{95}$ (error bar). The inset plot on the top right shows the median retrieved $p$-$T$ profile (dashed line) and CCI$_{95}$ (shaded region).
• Figure 3: Transmission spectrum with model fit solution from model tuning process. Both panels show wavelength (in µm) on the x-axis against transit depth (in %) on the y-axis, as well as the data points and error bars (grey) from the spectrum produced in this work (SP-TW). (Top) Median model solution (solid black line) and corresponding 95% CCI (shaded area). (Bottom) Contributions of individual molecular opacity sources (colour-coded by molecule) and the flat-opacity cloud deck (dashed grey line).
• Figure 4: Posterior distributions of select forward model parameters for atmospheric retrievals on scattered instances of SP-TW, showing inferred parameter values (x-axis) against weighted counts (y-axis) in all panels. The parameter posterior distributions are the vmr of CO2 (top left), CO (top right), and SO2 (bottom right). Marginalised posteriors and CCIs from the initial instance of SP-TW are shown in black, while the results from the scattered instances of SP-TW are colour-coded.
• Figure 5: Results of atmospheric retrieval performed on three transmission spectra derived from the same observation. Results achieved with SP-TW (the spectrum produced in our work), as well as with RU-23 and CA-24 are shown in black, red, and blue, respectively. (Left) The grid of smaller panels shows the marginalised posterior distributions of the molecular mixing ratios and cloud-top pressure (Right) Retrieved 4-point pressure temperature profiles, where dashed lines indicate the posterior median, and shaded area the corresponding CCI$_{95}$.