Robustness of Neural Ratio and Posterior Estimators to Distributional Shifts for Population-Level Dark Matter Analysis in Strong Gravitational Lensing

TL;DR

This work investigates how distributional shifts between training simulations and real data affect neural ratio estimators (NREs) and sequential neural posterior estimators (SNPEs) in inferring the population-level dark matter subhalo mass function from strong gravitational lensing. By building two distinct likelihood-free pipelines—NRE and SNPE—and subjecting them to controlled nuisance-parameter shifts, the study demonstrates that both methods lose reliability under even modest out-of-distribution conditions, with biases that intensify as more lenses are combined. The SNPE approach, equipped with hierarchical inference, can mitigate some misspecifications within the test distribution, but substantial biases remain for larger shifts, particularly in background source morphologies. The results emphasize the need for careful validation, domain adaptation, and robust calibration when applying these methods to real astrophysical data, where true distributions are never perfectly known.

Abstract

We investigate the robustness of Neural Ratio Estimators (NREs) and Neural Posterior Estimators (NPEs) to distributional shifts in the context of measuring the abundance of dark matter subhalos using strong gravitational lensing data. While these data-driven inference frameworks can be accurate on test data from the same distribution as the training sets, in real applications, it is expected that simulated training data and true observational data will differ in their distributions. We explore the behavior of a trained NRE and trained sequential NPEs to estimate the population-level parameters of dark matter subhalos from a large sample of images of strongly lensed galaxies with test data presenting distributional shifts within and beyond the bounds of the training distribution in the nuisance parameters (e.g., the background source morphology). While our results show that NREs and NPEs perform well when tested perfectly in distribution, they exhibit significant biases when confronted with slight deviations from the examples seen in the training distribution. This indicates the necessity for caution when applying NREs and NPEs to real astrophysical data, where high-dimensional underlying distributions are not perfectly known.

Figures (8)

• Figure 1: From left to right: a convergence map $\kappa_{\rm SIE}$ created with an SIE profile, the convergence map of a sample of dark matter halos $\kappa_{\rm NFW}$, a realization of the source light, the lensed image without PSF convolution and noise, and the lensed image with PSF convolution and noise, generated with the provided kappa maps and source light.
• Figure 2: Sample of 40 random strong lenses, generated from the training data distribution for the NRE. The lens systems include a wide variety of closed Einstein rings, doubly imaged sources, and quadruply imaged sources. Such diversity is also seen in real observed lens systems.
• Figure 3: Sample of 40 random strong lenses generated from the training data distribution of the SNPE. The lens systems include a wide variety of closed Einstein rings, doubly imaged sources, and quadruply imaged sources.
• Figure 4: The evaluation of the NRE is conducted on lenses drawn from the training distribution and from distributions with minor variations in the parameters, following Section \ref{['sec:tests']}. All evaluation datasets were generated with the ground truth parameters set at $f_{\rm sub} = 0.05$ and $\beta = -0.9$, which are in accordance with $\Lambda$CDM predictions and are marked by a cyan star. The modifications to the parameter distributions for the individual plots are detailed in Table \ref{['tab:mods']}.
• Figure 5: Evaluation of the SNPEs on lenses drawn from the training distribution and on variations in the underlying parameter distributions, following the tests described in Section \ref{['sec:tests']}. We modify the Einstein radius distribution, the subhalo mass profile, and the source profiles. Details can be found in Table \ref{['tab:mods_SNPE_tests']}. The red contours always show the SNPE results for inference on the training data distribution.