Simulation-based inference with neural posterior estimation applied to X-ray spectral fitting - III Deriving exact posteriors with dimension reduction and importance sampling

TL;DR

The paper demonstrates that simulation-based inference using neural posterior estimation, when combined with auto-encoder–based spectrum compression and likelihood-based importance sampling, can recover exact posterior distributions for complex X-ray spectral models at large speedups. The SIXSA pipeline iteratively refines the posterior through multi-round inferences with truncated proposals and employs a neural likelihood emulator to accelerate importance sampling, achieving posteriors statistically indistinguishable from nested sampling or BXA references. Across synthetic tests and XRISM-Resolve data, SIXSA outperforms traditional dimensionality reduction like PCA and remains robust to model complexity and degeneracies, enabling practical Bayesian X-ray spectral fitting on standard laptops. The approach preserves narrow spectral features, scales across instruments, and provides diagnostics to assess information retention and convergence, with an open-source SIXSA package for community use and further development.

Abstract

Simulation-based inference (SBI) with neural posterior estimation (NPE) provides rapid X-ray spectral fitting in both Gaussian and Poisson regimes by learning approximate parameter posteriors from simulations. We investigate auto-encoders for compressing high-resolution X-ray spectra, motivated by newAthena X-ray Integral Field Unit (X-IFU), and use likelihood-based importance sampling to refine NPE outputs. Our auto-encoder maps spectra to a low-dimensional latent space and is trained with a custom loss equal to the Cash statistic (C-stat) between simulated and reconstructed spectra. A neural density estimator is then trained on the latent representations. Both models are trained in multiple rounds: at each round, new simulations are drawn from a truncated proposal concentrated around the observation, improving efficiency as the proposal contracts. After NPE convergence, we apply likelihood-based importance sampling to correct the learned posterior. To assess information retention, we train a diagnostic network that predicts the original spectral parameters from the latent space, and we also train a network to learn the likelihood directly to accelerate importance sampling. On X-IFU-like simulations, the auto-encoder and multi-round NPE outperforms PCA and hand-crafted spectral summaries in accuracy and robustness. After importance sampling, the resulting posteriors are statistically indistinguishable from those obtained with nested sampling. On a standard laptop, the full pipeline (simulation, compression, inference, correction) delivers 10x speedups. We further demonstrate the approach on XRISM/Resolve and on lower-resolution NICER and XMM-Newton EPIC-pn data, confirming applicability across instruments and resolutions. Overall, NPE on compressed spectra paired with likelihood-based importance sampling offers an exact yet efficient alternative for Bayesian X-ray spectral fitting.

Figures (24)

• Figure 1: The SIXSA pipeline: The process begins with sampling parameters $\{\theta\}_i$ from a proposal distribution, followed by generating synthetic observations $\{x\}_i$, by passing these parameters to the model. These simulations are then compressed using various summarization techniques such as Principal Component Analysis (PCA), spectral summaries, or neural architectures like embedding networks and auto-encoders, yielding $\{S(x)\}_i$. These compressed spectra, along with their corresponding parameters, are used to train a neural density estimator (NDE). An optional parameter retriever neural network may be used to learn the mapping from the latent space back to the model parameters, aiding interpretation. For the observation, denoted as $x_{\rm obs}$, a truncated proposal network selectively focuses sampling on high-density regions of the parameter space. Likelihood-based importance sampling is ultimately performed when the training has converged. A likelihood emulator can also be integrated to approximate the true likelihood and accelerate importance sampling. This iterative process leads to the final posterior distribution $\text{Posterior}(\theta)$ for $x_{\rm obs}$.
• Figure 2: Left: The initial prior coverage of the targeted observation (black line), obtained with 20,000 spectra: twice the number used in the subsequent round. The range of the initial prior has been expanded to ensure a similar number of training sample spectra have count values both below and above the targeted observation (this forces the observation to be centered). Right: The prior coverage in round 2, based on 10,000 simulations. The allowed parameter space has shrunk considerably. This prior proposal could be used as input for running BXA, significantly speeding up the inference by reducing the parameter space that needs to be explored.
• Figure 3: Left: The histogram of the 24th of the 64 latent space dimensions for the auto-encoder training and test samples as derived from the initial prior shown in the left of Figure \ref{['fig:test-case-I-prior-coverage']}. The 24th latent dimension of the observation is materialized by the dashed vertical line. Right: The same histogram but derived from spectra simulated from the prior proposal generated after the single round of MRI (shown in the right of Figure \ref{['fig:test-case-I-prior-coverage']}). The shape of the histogram evolves towards a gaussian-like shape and is better centered around the observation.
• Figure 4: A random simulated spectrum including Poisson noise (blue), with its input model (black line). The spectrum reconstructed by the decoder part of the auto-encoder is shown in orange. The residuals, expressed in $\sigma$, between the simulated spectrum and both the input model and the reconstructed spectrum are shown in the bottom panels. The corresponding C-stat (without minimization) is listed for indication.
• Figure 5: The mapping by the Parameter_retriever between the model parameters and the latent space representations (64 dimensions) of 2,000 test spectra at the second round of inference. In each panel, the true parameter values are shown on the x-axis, and the predicted values on the y-axis. This result indicates that the autoencoder has successfully captured the relevant information from the simulated spectra. Moreover, the strong alignment between predicted and true values confirms that the observation is sensitive to all five model parameters. The target observation is marked with a square symbol for reference.