Analyzing Inverse Problems with Invertible Neural Networks

TL;DR

This work presents INNs as a robust framework for solving ill-posed inverse problems by modeling the full posterior $p(\mathbf{x}\,|\mathbf{y})$ through an invertible forward–inverse pair augmented with latent $\mathbf{z}$. By enforcing forward accuracy and distributional consistency with $p(\mathbf{y})p(\mathbf{z})$ via MMD, the method yields multi-modal, correlated posteriors and uncertainty quantification without restrictive parametric assumptions. Theoretical guarantees (asymptotic correctness) accompany empirical demonstrations on synthetic Gaussian mixtures and real-world domains including medical tissue spectroscopy and astrophysical spectral analysis, where INNs outperform ABC and cVAE baselines in posterior calibration and multimodality recovery. The approach offers a computationally efficient, flexible alternative for inverse problems, with potential for scaling to higher dimensions and integration of cycle-style losses in future work.

Abstract

In many tasks, in particular in natural science, the goal is to determine hidden system parameters from a set of measurements. Often, the forward process from parameter- to measurement-space is a well-defined function, whereas the inverse problem is ambiguous: one measurement may map to multiple different sets of parameters. In this setting, the posterior parameter distribution, conditioned on an input measurement, has to be determined. We argue that a particular class of neural networks is well suited for this task -- so-called Invertible Neural Networks (INNs). Although INNs are not new, they have, so far, received little attention in literature. While classical neural networks attempt to solve the ambiguous inverse problem directly, INNs are able to learn it jointly with the well-defined forward process, using additional latent output variables to capture the information otherwise lost. Given a specific measurement and sampled latent variables, the inverse pass of the INN provides a full distribution over parameter space. We verify experimentally, on artificial data and real-world problems from astrophysics and medicine, that INNs are a powerful analysis tool to find multi-modalities in parameter space, to uncover parameter correlations, and to identify unrecoverable parameters.

Figures (11)

• Figure 1: Abstract comparison of standard approach (left) and ours (right). The standard direct approach requires a discriminative, supervised loss ( SL) term between predicted and true $\mathbf{x}$, causing problems when $\mathbf{y} \rightarrow \mathbf{x}$ is ambiguous. Our network uses a supervised loss only for the well-defined forward process $\mathbf{x} \rightarrow \mathbf{y}$. Generated $\mathbf{x}$ are required to follow the prior $p(\mathbf{x})$ by an unsupervised loss ( USL), while the latent variables $\mathbf{z}$ are made to follow a Gaussian distribution, also by an unsupervised loss. See details in Section \ref{['sec:bidirectional-training']}.
• Figure 2: Viability of INN for a basic inverse problem. The task is to produce the correct (multi-modal) distribution of 2D points $\mathbf{x}$, given only the color label $\mathbf{y}^*$. When trained with all loss terms from Sec. \ref{['sec:bidirectional-training']}, the INN output matches ground truth almost exactly (2nd image). The ablations (3rd and 4th image) show that we need $\mathcal{L}_\mathbf{y}$ and $\mathcal{L}_\mathbf{z}$ to learn the conditioning correctly, whereas $\mathcal{L}_\mathbf{x}$ helps us remain faithful to the prior.
• Figure 3: Distribution over articulated poses $\mathbf{x}$, conditioned on the end point $\mathbf{y}^*$. The desired end point $\mathbf{y}^*$ is marked by a gray cross. A dotted line on the left represents the rail the arm is based on, and the faint colored lines indicate sampled arm configurations $\mathbf{x}$ taken from the true (ABC) or learned (INN, cVAE) posterior $p(\mathbf{x}\,|\,\mathbf{y}^*)$. The prior (right) is shown for reference. The actual end point of each sample may deviate slightly from the target $\mathbf{y}^*$; contour lines enclose the regions containing 97% of these end points. We emphasize the articulated arm with the highest estimated likelihood for illustrative purposes.
• Figure 4: Sampled posterior of 5 parameters for fixed $\mathbf{y}^*$ in medical application. For a fixed observation $\mathbf{y}^*$, we compare the estimated posteriors $p(\mathbf{x} \,|\, \mathbf{y}^*)$ of different methods. The bottom row also includes the point estimate (dashed green line). Ground truth values $\mathbf{x}^*$(dashed black line) and prior $p(\mathbf{x})$ over all data (gray area) are provided for reference.
• Figure 5: Astrophysics application. Properties $\mathbf{x}$ of star clusters in interstellar gas clouds are inferred from multispectral measurements $\mathbf{y}$. We train an INN on simulated data, and show the sampled posterior of 5 parameters for one $\mathbf{y}^*$ (colors as in Fig. \ref{['fig:dkfz_solution_example']}, second row). The peculiar shape of the prior is due to the dynamic nature of these simulations. We include this application as a real-world example for the INN's ability to recover multiple posterior modes, and strong correlations in $p(\mathbf{x} \,|\, \mathbf{y}^*)$, see details in appendix, Sec. \ref{['sec:astro_appendix']}.