The troublesome kernel -- On hallucinations, no free lunches and the accuracy-stability trade-off in inverse problems

TL;DR

The paper studies reliability of AI-based reconstruction in imaging inverse problems, formalizing the problem as $y = A x + e$ and showing that hallucinations, instability, and unpredictable performance arise from interaction with the forward operator's kernel. Through no-free-lunch theorems, it proves that overperforming reconstructions transfer details (hallucinations) and that an accuracy-stability trade-off is intrinsic, with instabilities not being rare under perturbations. It also shows that there may be model classes for which optimal reconstruction maps cannot be trained or even exist, highlighting fundamental limits of learning-based approaches. The results point to the need for robust, hybrid strategies and input-dependent architectures to mitigate hallucinations and instability in inverse-imaging problems, with broader implications for robustness and trustworthiness in AI.

Abstract

Methods inspired by Artificial Intelligence (AI) are starting to fundamentally change computational science and engineering through breakthrough performances on challenging problems. However, reliability and trustworthiness of such techniques is a major concern. In inverse problems in imaging, the focus of this paper, there is increasing empirical evidence that methods may suffer from hallucinations, i.e., false, but realistic-looking artifacts; instability, i.e., sensitivity to perturbations in the data; and unpredictable generalization, i.e., excellent performance on some images, but significant deterioration on others. This paper provides a theoretical foundation for these phenomena. We give mathematical explanations for how and when such effects arise in arbitrary reconstruction methods, with several of our results taking the form of `no free lunch' theorems. Specifically, we show that (i) methods that overperform on a single image can wrongly transfer details from one image to another, creating a hallucination, (ii) methods that overperform on two or more images can hallucinate or be unstable, (iii) optimizing the accuracy-stability trade-off is generally difficult, (iv) hallucinations and instabilities, if they occur, are not rare events, and may be encouraged by standard training, (v) it may be impossible to construct optimal reconstruction maps for certain problems. Our results trace these effects to the kernel of the forward operator whenever it is nontrivial, but also apply to the case when the forward operator is ill-conditioned. Based on these insights, our work aims to spur research into new ways to develop robust and reliable AI-based methods for inverse problems in imaging.

Figures (8)

• Figure 1: (AI-generated hallucinations in different imaging modalities). Trained NNs $\Psi \colon \mathbb{C}^{m} \to \mathbb{R}^{N}$ for different imaging modalities generate hallucinations, i.e., realistic-looking artifacts, when evaluated on test data. Note that $m$, $N$ and $A$ vary between the experiments. In the first three rows we consider trained NNs from the cited publications. In row four, we have trained a NN using data from qiao2021evaluation. For more information on the training procedure see § \ref{['s:figures-experiments']}.
• Figure 2: (Hallucinations due to detail transfer). A trained NN $\Psi \colon \mathbb{C}^m \rightarrow \mathbb{C}^N$ accurately recovers the detail image in the first column. But it hallucinates by incorrectly transferring the 'Mickey Mouse' detail $x_{\mathrm{mi}}$ in the first column when recovering the images in the second and third columns. The measurement matrix is a subsampled Fourier transform with $m/N = 20\%$, which models a MRI acquisition with $5$-fold acceleration. See § \ref{['s:figures-experiments']} for further details. Note that the the NN does not transfer the 'Thumb' detail $x_{\mathrm{th}}$. Theorem \ref{['thm:AIhal2']} explainns why this is the case.
• Figure 3: (Hallucinations and instabilities due to random noise) Two DL methods exhibit hallucinations and instabilities due to random noise. On the left, the DeepMRI-net SchCaH-17 reconstruction map is unstable to mean-zero Gaussian noise $v$. In this case, the NN hallucinates by removing a key image feature (see the red arrow). On the right, the AUTOMAP Bo-18 reconstruction map is unstable to Gaussian noise. The noise vector $e_0$ is drawn from a zero-mean Gaussian distribution, whereas the mean of the distribution, used to generate $e_1,e_2$ and $e_3$, is based on three worst-case noise vectors computed for AUTOMAP with respect to a different image. This makes the mean is image independent. The instability of the reconstruction map produces noticeable artefacts in all cases. The measurement matrix in these experiments is a subsampled Fourier transform with 33% (left) and 60% (right) subsampling, respectively. See § \ref{['s:figures-experiments']} for further information.
• Figure 4: (Hallucinations due to detail transfer). A NN $\Psi\colon \mathbb{R}^{m}\to \mathbb{R}^N$ is trained to accurately recover CT images and the image $x+x_{\mathrm{Det}}$ with the "SIAM" detail inserted, as shown in the first column. This causes the NN to incorrectly transfer the "SIAM" detail when reconstructing the detail-free image $x$ from measurements $Ax$, just as described by Theorem \ref{['thm:AIhal2']}. In this experiment the matrix $A$ is a subsampled Radon transform which samples 50 equally spaced angles, and the detail $x_{\mathrm{Det}}$ is designed such that $0 < \|Ax_{\mathrm{Det}}\|_{\ell^2} \ll \|Ax\|_{\ell^2}$.
• Figure 5: (The accuracy-stability trade-off) Three NNs are trained on the same dataset of images with noiseless, low-noise or high-noise measurements, respectively. The first has the highest accuracy, but the worst stability, while third has the lowest accuracy, but the best stability. The measurement matrix is a subsampled Fourier transform with $m/N \approx 17\%$. This experiment is based on one shown in genzel2020solving. See § \ref{['s:figures-experiments']} for further information.

Theorems & Definitions (18)

• Remark 1.1: Are these phenomena exclusive to AI-based methods?
• Theorem 4.1
• Remark 4.2
• Theorem 4.3
• Remark 4.4
• Theorem 4.5
• Theorem 4.6
• Theorem 4.7
• Definition 4.8: Optimal map
• Definition 4.9: Approximately optimal maps