Hypothesis Testing in Imaging Inverse Problems

TL;DR

The paper tackles semantic hypothesis testing in imaging inverse problems, where the observation $Y$ follows $Y \sim P(A x_\star)$ and hypotheses are formulated in natural language. It introduces a noise-injection measurement splitting scheme to enable replication-like testing on the same data, a CLIP-based test statistic in a shared image-text embedding space, and non-parametric testing with an e-value $E = \exp\{-t(Y_2)\}$ under Markov's inequality, with null/alternative $H_0: \mathbb{E}(E) \le 1$ vs $H_1: \mathbb{E}(E) > 1$. Theoretical analysis for a linearized VLM encoder and Gaussian/ exponential-family noise is provided, along with numerical experiments on image-based phenotyping that show strong Type I error control and improved power over zero-shot CLIP baselines. The approach leverages self-supervised reconstruction and foundation-model priors to reduce reliance on ground-truth data while enabling rigorous quantitative inference in scientific imaging pipelines.

Abstract

This paper proposes a framework for semantic hypothesis testing tailored to imaging inverse problems. Modern imaging methods struggle to support hypothesis testing, a core component of the scientific method that is essential for the rigorous interpretation of experiments and robust interfacing with decision-making processes. There are three main reasons why image-based hypothesis testing is challenging. First, the difficulty of using a single observation to simultaneously reconstruct an image, formulate hypotheses, and quantify their statistical significance. Second, the hypotheses encountered in imaging are mostly of semantic nature, rather than quantitative statements about pixel values. Third, it is challenging to control test error probabilities because the null and alternative distributions are often unknown. Our proposed approach addresses these difficulties by leveraging concepts from self-supervised computational imaging, vision-language models, and non-parametric hypothesis testing with e-values. We demonstrate our proposed framework through numerical experiments related to image-based phenotyping, where we achieve excellent power while robustly controlling Type I errors.

Figures (3)

• Figure 1: Overview of the proposed method. Given a noisy measurement $y = Ax_\star + \sqrt{\Sigma}\epsilon$, noise-injection generates two new conditionally independent measurements $y_1$ and $y_2$, mimicking experiment replication: $y_1$ is used for image reconstruction and formulation of the hypothesis test, $y_2$ resolves the tests non-parametrically in the shared representation of a VLM and reports a p-value.
• Figure 2: Example images from Oxford 102 Flower and ImageWoof experiments: true image $x_*$, measurements $y$ and $y_1$, supervised and self-supervised estimates $\hat{x}_{sup}(y_1)$ and $\hat{x}_{self}(y_1)$).
• Figure 3: Trade-off between reconstruction quality and statistical power for different values of $\tau$. The x-axis shows the PSNR of $y_1$, while the y-axis represents the statistical power obtained from $y_2$ at different significance levels. PSNR values are labeled according to the corresponding values of $\tau$.