Standardized Images and Evaluation Metrics for Tomography

TL;DR

This work addresses the challenge of evaluating high-fidelity tomographic reconstructions, where traditional global metrics saturate and fail to distinguish subtle residual structure. It introduces a physics-grounded framework built on four standardized reference images—$Source$, $Detector$, $Ideal$, and $Realistic$—and a multi-domain diagnostic toolkit that includes $χ^2$ maps, image/sinogram difference maps, Structure–Contrast Index ($SCI$), spectral decomposition, and Region-of-Interest (RoI) analysis. The approach is demonstrated on SPECT data using the MLEM and RISE-1 reconstructions, showing that the new diagnostics reveal structured residuals and local improvements that global metrics miss, and that the framework generalizes to hardware phantoms and field data. The framework offers a reproducible, interpretable benchmark for reconstruction fidelity and has potential applicability to PET, CT, MRI, and other modalities, paving the way for more robust protocol optimization and algorithm development.

Abstract

Advances in instrumentation and computation have enabled increasingly sophisticated tomographic reconstruction methods. However, existing evaluation practices -- often based on simple phantoms and global image metrics -- are limited in their ability to differentiate among modern high-fidelity reconstructions. A standardized, quantitative framework capable of revealing subtle yet meaningful differences is therefore required. We introduce such a framework, built upon two core components. The first is a set of four standardized reference images -- Source, Detector, Ideal, and Realistic -- each derived from physical modeling and representing a distinct stage in the imaging and reconstruction chain. The second is a suite of diagnostic and quantitative tools that remain sensitive in regimes where conventional metrics (e.g., SSIM, PSNR, NMSE, CC) tend to saturate. These include pixel-wise $χ^2$ and difference maps, their quantitative characterization, spectral decomposition of intensity distributions, and Region-of-Interest (RoI)-based metrics. Application of this framework to MLEM and RISE-1 reconstructions using software phantoms demonstrates its ability to expose discrepancies that might elude detection by conventional global metrics. While developed in the context of SPECT, the methodology generalizes to other tomographic modalities, providing a reproducible, interpretable, and physically grounded basis for evaluating reconstruction fidelity in the high-performance regime.

Figures (22)

• Figure S1: Graphical representation of the specific activity of the two-dimensional "Source Images" of the Shepp-Logan phantom. Selected Regions of Interest (RoIs) are shown and will be discussed at Section \ref{['sec:roi_def']}.
• Figure S2: Source and Detector images from simulations of the modified Shepp–Logan phantom. The "Source Image" includes all emitted photons; the Detector image includes only detected ones. The rightmost panels show their difference; photon absorption within the phantom is the dominant effect.
• Figure S3: Simulated Shepp-Logan image of the “Source Image” and the “Ideal Image”, together with their difference.
• Figure S6: The top row depicts the "Ideal Sinogram" (a) and those resulting from different stages of MLEM (3, 9, 24, and 48 iterations) (b–e). The bottom row shows the "Ideal Image" (f) and the corresponding MLEM reconstructions (g–j).
• Figure S7: The first two rows on the left provide the difference and $\chi^2$ maps for the image and sinogram, for the MLEM reconstruction after 9 iterations. The last two columns on the right show the same maps for the case of MLEM after 48 iterations. The difference and $\chi^2$ maps reveal areas of the images that are not well-represented by the reconstruction.