A study of why we need to reassess full reference image quality assessment with medical images
Anna Breger, Ander Biguri, Malena Sabaté Landman, Ian Selby, Nicole Amberg, Elisabeth Brunner, Janek Gröhl, Sepideh Hatamikia, Clemens Karner, Lipeng Ning, Sören Dittmer, Michael Roberts, AIX-COVNET Collaboration, Carola-Bibiane Schönlieb
TL;DR
This paper interrogates the suitability of full-reference IQA measures, notably $PSNR$, $SSIM$, and $LPIPS$, for medical imaging across CT, MRI, X-ray, OCT, digital pathology, and photoacoustic modalities. It demonstrates through modality-specific examples that standard FR-IQA rankings often conflict with clinical and perceptual quality, especially in reconstruction, post-processing, and acquisition settings. The authors argue for task-informed FR-IQA, the development of task-specific metrics, and better reporting and sharing of data and evaluation frameworks to bridge the gap between algorithm development and clinical applicability. The work provides a structured synthesis of failure patterns and proposes guidelines and future research directions to improve reliability and explainability in medical image quality assessment. The overarching message is that FR-IQA should be used with caution in medical imaging, complemented by NR-IQA where appropriate, and guided by explicit task requirements and standardized reporting.
Abstract
Image quality assessment (IQA) is indispensable in clinical practice to ensure high standards, as well as in the development stage of machine learning algorithms that operate on medical images. The popular full reference (FR) IQA measures PSNR and SSIM are known and tested for working successfully in many natural imaging tasks, but discrepancies in medical scenarios have been reported in the literature, highlighting the gap between development and actual clinical application. Such inconsistencies are not surprising, as medical images have very different properties than natural images, and PSNR and SSIM have neither been targeted nor properly tested for medical images. This may cause unforeseen problems in clinical applications due to wrong judgment of novel methods. This paper provides a structured and comprehensive overview of examples where PSNR and SSIM prove to be unsuitable for the assessment of novel algorithms using different kinds of medical images, including real-world MRI, CT, OCT, X-Ray, digital pathology and photoacoustic imaging data. Therefore, improvement is urgently needed in particular in this era of AI to increase reliability and explainability in machine learning for medical imaging and beyond. Lastly, we will provide ideas for future research as well as suggesting guidelines for the usage of FR-IQA measures applied to medical images.