A Formal Evaluation of PSNR as Quality Measurement Parameter for Image Segmentation Algorithms

TL;DR

The paper investigates whether PSNR is a reliable analytic metric for image segmentation quality. By comparing PSNR between original images and both good (threshold-based) and artificially degraded masks derived from ground-truth data, it employs Fisher's F-test and Welch's t-test to assess variance and mean differences. The results show that degraded masks yield higher PSNR values, indicating PSNR is not suitable as a general segmentation quality metric, though it may still aid in measuring image discrepancies or guiding edge-detection comparisons. The study cautions against using PSNR for segmentation evaluation and points to future work on multi-threshold assessment and the influence of label values.

Abstract

Quality evaluation of image segmentation algorithms are still subject of debate and research. Currently, there is no generic metric that could be applied to any algorithm reliably. This article contains an evaluation for the PSRN (Peak Signal-To-Noise Ratio) as a metric which has been used to evaluate threshold level selection as well as the number of thresholds in the case of multi-level segmentation. The results obtained in this study suggest that the PSNR is not an adequate quality measurement for segmentation algorithms.

Figures (8)

• Figure 1: Example of an image with foreground and background
• Figure 2: Gray level histogram with detected threshold $t=118$
• Figure 3: Resulting image after threshold based segmentation with $t=118$
• Figure 4: Example of an image from the database (a) and it's respective ground truth (b)
• Figure 5: Automatically filled ground truth image (a) and obtained binary mask (b)