Image Statistics Predict the Sensitivity of Perceptual Quality Metrics

TL;DR

This work directly links perceptual sensitivity of image-quality metrics to image statistics by estimating natural-image probabilities with a modern generative model (PixelCNN++). It finds that simple, probability-based factors—most notably the log-probability of the distorted image $\log(p(\tilde{\mathbf{x}}))$ and the original image’s standard deviation $\sigma(\mathbf{x})$—can predict metric sensitivity with correlations up to $\rho \approx 0.77$, approaching human-like performance in some evaluations. The authors validate the approach on natural-image psychophysics, reproduce several classical psychophysical trends (Weber law, CSF, masking), and show a non-parametric model can reach $\rho \approx 0.85$, with a simple interpretable form achieving $\rho \approx 0.77$. The results provide direct, quantitative support for a probabilistic, information-theoretic view of vision and offer a practical framework for predicting perceptual sensitivity from image statistics, despite limitations related to dataset scope and synthetic-stimulus generalization.

Abstract

Previously, Barlow and Attneave hypothesised a link between biological vision and information maximisation. Following Shannon, information was defined using the probability of natural images. Several physiological and psychophysical phenomena have been derived from principles like info-max, efficient coding, or optimal denoising. However, it remains unclear how this link is expressed in mathematical terms from image probability. Classical derivations were subjected to strong assumptions on the probability models and on the behaviour of the sensors. Moreover, the direct evaluation of the hypothesis was limited by the inability of classical image models to deliver accurate estimates of the probability. Here, we directly evaluate image probabilities using a generative model for natural images, and analyse how probability-related factors can be combined to predict the sensitivity of state-of-the-art subjective image quality metrics, a proxy for human perception. We use information theory and regression analysis to find a simple model that when combining just two probability-related factors achieves 0.77 correlation with subjective metrics. This probability-based model is validated in two ways: through direct comparison with the opinion of real observers in a subjective quality experiment, and by reproducing basic trends of classical psychophysical facts such as the Contrast Sensitivity Function, the Weber-law, and contrast masking.

Figures (11)

• Figure 1: The concept: visual sensitivity may be easy to predict from image probability. Different images are corrupted by uniform noise of the same energy (on the surface of a sphere around ${\bf x}$ of Euclidean radius $\epsilon=1$), with the same RMSE = 0.018 (in [0, 1] range). Due to visual maskingLegge80, noise is more visible, i.e. human sensitivity is bigger, for smooth images. This is consistent with the reported NLPD distance, and interestingly, sensitivity is also bigger for more probable images, $\log(p({\bf x}))$ via PixelCNN++.
• Figure 2: Conditional histograms for the dataset described in Sec.\ref{['noisy_images']}. x-axis is the probability factor and y-axis is the sensitivity (Eq. \ref{['eq:sensitivity']}) of the metric per row. Spearman correlation for each combination is in the title.
• Figure 3: Information Coefficient of Correlation (ICC) between the sensitivity of different perceptual distances and isolated probabilistic factors (left: 1 factor) and different pairs of probabilistic factors (right: 2 factors). The considered factors are explicitly listed at the axes of the matrix of pairs for the MS-SSIM. The meaning of the entries for the matrices of the other distances is the same and all share the same colourbar for ICC.
• Figure 4: The method of quadruples maloney2020measuring experimental setup used, where participants are asked which difference is less visible; the left pair or right pair?
• Figure 5: Human Response to Noise. Left most figure is Observer 1 vs the total points of the other 5 observers. For the 1F and 2F models, we plot the sensitivity $S$ of the distance against the total points of 5 observers. For the perceptual distances, we treat the distance as a participant and repeat the experiment. The Spearman correlation is in the title for each plot.