From Global to Granular: Revealing IQA Model Performance via Correlation Surface

TL;DR

This paper tackles the inadequacy of global correlation metrics ($PLCC$, $SRCC$, $KRCC$) for IQA by introducing Granularity-Modulated Correlation (GMC), a framework that yields a 3D correlation surface over absolute quality ($MOS$) and quality differences ($|\Delta MOS|$). GMC combines a Granularity Modulator with a Distribution Regulator to compute localized correlations $\Gamma_k$ that are then embedded into a continuous surface via Latin Hypercube Sampling and Local Linear Kernel Regression, producing a robust global score $\text{GMC}_g$ through surface integration. The approach reveals nuanced model behaviors—such as high performance in high-MOS or fine-grained discrimination regimes—that traditional global metrics miss, and demonstrates robustness to non-uniform MOS distributions. GMC also supports scenario-specific model selection and integration, providing a practical diagnostic tool for deployment and dataset design with strong potential to guide future IQA benchmarks and algorithm development.

Abstract

Evaluation of Image Quality Assessment (IQA) models has long been dominated by global correlation metrics, such as Pearson Linear Correlation Coefficient (PLCC) and Spearman Rank-Order Correlation Coefficient (SRCC). While widely adopted, these metrics reduce performance to a single scalar, failing to capture how ranking consistency varies across the local quality spectrum. For example, two IQA models may achieve identical SRCC values, yet one ranks high-quality images (related to high Mean Opinion Score, MOS) more reliably, while the other better discriminates image pairs with small quality/MOS differences (related to $|Δ$MOS$|$). Such complementary behaviors are invisible under global metrics. Moreover, SRCC and PLCC are sensitive to test-sample quality distributions, yielding unstable comparisons across test sets. To address these limitations, we propose \textbf{Granularity-Modulated Correlation (GMC)}, which provides a structured, fine-grained analysis of IQA performance. GMC includes: (1) a \textbf{Granularity Modulator} that applies Gaussian-weighted correlations conditioned on absolute MOS values and pairwise MOS differences ($|Δ$MOS$|$) to examine local performance variations, and (2) a \textbf{Distribution Regulator} that regularizes correlations to mitigate biases from non-uniform quality distributions. The resulting \textbf{correlation surface} maps correlation values as a joint function of MOS and $|Δ$MOS$|$, providing a 3D representation of IQA performance. Experiments on standard benchmarks show that GMC reveals performance characteristics invisible to scalar metrics, offering a more informative and reliable paradigm for analyzing, comparing, and deploying IQA models. Codes are available at https://github.com/Dniaaa/GMC.

Figures (11)

• Figure 1: Complementary IQA behaviors along two coupled assessment dimensions on the SPAQ dataset fang2020perceptual. The overall performance of an IQA model is governed by its prediction accuracy relative to absolute quality (MOS) and its discrimination capability regarding pairwise differences ($|\Delta\text{MOS}|$). (a) Performance snapshots: Images from the SPAQ dataset are partitioned into four subsets based on the medians of MOS and $|\Delta\text{MOS}|$. PLCC scores on these quadrants reveal the distinct regimes where models excel. For instance, CLIP-IQA wang2023exploring is proficient in high-quality and fine-grained discrimination, while NIQE mittal2012making is more robust for severe degradations. (b) Correlation surfaces estimated by the proposed GMC provide a unified and comprehensive view, capturing how correlation performance (e.g., PLCC) evolves seamlessly across the joint MOS and $|\Delta\text{MOS}|$ space.
• Figure 2: Sensitivity of global correlation metrics (PLCC and SRCC) to shifts in quality score distributions. Two subsets are sampled from the PIPAL dataset with identical sample sizes but different MOS distributions. The first column shows the MOS distributions of the sampled subsets, while the second column reports the PLCC, SRCC, and the proposed global GMC score ($\mathrm{GMC}_g$) for PSNR and SSIM. Although PSNR and SSIM exhibit reversed performance rankings under PLCC and SRCC under distributional shifts, $\mathrm{GMC}_g$ remains stable and preserves consistent model ordering.
• Figure 3: Overview of the proposed 3D GMC performance surface for fine-grained IQA evaluation. The framework consists of a Granularity Modulator for analyzing local performance variations conditioned on MOS and $|\Delta$MOS$|$, and a Distribution Regulator for mitigating biases from non-uniform quality distributions. The resulting correlation surface represents performance as a joint function of MOS and $|\Delta$MOS$|$.
• Figure 4: Density estimation at a target quality score $q_i$. Each sample $q_u$ contributes to the density at $q_i$ via a Gaussian kernel with sample-dependent standard deviation.
• Figure 5: Visualization of correlation surfaces generated by different IQA models. The "$\text{GMC}$" response (vertical axis) is shown as a function of the "MOS" (left horizontal axis) and "$|\Delta\text{MOS}|$" (right horizontal axis).