Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Decoupling Perception and Calibration: Label-Efficient Image Quality Assessment Framework

Xinyue Li, Zhichao Zhang, Zhiming Xu, Shubo Xu, Xiongkuo Min, Yitong Chen, Guangtao Zhai

TL;DR

This work tackles the MOS annotation bottleneck in multimodal LLM–based image quality assessment by decoupling perceptual quality from dataset-specific MOS calibration. It introduces LEAF, a two-stage framework where a frozen MLLM teacher provides dense point-wise judgments and confidence-weighted pairwise preferences to train a lightweight student without MOS labels in Stage 1, followed by Calibration Fine-Tuning on a small MOS subset to align with human judgments. Empirical results across AIGC and UGC benchmarks show strong MOS-aligned correlations under low annotation budgets, with substantial gains even at 10–30% MOS and robust performance across backbones and teachers. The approach offers a practical path toward scalable, on-device or large-scale IQA with limited labeled data, highlighting the practical value of separating perception from calibration in quality assessment.

Abstract

Recent multimodal large language models (MLLMs) have demonstrated strong capabilities in image quality assessment (IQA) tasks. However, adapting such large-scale models is computationally expensive and still relies on substantial Mean Opinion Score (MOS) annotations. We argue that for MLLM-based IQA, the core bottleneck lies not in the quality perception capacity of MLLMs, but in MOS scale calibration. Therefore, we propose LEAF, a Label-Efficient Image Quality Assessment Framework that distills perceptual quality priors from an MLLM teacher into a lightweight student regressor, enabling MOS calibration with minimal human supervision. Specifically, the teacher conducts dense supervision through point-wise judgments and pair-wise preferences, with an estimate of decision reliability. Guided by these signals, the student learns the teacher's quality perception patterns through joint distillation and is calibrated on a small MOS subset to align with human annotations. Experiments on both user-generated and AI-generated IQA benchmarks demonstrate that our method significantly reduces the need for human annotations while maintaining strong MOS-aligned correlations, making lightweight IQA practical under limited annotation budgets.

Decoupling Perception and Calibration: Label-Efficient Image Quality Assessment Framework

TL;DR

This work tackles the MOS annotation bottleneck in multimodal LLM–based image quality assessment by decoupling perceptual quality from dataset-specific MOS calibration. It introduces LEAF, a two-stage framework where a frozen MLLM teacher provides dense point-wise judgments and confidence-weighted pairwise preferences to train a lightweight student without MOS labels in Stage 1, followed by Calibration Fine-Tuning on a small MOS subset to align with human judgments. Empirical results across AIGC and UGC benchmarks show strong MOS-aligned correlations under low annotation budgets, with substantial gains even at 10–30% MOS and robust performance across backbones and teachers. The approach offers a practical path toward scalable, on-device or large-scale IQA with limited labeled data, highlighting the practical value of separating perception from calibration in quality assessment.

Abstract

Recent multimodal large language models (MLLMs) have demonstrated strong capabilities in image quality assessment (IQA) tasks. However, adapting such large-scale models is computationally expensive and still relies on substantial Mean Opinion Score (MOS) annotations. We argue that for MLLM-based IQA, the core bottleneck lies not in the quality perception capacity of MLLMs, but in MOS scale calibration. Therefore, we propose LEAF, a Label-Efficient Image Quality Assessment Framework that distills perceptual quality priors from an MLLM teacher into a lightweight student regressor, enabling MOS calibration with minimal human supervision. Specifically, the teacher conducts dense supervision through point-wise judgments and pair-wise preferences, with an estimate of decision reliability. Guided by these signals, the student learns the teacher's quality perception patterns through joint distillation and is calibrated on a small MOS subset to align with human annotations. Experiments on both user-generated and AI-generated IQA benchmarks demonstrate that our method significantly reduces the need for human annotations while maintaining strong MOS-aligned correlations, making lightweight IQA practical under limited annotation budgets.
Paper Structure (34 sections, 12 equations, 4 figures, 4 tables)

This paper contains 34 sections, 12 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Image Quality Assessment
  4. Handcrafted IQA
  5. Deep Learning--based IQA
  6. Multi-modal Model-based IQA
  7. Label-Free IQA Methods
  8. Classical NSS-based Model
  9. Deep Distribution Model
  10. Pretraining and Quality Representation
  11. Method
  12. Overview
  13. Teacher-Guided Distillation
  14. Calibration Fine-Tuning
  15. Teacher-Guided Distillation (TGD)
  16. ...and 19 more sections

Figures (4)

  • Figure 1: Using MLLMs for IQA is computationally expensive and annotation-intensive. MLLMs can perceive the key information of image quality. This work advocates decoupling quality perception from MOS calibration, thereby enabling efficient image quality assessment with minimal human supervision.
  • Figure 2: Direct MLLM scoring on AGIQA-3K exhibits strong monotonic correlation with MOS (SRCC=$0.816$) but suffers from pronounced scale bias and non-linear mapping (shown in (a)). Calibrating a Linear with only $10\%$ MOS substantially improves MOS alignment (PLCC from $0.824$ to $0.907$) and reduces residual bias (mean residual from $0.263$ to $0.006$).
  • Figure 3: Overview of LEAF. Label-efficient IQA framework: Stage-1 jointly distills point-wise judgments and pair-wise preferences from an MLLM teacher into a lightweight student. Stage-2 calibrates the student to human MOS using a small annotated subset.
  • Figure 4: Impact of MOS availability during MOS calibration on AGIQA-3K.