Task-Specific Normalization for Continual Learning of Blind Image Quality Models

TL;DR

This work tackles continual learning for blind image quality assessment by addressing subpopulation shifts and limited data scenarios. It introduces TSN-IQA, which freezes convolutional features while learning per-task BN parameters and per-task prediction heads, with a lightweight K-means gating mechanism to fuse predictions across tasks. The approach yields improved accuracy and a favorable plasticity-stability trade-off, demonstrated across six IQA datasets and multiple backbone configurations, while maintaining bounded memory footprint. The study also provides analyses of dataset relationships, generalization to unseen datasets, and computational considerations, highlighting practical applicability and scalability for online BIQA deployment.

Abstract

In this paper, we present a simple yet effective continual learning method for blind image quality assessment (BIQA) with improved quality prediction accuracy, plasticity-stability trade-off, and task-order/-length robustness. The key step in our approach is to freeze all convolution filters of a pre-trained deep neural network (DNN) for an explicit promise of stability, and learn task-specific normalization parameters for plasticity. We assign each new IQA dataset (i.e., task) a prediction head, and load the corresponding normalization parameters to produce a quality score. The final quality estimate is computed by black a weighted summation of predictions from all heads with a lightweight $K$-means gating mechanism. Extensive experiments on six IQA datasets demonstrate the advantages of the proposed method in comparison to previous training techniques for BIQA.

Figures (5)

• Figure 1: Illustration of continual learning for BIQA. The grey cylinders denote the inaccessibility of previous and future training data. During testing, we use all previous and the current test sets (indicated by dashed rectangles) to evaluate the stability and plasticity of the learned BIQA model.
• Figure 2: Illustration of task-specific BN. The parameters of all convolutions are frozen and shared across all tasks. A group of BN parameters is customized for each task.
• Figure 3: Overview of the inference process. The solid line indicates the pipeline of loading task-specific BN parameters learned for different tasks (denoted by different colors) and making the quality prediction using the corresponding prediction heads. The dotted line indicates the process of loading distortion-aware BN parameters and computing weightings for all prediction heads with the KG module.
• Figure 4: ${\mathrm{PSI}}_t$ as a function of the task index $t$.
• Figure 5: Learned common perceptual scale to embed images from the six IQA datasets. The bar charts of weightings and quality predictions are also presented alongside each image. The dataset in bold indicates the origin of the test image. The final quality prediction $\hat{q}(x)$ is shown in the subcaption. Zoom in for better distortion visibility.